[GitHub] spark pull request: [SPARK-4057] Use -agentlib instead of -Xdebug ...

2014-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2904#issuecomment-60196011 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22063/consoleFull) for PR 2904 at commit

[GitHub] spark pull request: [WIP][SPARK-3795] Heuristics for dynamically s...

2014-10-23 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/2746#issuecomment-60196383 @sryza just so I understand. I tell YARN I want 10 executors to be pending. Then say YARN grants me two executors. Does it internally decrement the pending number to 8

[GitHub] spark pull request: [SPARK-4055][MLlib] Inconsistent spelling 'MLl...

2014-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2903#issuecomment-60196521 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22064/consoleFull) for PR 2903 at commit

[GitHub] spark pull request: [SPARK-3988][SQL] add public API for date type

2014-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2901#issuecomment-60196729 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22058/consoleFull) for PR 2901 at commit

[GitHub] spark pull request: [SPARK-3988][SQL] add public API for date type

2014-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2901#issuecomment-60196733 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [BUILD] Fixed resolver for scalastyle plugin a...

2014-10-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2877 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-4058] [PySpark] Log file name is hard c...

2014-10-23 Thread sarutak
GitHub user sarutak opened a pull request: https://github.com/apache/spark/pull/2905 [SPARK-4058] [PySpark] Log file name is hard coded even though there is a variable '$LOG_FILE ' In a script 'python/run-tests', log file name is represented by a variable 'LOG_FILE' and it is used

[GitHub] spark pull request: [WIP][SPARK-3795] Heuristics for dynamically s...

2014-10-23 Thread sryza
Github user sryza commented on the pull request: https://github.com/apache/spark/pull/2746#issuecomment-60197171 So yeah it internally decrements the pending number to 8. The app can and is expected to infer YARN has decremented the counter. Maybe TMI, but for getting a grasp on

[GitHub] spark pull request: [SPARK-3988][SQL] add public API for date type

2014-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2901#issuecomment-60197221 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3988][SQL] add public API for date type

2014-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2901#issuecomment-60197217 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22059/consoleFull) for PR 2901 at commit

[GitHub] spark pull request: [SPARK-4058] [PySpark] Log file name is hard c...

2014-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2905#issuecomment-60197430 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22065/consoleFull) for PR 2905 at commit

[GitHub] spark pull request: SPARK-2621. Update task InputMetrics increment...

2014-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2087#issuecomment-60198186 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22061/consoleFull) for PR 2087 at commit

[GitHub] spark pull request: SPARK-2621. Update task InputMetrics increment...

2014-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2087#issuecomment-60198189 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [Spark-4041][SQL]attributes names in table sca...

2014-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2884#issuecomment-60198477 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/411/consoleFull) for PR 2884 at commit

[GitHub] spark pull request: [SPARK-4055][MLlib] Inconsistent spelling 'MLl...

2014-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2903#issuecomment-60198488 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22062/consoleFull) for PR 2903 at commit

[GitHub] spark pull request: [WIP][SPARK-3795] Heuristics for dynamically s...

2014-10-23 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/2746#issuecomment-60198472 Yep - that's exactly what I was wondering about. If YARN doesn't expose the internal counter it seems like there is a race (actually even if it does expose it,

[GitHub] spark pull request: [SPARK-3677] [BUILD] [YARN] pom.xml and SparkB...

2014-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2520#issuecomment-60198448 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22066/consoleFull) for PR 2520 at commit

[GitHub] spark pull request: [SPARK-4055][MLlib] Inconsistent spelling 'MLl...

2014-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2903#issuecomment-60198490 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4032] Deprecate YARN alpha support in S...

2014-10-23 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/2878#issuecomment-60198553 Yeah, maybe the output here is too noisy for it to be noticable. I agree having something in the Client itself is a good idea. --- If your project is set up for it,

[GitHub] spark pull request: [SPARK-4037][SQL] Removes the SessionState ins...

2014-10-23 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2887#discussion_r19261442 --- Diff: sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2Suite.scala --- @@ -150,10 +150,12 @@ class

[GitHub] spark pull request: [Spark-4041][SQL]attributes names in table sca...

2014-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2884#issuecomment-60198722 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/411/consoleFull) for PR 2884 at commit

[GitHub] spark pull request: [ SPARK-1812] Adjust build system and tests to...

2014-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2615#issuecomment-60200516 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [ SPARK-1812] Adjust build system and tests to...

2014-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2615#issuecomment-60200512 **[Tests timed out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22060/consoleFull)** for PR 2615 at commit

[GitHub] spark pull request: [SPARK-4026][Streaming] Write ahead log manage...

2014-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2882#issuecomment-60201254 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22067/consoleFull) for PR 2882 at commit

[GitHub] spark pull request: [SPARK-4019] [SPARK-3740] Fix MapStatus compre...

2014-10-23 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/2866#issuecomment-60201281 @JoshRosen thanks for doing this. There is a chance that a normal hashset is much slower than a bitmap. Can you test that? It might make a lot more sense to use an

[GitHub] spark pull request: [SPARK-4055][MLlib] Inconsistent spelling 'MLl...

2014-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2903#issuecomment-60201398 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22064/consoleFull) for PR 2903 at commit

[GitHub] spark pull request: [SPARK-4055][MLlib] Inconsistent spelling 'MLl...

2014-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2903#issuecomment-60201403 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [Spark-4041][SQL]attributes names in table sca...

2014-10-23 Thread scwf
Github user scwf commented on the pull request: https://github.com/apache/spark/pull/2884#issuecomment-60201488 test failed due to streaming compile error, can you retest this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-4058] [PySpark] Log file name is hard c...

2014-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2905#issuecomment-60202537 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22065/consoleFull) for PR 2905 at commit

[GitHub] spark pull request: [SPARK-4058] [PySpark] Log file name is hard c...

2014-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2905#issuecomment-60202542 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3677] [BUILD] [YARN] pom.xml and SparkB...

2014-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2520#issuecomment-60203866 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22066/consoleFull) for PR 2520 at commit

[GitHub] spark pull request: [SPARK-3677] [BUILD] [YARN] pom.xml and SparkB...

2014-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2520#issuecomment-60203876 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4026][Streaming] Write ahead log manage...

2014-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2882#issuecomment-60208450 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22067/consoleFull) for PR 2882 at commit

[GitHub] spark pull request: [SPARK-4026][Streaming] Write ahead log manage...

2014-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2882#issuecomment-60208456 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [ SPARK-1812] Adjust build system and tests to...

2014-10-23 Thread ScrapCodes
Github user ScrapCodes commented on a diff in the pull request: https://github.com/apache/spark/pull/2615#discussion_r19265089 --- Diff: dev/change-version-to-2.10.sh --- @@ -0,0 +1,20 @@ +#!/usr/bin/env bash + +# +# Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request: [Spark-4041][SQL]attributes names in table sca...

2014-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2884#issuecomment-60209325 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/412/consoleFull) for PR 2884 at commit

[GitHub] spark pull request: [Spark-4041][SQL]attributes names in table sca...

2014-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2884#issuecomment-60209693 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/412/consoleFull) for PR 2884 at commit

[GitHub] spark pull request: [SPARK-4026][Streaming] Write ahead log manage...

2014-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2882#issuecomment-60213859 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22068/consoleFull) for PR 2882 at commit

[GitHub] spark pull request: [SPARK-2429] [MLlib] Hierarchical Implementati...

2014-10-23 Thread yu-iskw
GitHub user yu-iskw opened a pull request: https://github.com/apache/spark/pull/2906 [SPARK-2429] [MLlib] Hierarchical Implementation of KMeans I want to add a divisive hierarchical clustering algorithm implementation to MLlib. I don't support distance metrics other Euclidean

[GitHub] spark pull request: [SPARK-4026][Streaming] Write ahead log manage...

2014-10-23 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/2882#issuecomment-60213891 @JoshRosen @harishreedharan addressed all your comments, and also simplified the writer code I did some further cleanups, and also added two new unit tests that

[GitHub] spark pull request: [SPARK-2429] [MLlib] Hierarchical Implementati...

2014-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2906#issuecomment-60214129 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-3812] [BUILD] Adapt maven build to publ...

2014-10-23 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/2673#issuecomment-60214941 @ScrapCodes @pwendell This patch will cause `maven-assembly-plugin` error: ` ./make-distribution.sh -Dhadoop.version=2.3.0-cdh5.0.1 -Dyarn.version=2.3.0-cdh5.0.1

[GitHub] spark pull request: [SPARK-4057] Use -agentlib instead of -Xdebug ...

2014-10-23 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/2904#issuecomment-60215591 +1 I can confirm that `-Xdebug` went away in Java 5 I think and this is the modern invocation of the debugger. --- If your project is set up for it, you can reply to

[GitHub] spark pull request: [SPARK-2429] [MLlib] Hierarchical Implementati...

2014-10-23 Thread rnowling
Github user rnowling commented on a diff in the pull request: https://github.com/apache/spark/pull/2906#discussion_r19267797 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/HierarchicalClusteringModel.scala --- @@ -0,0 +1,79 @@ +/* + * Licensed to the

[GitHub] spark pull request: [Spark-4041][SQL]attributes names in table sca...

2014-10-23 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/2884#issuecomment-60216081 Hm, the failure was caused by a known Jenkins configuration issue. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-2429] [MLlib] Hierarchical Implementati...

2014-10-23 Thread rnowling
Github user rnowling commented on a diff in the pull request: https://github.com/apache/spark/pull/2906#discussion_r19267891 --- Diff: python/pyspark/mllib/clustering.py --- @@ -91,6 +99,58 @@ def train(cls, rdd, k, maxIterations=100, runs=1, initializationMode=k-means||

[GitHub] spark pull request: [SPARK-3954][Streaming] promote the speed of c...

2014-10-23 Thread surq
Github user surq commented on the pull request: https://github.com/apache/spark/pull/2811#issuecomment-60217711 Does someone take notice of this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: MLlib, exposing special rdd functions to the p...

2014-10-23 Thread numbnut
GitHub user numbnut opened a pull request: https://github.com/apache/spark/pull/2907 MLlib, exposing special rdd functions to the public You can merge this pull request into a Git repository by running: $ git pull https://github.com/numbnut/spark master Alternatively you can

[GitHub] spark pull request: [Spark-4060] [MLlib] exposing special rdd func...

2014-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2907#issuecomment-60218336 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-4061] We cannot use EOL character in th...

2014-10-23 Thread sarutak
GitHub user sarutak opened a pull request: https://github.com/apache/spark/pull/2908 [SPARK-4061] We cannot use EOL character in the operand of LIKE predicate. We cannot use EOL character like \n or \r in the operand of LIKE predicate. So following condition is never true.

[GitHub] spark pull request: [SPARK-4061] We cannot use EOL character in th...

2014-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2908#issuecomment-60218997 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22069/consoleFull) for PR 2908 at commit

[GitHub] spark pull request: [SPARK-4026][Streaming] Write ahead log manage...

2014-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2882#issuecomment-60219275 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4026][Streaming] Write ahead log manage...

2014-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2882#issuecomment-60219269 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22068/consoleFull) for PR 2882 at commit

[GitHub] spark pull request: [SPARK-4006] In long running contexts, we enco...

2014-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2886#issuecomment-60220879 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22070/consoleFull) for PR 2886 at commit

[GitHub] spark pull request: [SPARK-3900][YARN] ApplicationMaster's shutdow...

2014-10-23 Thread sarutak
Github user sarutak commented on the pull request: https://github.com/apache/spark/pull/2755#issuecomment-60220899 /CC @tgravescs --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: specify unidocGenjavadocVersion of 0.8

2014-10-23 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/2893#issuecomment-60221337 This is for SPARK-3359. LGTM, thank you. This gets past some errors, and turns up more, which I'll comment on in the JIRA. But this is a step forward. --- If your

[GitHub] spark pull request: [SPARK-4006] In long running contexts, we enco...

2014-10-23 Thread tsliwowicz
Github user tsliwowicz commented on the pull request: https://github.com/apache/spark/pull/2886#issuecomment-60221362 @andrewor14 - thanks for the comments. I believe I fixed them all. Let me know! --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-4006] In long running contexts, we enco...

2014-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2886#issuecomment-60221739 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22072/consoleFull) for PR 2886 at commit

[GitHub] spark pull request: [SPARK-4006] In long running contexts, we enco...

2014-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2886#issuecomment-60222452 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4006] In long running contexts, we enco...

2014-10-23 Thread tsliwowicz
Github user tsliwowicz commented on the pull request: https://github.com/apache/spark/pull/2886#issuecomment-60222733 the failure seems technical (not related to my fix), I think. Local maven build works fine for me. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-4061] We cannot use EOL character in th...

2014-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2908#issuecomment-60223794 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4061] We cannot use EOL character in th...

2014-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2908#issuecomment-60223791 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22069/consoleFull) for PR 2908 at commit

[GitHub] spark pull request: [SPARK-4006] In long running contexts, we enco...

2014-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2886#issuecomment-60227754 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22070/consoleFull) for PR 2886 at commit

[GitHub] spark pull request: [SPARK-4006] In long running contexts, we enco...

2014-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2886#issuecomment-60227762 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4006] In long running contexts, we enco...

2014-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2886#issuecomment-60228517 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4006] In long running contexts, we enco...

2014-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2886#issuecomment-60228510 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22072/consoleFull) for PR 2886 at commit

[GitHub] spark pull request: Clarify docstring for Pyspark's foreachPartiti...

2014-10-23 Thread tdhopper
Github user tdhopper commented on the pull request: https://github.com/apache/spark/pull/2895#issuecomment-60234425 Oh. Now that I look at master, @JoshRosen, I see that it's already been fixed by @davis

[GitHub] spark pull request: [SPARK-732][SPARK-3628][CORE][RESUBMIT] make i...

2014-10-23 Thread CodingCat
Github user CodingCat commented on the pull request: https://github.com/apache/spark/pull/2524#issuecomment-60237457 ping --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request: [SPARK-3900][YARN] ApplicationMaster's shutdow...

2014-10-23 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/2755#issuecomment-60237621 Jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-3900][YARN] ApplicationMaster's shutdow...

2014-10-23 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/2755#issuecomment-60237859 Changes look good. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-3904] [SQL] add constant objectinspecto...

2014-10-23 Thread chenghao-intel
Github user chenghao-intel commented on the pull request: https://github.com/apache/spark/pull/2762#issuecomment-60242669 Thank you @liancheng, I've updated the code accordingly. You're right, the conversion is not so efficient, probably we need to add some Expression nodes

[GitHub] spark pull request: [SPARK-3900][YARN] ApplicationMaster's shutdow...

2014-10-23 Thread sarutak
Github user sarutak commented on the pull request: https://github.com/apache/spark/pull/2755#issuecomment-60250844 Hm... test wouldn't start... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-4061][SQL] We cannot use EOL character ...

2014-10-23 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/2908#issuecomment-60256774 Good catch! Would you mind to add a unit test for this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-4061][SQL] We cannot use EOL character ...

2014-10-23 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2908#discussion_r19284542 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringOperations.scala --- @@ -103,21 +103,21 @@ case class Like(left:

[GitHub] spark pull request: [SPARK-2663] [SQL] Support the Grouping Set

2014-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1567#issuecomment-60257141 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22073/consoleFull) for PR 1567 at commit

[GitHub] spark pull request: SPARK-3359 [DOCS] sbt/sbt unidoc doesn't work ...

2014-10-23 Thread srowen
GitHub user srowen opened a pull request: https://github.com/apache/spark/pull/2909 SPARK-3359 [DOCS] sbt/sbt unidoc doesn't work with Java 8 This follows https://github.com/apache/spark/pull/2893 , but does not completely fix SPARK-3359 either. This fixes minor scaladoc/javadoc

[GitHub] spark pull request: SPARK-3359 [DOCS] sbt/sbt unidoc doesn't work ...

2014-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2909#issuecomment-60262260 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22074/consoleFull) for PR 2909 at commit

[GitHub] spark pull request: [SPARK-4055][MLlib] Inconsistent spelling 'MLl...

2014-10-23 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/2903#issuecomment-60265674 LGTM. Merged into master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-2429] [MLlib] Hierarchical Implementati...

2014-10-23 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/2906#discussion_r19288355 --- Diff: docs/mllib-clustering.md --- @@ -153,3 +157,152 @@ provided in the [Self-Contained Applications](quick-start.html#self-contained-ap section of

[GitHub] spark pull request: [SPARK-2663] [SQL] Support the Grouping Set

2014-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1567#issuecomment-60266002 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22073/consoleFull) for PR 1567 at commit

[GitHub] spark pull request: [SPARK-2663] [SQL] Support the Grouping Set

2014-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1567#issuecomment-60266012 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4052][SQL] Use scala.collection.Map for...

2014-10-23 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2899#discussion_r19288406 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala --- @@ -18,6 +18,7 @@ package

[GitHub] spark pull request: [SPARK-2429] [MLlib] Hierarchical Implementati...

2014-10-23 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/2906#discussion_r19288604 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/HierarchicalClustering.scala --- @@ -0,0 +1,549 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-2429] [MLlib] Hierarchical Implementati...

2014-10-23 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/2906#discussion_r19288634 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/HierarchicalClustering.scala --- @@ -0,0 +1,549 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-2429] [MLlib] Hierarchical Implementati...

2014-10-23 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/2906#discussion_r19288713 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/HierarchicalClustering.scala --- @@ -0,0 +1,549 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-2429] [MLlib] Hierarchical Implementati...

2014-10-23 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/2906#discussion_r19288686 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/HierarchicalClustering.scala --- @@ -0,0 +1,549 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-4055][MLlib] Inconsistent spelling 'MLl...

2014-10-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2903 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-2429] [MLlib] Hierarchical Implementati...

2014-10-23 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/2906#discussion_r19288793 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/HierarchicalClustering.scala --- @@ -0,0 +1,549 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-2429] [MLlib] Hierarchical Implementati...

2014-10-23 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/2906#discussion_r19288871 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/HierarchicalClustering.scala --- @@ -0,0 +1,549 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-2429] [MLlib] Hierarchical Implementati...

2014-10-23 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/2906#discussion_r19289138 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/HierarchicalClustering.scala --- @@ -0,0 +1,549 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-2429] [MLlib] Hierarchical Implementati...

2014-10-23 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/2906#discussion_r19289245 --- Diff: mllib/src/test/java/org/apache/spark/mllib/clustering/JavaHierarchicalClusteringSuite.java --- @@ -0,0 +1,78 @@ +/* + * Licensed to the

[GitHub] spark pull request: [SPARK-2429] [MLlib] Hierarchical Implementati...

2014-10-23 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/2906#issuecomment-60268305 I just gave this a quick read-through, and the structure makes sense. I left several small comments. I see the chunks of logic I would expect, but did not evaluate it in

[GitHub] spark pull request: [SPARK-3911] [SQL] HiveSimpleUdf can not be op...

2014-10-23 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/2771#discussion_r19289432 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/QueryTest.scala --- @@ -74,4 +76,30 @@ class QueryTest extends FunSuite { .stripMargin)

[GitHub] spark pull request: [SPARK-4026][Streaming] Write ahead log manage...

2014-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2882#issuecomment-60274484 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22075/consoleFull) for PR 2882 at commit

[GitHub] spark pull request: SPARK-3883 SSL support for HttpServer and Akka

2014-10-23 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/2739#issuecomment-60275209 Hi @jacek-lewandowski, Now that I finally noticed you built this on top of branch-1.1, some of the choices you made make a lot more sense. (I always assume people

[GitHub] spark pull request: SPARK-3359 [DOCS] sbt/sbt unidoc doesn't work ...

2014-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2909#issuecomment-60276342 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: SPARK-3359 [DOCS] sbt/sbt unidoc doesn't work ...

2014-10-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2909#issuecomment-60276331 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22074/consoleFull) for PR 2909 at commit

[GitHub] spark pull request: [SPARK-4065] Add check for IPython on Windows

2014-10-23 Thread msjgriffiths
GitHub user msjgriffiths opened a pull request: https://github.com/apache/spark/pull/2910 [SPARK-4065] Add check for IPython on Windows This issue employs logic similar to the bash launcher (pyspark) to check if IPTYHON=1, and if so launch ipython with options in IPYTHON_OPTS.

[GitHub] spark pull request: [SPARK-4065] Add check for IPython on Windows

2014-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2910#issuecomment-60278572 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-4006] In long running contexts, we enco...

2014-10-23 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/2886#discussion_r19294252 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManagerMasterActor.scala --- @@ -325,22 +325,23 @@ class BlockManagerMasterActor(val isLocal:

  1   2   3   4   5   >