[GitHub] incubator-spark pull request: Spark 1095 : Adding explicit return ...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/610#issuecomment-35360834 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please t

[GitHub] incubator-spark pull request: Spark 1095 : Adding explicit return ...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/610#issuecomment-35360835 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top-

[GitHub] incubator-spark pull request: For SPARK-1082, Use Curator for ZK i...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/611#issuecomment-35361023 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12757/ --- If you

[GitHub] incubator-spark pull request: For SPARK-1082, Use Curator for ZK i...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/611#issuecomment-35361022 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top

[GitHub] incubator-spark pull request: For SPARK-1082, Use Curator for ZK i...

2014-02-18 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/incubator-spark/pull/611#discussion_r9815215 explicit return type --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top-post you

[GitHub] incubator-spark pull request: For SPARK-1082, Use Curator for ZK i...

2014-02-18 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/incubator-spark/pull/611#discussion_r9815294 extra newline? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top-post your resp

[GitHub] incubator-spark pull request: For SPARK-1082, Use Curator for ZK i...

2014-02-18 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/incubator-spark/pull/611#discussion_r9815286 Let's not have a default value for this and throw an error if it's not defined. (I know that wasn't the original behavior, but I think this behavior was sorta an

[GitHub] incubator-spark pull request: For SPARK-1082, Use Curator for ZK i...

2014-02-18 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/incubator-spark/pull/611#discussion_r9815316 This suggests we may be able to do away with the actor-ness of this class -- I haven't given it sufficient thought, but I suspect it's the case. --- If your proj

[GitHub] incubator-spark pull request: For SPARK-1082, Use Curator for ZK i...

2014-02-18 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/incubator-spark/pull/611#discussion_r9815357 This statement doesn't really give us anything if we're not behind a lock. I think we should probably synchronize this method and notLeader and then these stateme

[GitHub] incubator-spark pull request: For SPARK-1082, Use Curator for ZK i...

2014-02-18 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/incubator-spark/pull/611#issuecomment-35361778 Thanks for doing this! Took a very superficial pass, will attempt a deeper one later. Have you tried running FaultToleranceTest? If you're not on a Linux box, i

[GitHub] incubator-spark pull request: Spark 1095 : Adding explicit return ...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/610#issuecomment-35362367 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12758/ --- If you

[GitHub] incubator-spark pull request: Spark 1095 : Adding explicit return ...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/610#issuecomment-35362366 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top

[GitHub] incubator-spark pull request: Spark 1095 : Adding explicit return ...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/610#issuecomment-35368102 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please t

[GitHub] incubator-spark pull request: Spark 1095 : Adding explicit return ...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/610#issuecomment-35368104 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top-

[GitHub] incubator-spark pull request: Spark 1095 : Adding explicit return ...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/610#issuecomment-35370102 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12759/ --- If you

[GitHub] incubator-spark pull request: Spark 1095 : Adding explicit return ...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/610#issuecomment-35370100 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top

[GitHub] incubator-spark pull request: Fix ExternalMap on case of key's has...

2014-02-18 Thread guojc
GitHub user guojc opened a pull request: https://github.com/apache/incubator-spark/pull/612 Fix ExternalMap on case of key's hashCode equal to Int.Maxvalue Currently StreamBuffer use hash value to compare between buffers. And it want emmit buffer with lowest HashValue from Prio

[GitHub] incubator-spark pull request: Fix ExternalMap on case of key's has...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/612#issuecomment-35372535 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To

[GitHub] incubator-spark pull request: Spark 1095 : Adding explicit return ...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/610#issuecomment-35373439 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please t

[GitHub] incubator-spark pull request: Spark 1095 : Adding explicit return ...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/610#issuecomment-35373442 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top-

[GitHub] incubator-spark pull request: Spark 1095 : Adding explicit return ...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/610#issuecomment-35375271 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top

[GitHub] incubator-spark pull request: Spark 1095 : Adding explicit return ...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/610#issuecomment-35375272 One or more automated tests failed Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12760/ ---

[GitHub] incubator-spark pull request: [SPARK-1094] Support MiMa for report...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/585#issuecomment-35375462 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please t

[GitHub] incubator-spark pull request: [SPARK-1094] Support MiMa for report...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/585#issuecomment-35375464 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top-

[GitHub] incubator-spark pull request: [SPARK-1094] Support MiMa for report...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/585#issuecomment-35375698 One or more automated tests failed Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12761/ ---

[GitHub] incubator-spark pull request: [SPARK-1094] Support MiMa for report...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/585#issuecomment-35375697 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top

[GitHub] incubator-spark pull request: [SPARK-1094] Support MiMa for report...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/585#issuecomment-35376471 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top-

[GitHub] incubator-spark pull request: [SPARK-1094] Support MiMa for report...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/585#issuecomment-35376470 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please t

[GitHub] incubator-spark pull request: [SPARK-1094] Support MiMa for report...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/585#issuecomment-35376802 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please t

[GitHub] incubator-spark pull request: [SPARK-1094] Support MiMa for report...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/585#issuecomment-35376803 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top-

[GitHub] incubator-spark pull request: [SPARK-1094] Support MiMa for report...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/585#issuecomment-35378476 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12762/ --- If you

[GitHub] incubator-spark pull request: [SPARK-1094] Support MiMa for report...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/585#issuecomment-35378475 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top

[GitHub] incubator-spark pull request: [SPARK-1094] Support MiMa for report...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/585#issuecomment-35379064 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top

[GitHub] incubator-spark pull request: [SPARK-1094] Support MiMa for report...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/585#issuecomment-35379065 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12763/ --- If you

IntelliJ inflates the CPU utilization when working on SparkIMain.scala

2014-02-18 Thread Nan Zhu
Hi, all Anyone else meets the same problem? When I work on SparkIMain.scala, my IntelliJ inflates the CPU utilization a lot, the top shows: PID COMMAND %CPU TIME #TH #WQ #PORT #MREG MEMRPRVT PURG CMPR VPRVT VSIZE PGRP PPID STATE 541 mdworker 1.3 00:00.07 5

[GitHub] incubator-spark pull request: Optimized imports

2014-02-18 Thread NirmalReddy
GitHub user NirmalReddy opened a pull request: https://github.com/apache/incubator-spark/pull/613 Optimized imports Optimized imports and arranged according to scala style guide @ https://cwiki.apache.org/confluence/display/SPARK/Spark+Code+Style+Guide#SparkCodeStyleGuide-Import

[GitHub] incubator-spark pull request: Optimized imports

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/613#issuecomment-35399134 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please t

[GitHub] incubator-spark pull request: Optimized imports

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/613#issuecomment-35399136 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top-

Signal/Noise Ratio

2014-02-18 Thread Heiko Braun
Wouldn't it be better to move the github messages to a dedicated email list? Regards, Heiko

[GitHub] incubator-spark pull request: [SPARK-1089] fix the problem in 0.9 ...

2014-02-18 Thread CodingCat
GitHub user CodingCat opened a pull request: https://github.com/apache/incubator-spark/pull/614 [SPARK-1089] fix the problem in 0.9 that ADD_JARS value was not recognized https://spark-project.atlassian.net/browse/SPARK-1089 load jar in process() and work around for scala i

[GitHub] incubator-spark pull request: Optimized imports

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/613#issuecomment-35402590 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top

[GitHub] incubator-spark pull request: Optimized imports

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/613#issuecomment-35402592 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12764/ --- If you

Re: Signal/Noise Ratio

2014-02-18 Thread Pascal Voitot Dev
+1 On Tue, Feb 18, 2014 at 5:24 PM, Heiko Braun wrote: > > > Wouldn't it be better to move the github messages to a dedicated email > list? > > Regards, Heiko >

Re: Signal/Noise Ratio

2014-02-18 Thread Surendranauth Hiraman
github or specifically jenkins? I find the actual github comments threads very useful. -Suren On Tue, Feb 18, 2014 at 11:26 AM, Pascal Voitot Dev < pascal.voitot@gmail.com> wrote: > +1 > > > On Tue, Feb 18, 2014 at 5:24 PM, Heiko Braun >wrote: > > > > > > > Wouldn't it be better to move t

[GitHub] incubator-spark pull request: [SPARK-1089] fix the problem in 0.9 ...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/614#issuecomment-35402826 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To

Re: Signal/Noise Ratio

2014-02-18 Thread Heiko Braun
AFAIKT both end up in the PR messages that are send to the dev list. On 18 Feb 2014, at 17:51, Surendranauth Hiraman wrote: > github or specifically jenkins? > > I find the actual github comments threads very useful. > > -Suren > > > On Tue, Feb 18, 2014 at 11:26 AM, Pascal Voitot Dev < > pa

Re: Signal/Noise Ratio

2014-02-18 Thread Michael Ernest
+1 On Tue, Feb 18, 2014 at 8:24 AM, Heiko Braun wrote: > > > Wouldn't it be better to move the github messages to a dedicated email > list? > > Regards, Heiko > -- Michael Ernest Sr. Solutions Consultant West Coast

[GitHub] incubator-spark pull request: Fix ExternalMap on case of key's has...

2014-02-18 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/incubator-spark/pull/612#discussion_r9833685 This could be written pretty trivially using an Ordering, but I fear for the performance characteristics here, since this is a fast path. How does this look:

[GitHub] incubator-spark pull request: Fix ExternalMap on case of key's has...

2014-02-18 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/incubator-spark/pull/612#issuecomment-35413065 Jenkins, this is ok to test. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please to

[GitHub] incubator-spark pull request: Fix ExternalMap on case of key's has...

2014-02-18 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/incubator-spark/pull/612#issuecomment-35413258 Great find here! Thanks very much for submitting this patch. I just had a suggestion regarding the style of your check. It might be neat if we could add a test

[GitHub] incubator-spark pull request: Fix ExternalMap on case of key's has...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/612#issuecomment-35413691 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top-

[GitHub] incubator-spark pull request: Fix ExternalMap on case of key's has...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/612#issuecomment-35413690 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please t

[GitHub] incubator-spark pull request: Optimized imports

2014-02-18 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/incubator-spark/pull/613#discussion_r9833979 Feel free to put them on the same line, we discussed that overly long imports are perfectly fine. --- If your project is set up for it, you can reply to this ema

Re: Signal/Noise Ratio

2014-02-18 Thread Gerard Maas
+1 please. On Tue, Feb 18, 2014 at 6:04 PM, Michael Ernest wrote: > +1 > > > On Tue, Feb 18, 2014 at 8:24 AM, Heiko Braun >wrote: > > > > > > > Wouldn't it be better to move the github messages to a dedicated email > > list? > > > > Regards, Heiko > > > > > > -- > Michael Ernest > Sr. Solutions

[GitHub] incubator-spark pull request: Optimized imports

2014-02-18 Thread NirmalReddy
Github user NirmalReddy commented on the pull request: https://github.com/apache/incubator-spark/pull/613#discussion_r9834035 Ohh sure will change it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top-p

[GitHub] incubator-spark pull request: Optimized imports

2014-02-18 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/incubator-spark/pull/613#discussion_r9834070 braces not required --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top-post your

[GitHub] incubator-spark pull request: Optimized imports

2014-02-18 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/incubator-spark/pull/613#discussion_r9834055 These imports of Some are not actually required, just inserted by IDEs occasionally. This is actually the only remaining usage in our code base, so let's have awa

[GitHub] incubator-spark pull request: Optimized imports

2014-02-18 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/incubator-spark/pull/613#discussion_r9834148 extra whitespace here --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top-post yo

[GitHub] incubator-spark pull request: Optimized imports

2014-02-18 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/incubator-spark/pull/613#discussion_r9834189 nice! Deleted imports make me happy... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, p

[GitHub] incubator-spark pull request: Optimized imports

2014-02-18 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/incubator-spark/pull/613#issuecomment-35414789 Woo-hoo! Thanks for doing this. This could generate a lot of merge conflicts, so I will merge after comments are addressed. Since everything compiles, it's unli

[GitHub] incubator-spark pull request: Optimized imports

2014-02-18 Thread NirmalReddy
Github user NirmalReddy commented on the pull request: https://github.com/apache/incubator-spark/pull/613#discussion_r9834239 I dint get it !! Where the extra whitespace ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. T

[GitHub] incubator-spark pull request: [SPARK-1094] Support MiMa for report...

2014-02-18 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/incubator-spark/pull/585#discussion_r9834337 is it possible to exclude the entire `org.apache.spark.deploy` package? --- If your project is set up for it, you can reply to this email and have your reply appe

[GitHub] incubator-spark pull request: Optimized imports

2014-02-18 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/incubator-spark/pull/613#discussion_r9834309 Sorry, just meant there are 2 newlines between the last import and the following Javadoc. Not a change caused by you, but I thought I'd mention it anyway. --- I

[GitHub] incubator-spark pull request: Optimized imports

2014-02-18 Thread NirmalReddy
Github user NirmalReddy commented on the pull request: https://github.com/apache/incubator-spark/pull/613#discussion_r9834422 Ya that is a clarification for me as i left of 2 newlines in many cases as i am not sure .. will complete all those in my next commit !! Thanks !! --- If you

[GitHub] incubator-spark pull request: [SPARK-1094] Support MiMa for report...

2014-02-18 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/incubator-spark/pull/585#discussion_r9834543 I think you can do it with excludePackage: https://github.com/typesafehub/migration-manager/blob/master/core/src/main/scala/com/typesafe/tools/mima/core/Filter

[GitHub] incubator-spark pull request: [SPARK-1094] Support MiMa for report...

2014-02-18 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/incubator-spark/pull/585#discussion_r9834629 Then you can add a comment that says "Scheduler is not considered a public API" --- If your project is set up for it, you can reply to this email and have your r

[GitHub] incubator-spark pull request: [SPARK-1094] Support MiMa for report...

2014-02-18 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/incubator-spark/pull/585#discussion_r9834615 Let's just exclude the entire `ExternalAppendOnlyMap` class. The you can add a comment that says "Was made private in 1.0" --- If your project is set up for it,

[GitHub] incubator-spark pull request: [SPARK-1094] Support MiMa for report...

2014-02-18 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/incubator-spark/pull/585#discussion_r9834854 This and the one below were changed to private APIs. Also cogroupResult2ToJava, but that's not here for some reason. --- If your project is set up for it, you ca

[GitHub] incubator-spark pull request: Fix ExternalMap on case of key's has...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/612#issuecomment-35416706 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top

[GitHub] incubator-spark pull request: Fix ExternalMap on case of key's has...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/612#issuecomment-35416708 One or more automated tests failed Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12765/ ---

[GitHub] incubator-spark pull request: [SPARK-1094] Support MiMa for report...

2014-02-18 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/incubator-spark/pull/585#issuecomment-35416994 @ScrapCodes hm... I noticed a bigger issue that things which are package private are not being filtered in MIMA and this is leading to a bunch of the false posi

[GitHub] incubator-spark pull request: [java8API] SPARK-964 Investigate the...

2014-02-18 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/incubator-spark/pull/539#issuecomment-35420086 I just meant don't make the "of" methods public, since they're confusing for users to see. --- If your project is set up for it, you can reply to this email and

Re: IntelliJ inflates the CPU utilization when working on SparkIMain.scala

2014-02-18 Thread Konstantin Boudnik
I haven't seen anything like this recently. But if anything, the bug is most likely to be in the Scala plugin (or Scala itself [wink,wink]) than in the core IDEA which has proven over the years to be incredibly stable. If you continue seeing this, consider getting in touch with IntelliJ dev forum

[GitHub] incubator-spark pull request: Optimized imports

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/613#issuecomment-35426876 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top-

[GitHub] incubator-spark pull request: Optimized imports

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/613#issuecomment-35426875 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please t

[GitHub] incubator-spark pull request: SPARK-929: Fully deprecate usage of ...

2014-02-18 Thread aarondav
GitHub user aarondav opened a pull request: https://github.com/apache/incubator-spark/pull/615 SPARK-929: Fully deprecate usage of SPARK_MEM This patch cements our deprecation of the SPARK_MEM environment variable by replacing it with three more specialized variables: SPARK_DAEM

[GitHub] incubator-spark pull request: Optimized imports

2014-02-18 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/incubator-spark/pull/613#discussion_r9839626 Nice catch on this one. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top-post

[GitHub] incubator-spark pull request: SPARK-929: Fully deprecate usage of ...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/615#issuecomment-35427919 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please t

[GitHub] incubator-spark pull request: SPARK-929: Fully deprecate usage of ...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/615#issuecomment-35427920 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top-

Re: Signal/Noise Ratio

2014-02-18 Thread Aaron Davidson
This is due, unfortunately, to Apache policies that all development-related discussion should take place on the dev list. As we are attempting to graduate from an incubating project to an Apache top level project, there were some concerns raised about GitHub, and the fastest solution to avoid confl

[GitHub] incubator-spark pull request: Optimized imports

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/613#issuecomment-35429823 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12766/ --- If you

[GitHub] incubator-spark pull request: Optimized imports

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/613#issuecomment-35429821 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top

[GitHub] incubator-spark pull request: Optimized imports

2014-02-18 Thread NirmalReddy
Github user NirmalReddy commented on the pull request: https://github.com/apache/incubator-spark/pull/613#issuecomment-35430115 @aarondav Now is it all ok to merge ?? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do

[GitHub] incubator-spark pull request: Optimized imports

2014-02-18 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/613#discussion_r9840393 We should probably rename this JDouble or just use java.lang.Double in the codebase. --- If your project is set up for it, you can reply to this email and have your

[GitHub] incubator-spark pull request: Optimized imports

2014-02-18 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/incubator-spark/pull/613#issuecomment-35431514 I made one comment on the ambiguity of Double. Other than that, LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear o

Re: Signal/Noise Ratio

2014-02-18 Thread Matei Zaharia
Yup, because these are development discussions, they need to be archived, even though some users may not view them this way (but rather subscribe using GitHub). Apache wants to make sure that all work on its projects is done “in the open” and is accessible in a central place. Just filter them ou

[GitHub] incubator-spark pull request: Migrate Java code to Scala or move i...

2014-02-18 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/incubator-spark/pull/605#discussion_r9840846 Hm, let's use JavaSparkContext's fakeClassTag method, so we link to the documentation on why we're doing this. You may have to rebase to get this, it was a recent

[GitHub] incubator-spark pull request: Migrate Java code to Scala or move i...

2014-02-18 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/incubator-spark/pull/605#discussion_r9841070 same here `def returnType(): ClassTag[R] = JavaSparkContext.fakeClassTag` --- If your project is set up for it, you can reply to this email and have your repl

[GitHub] incubator-spark pull request: Migrate Java code to Scala or move i...

2014-02-18 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/incubator-spark/pull/605#discussion_r9841053 I'm like 85% sure we can use scala.Serializable (which extends java.io.Serializable) and should not require an import. --- If your project is set up for it, you

[GitHub] incubator-spark pull request: Migrate Java code to Scala or move i...

2014-02-18 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/incubator-spark/pull/605#issuecomment-35432171 Any reason not to convert StorageLevels as well? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] incubator-spark pull request: Optimized imports

2014-02-18 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/incubator-spark/pull/613#issuecomment-35432325 LGTM too after @rxin's change. Since Double is used throughout that file, I think renaming JDouble is cleanest. --- If your project is set up for it, you can r

[GitHub] incubator-spark pull request: SPARK-929: Fully deprecate usage of ...

2014-02-18 Thread sryza
Github user sryza commented on the pull request: https://github.com/apache/incubator-spark/pull/615#issuecomment-35432894 Why use MEMORY for the daemon, but MEM for the driver? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as we

[GitHub] incubator-spark pull request: [Proposal] Adding sparse data suppor...

2014-02-18 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/incubator-spark/pull/575#issuecomment-35433131 Thanks all for the suggestions! @srowen @giyengar I updated the small benchmark suite to include commons-math3. It seems to me commons-math3 has couple d

[GitHub] incubator-spark pull request: SPARK-929: Fully deprecate usage of ...

2014-02-18 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/incubator-spark/pull/615#issuecomment-35433301 I'm amenable to switching to SPARK_DRIVER_MEMORY. I avoided it for two reasons: 1) Following SPARK_MEM -> SPARK_DRIVER_MEM / SPARK_EXECUTOR_MEM 2) It's s

[GitHub] incubator-spark pull request: SPARK-929: Fully deprecate usage of ...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/615#issuecomment-35433388 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12767/ --- If you

[GitHub] incubator-spark pull request: SPARK-929: Fully deprecate usage of ...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/615#issuecomment-35433387 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top

[GitHub] incubator-spark pull request: Optimized imports

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/613#issuecomment-35434152 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please t

[GitHub] incubator-spark pull request: Optimized imports

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/613#issuecomment-35434153 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top-

[GitHub] incubator-spark pull request: Optimized imports

2014-02-18 Thread NirmalReddy
Github user NirmalReddy commented on the pull request: https://github.com/apache/incubator-spark/pull/613#issuecomment-35434195 @aarondav @rxin Thanks !! All the comments addressed !! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHu

Re: Signal/Noise Ratio

2014-02-18 Thread Pascal Voitot Dev
so -1 = 0 for me :D I can understand and I'll use filter ;) thks Pascal On Tue, Feb 18, 2014 at 9:46 PM, Matei Zaharia wrote: > Yup, because these are development discussions, they need to be archived, > even though some users may not view them this way (but rather subscribe > using GitHub). Ap

  1   2   3   >