[GitHub] spark pull request: SPARK-2553. CoGroupedRDD unnecessarily allocat...

2014-07-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/1461 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: SPARK-2553. CoGroupedRDD unnecessarily allocat...

2014-07-18 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/1461#issuecomment-49401094 Merged this, thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: SPARK-2553. CoGroupedRDD unnecessarily allocat...

2014-07-18 Thread sryza
Github user sryza commented on the pull request: https://github.com/apache/spark/pull/1461#issuecomment-49401593 It looks like there was actually a compile error here that I missed last night. Uploading the fix. --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: SPARK-2553. CoGroupedRDD unnecessarily allocat...

2014-07-18 Thread sryza
Github user sryza commented on the pull request: https://github.com/apache/spark/pull/1461#issuecomment-49402486 Fix: https://github.com/apache/spark/pull/1479 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: SPARK-2553. CoGroupedRDD unnecessarily allocat...

2014-07-18 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/spark/pull/1461#issuecomment-49405145 This PR also changed the behavior with respect to mutating CoGroups -- now we mutate combiner1 in place rather than returning a new ArrayBuffer. This is _probably_ not

[GitHub] spark pull request: SPARK-2553. CoGroupedRDD unnecessarily allocat...

2014-07-18 Thread sryza
Github user sryza commented on the pull request: https://github.com/apache/spark/pull/1461#issuecomment-49406581 I actually did consider this, I should have made a note in the PR. Agreed that we should be careful with the implications of these small changes. --- If your project is

[GitHub] spark pull request: SPARK-2553. CoGroupedRDD unnecessarily allocat...

2014-07-18 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/1461#issuecomment-49496955 Oops, sorry about that, the Jenkins messages look too similar now :) --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: SPARK-2553. CoGroupedRDD unnecessarily allocat...

2014-07-17 Thread sryza
GitHub user sryza opened a pull request: https://github.com/apache/spark/pull/1461 SPARK-2553. CoGroupedRDD unnecessarily allocates a Tuple2 per dependency... ... per key My humble opinion is that avoiding allocations in this performance-critical section is worth the extra

[GitHub] spark pull request: SPARK-2553. CoGroupedRDD unnecessarily allocat...

2014-07-17 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/1461#issuecomment-49385926 Jenkins, add to whitelist and test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: SPARK-2553. CoGroupedRDD unnecessarily allocat...

2014-07-17 Thread mateiz
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/1461#issuecomment-49386002 Looks good to me, though this isn't avoiding *that* much since you are appending to an ArrayBuffer anyway. But no reason not to do it. --- If your project is set up for

[GitHub] spark pull request: SPARK-2553. CoGroupedRDD unnecessarily allocat...

2014-07-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1461#issuecomment-49386013 QA tests have started for PR 1461. This patch merges cleanly. brView progress: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16800/consoleFull ---

[GitHub] spark pull request: SPARK-2553. CoGroupedRDD unnecessarily allocat...

2014-07-17 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1461#issuecomment-49386170 QA results for PR 1461:br- This patch FAILED unit tests.br- This patch merges cleanlybr- This patch adds no public classesbrbrFor more information see test