[GitHub] incubator-spark pull request: Fix ExternalMap on case of key's has...

2014-02-20 Thread guojc
Github user guojc commented on the pull request: https://github.com/apache/incubator-spark/pull/612#issuecomment-35694101 Yes, I'm Jiacheng Guo. It's ok if you can find another good solution for this bug. Thanks for your guy's work. --- If your project is set up for it, you can reply

[GitHub] incubator-spark pull request: Fix ExternalMap on case of key's has...

2014-02-20 Thread guojc
Github user guojc closed the pull request at: https://github.com/apache/incubator-spark/pull/612 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top-post your response. If your project does not have this feature

[GitHub] incubator-spark pull request: Fix ExternalMap on case of key's has...

2014-02-20 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/incubator-spark/pull/612#issuecomment-35681966 @guojc Hey, we discussed about this a little more and we thought of a different way of solving this that also simplifies some of the existing logic. I will cr

[GitHub] incubator-spark pull request: Fix ExternalMap on case of key's has...

2014-02-20 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/incubator-spark/pull/612#issuecomment-35681623 @guojc - hey since Andrew may propose a slightly different fix, I want to make sure you are credited with this in our release notes. Are you Jiacheng Guo? Found

[GitHub] incubator-spark pull request: Fix ExternalMap on case of key's has...

2014-02-20 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/incubator-spark/pull/612#discussion_r9925806 --- Diff: core/src/test/scala/org/apache/spark/util/collection/ExternalAppendOnlyMapSuite.scala --- @@ -83,6 +83,28 @@ class ExternalAppendOnlyMap

[GitHub] incubator-spark pull request: Fix ExternalMap on case of key's has...

2014-02-20 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/incubator-spark/pull/612#issuecomment-35672119 Thanks again for finding this bug. A number of users have reported it before and we had not been able to provide a good answer, but this patch pretty much exp

[GitHub] incubator-spark pull request: Fix ExternalMap on case of key's has...

2014-02-20 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/incubator-spark/pull/612#discussion_r9925630 --- Diff: core/src/test/scala/org/apache/spark/util/collection/ExternalAppendOnlyMapSuite.scala --- @@ -83,6 +83,28 @@ class ExternalAppendOnlyMap

[GitHub] incubator-spark pull request: Fix ExternalMap on case of key's has...

2014-02-20 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/incubator-spark/pull/612#discussion_r9925460 --- Diff: core/src/test/scala/org/apache/spark/util/collection/ExternalAppendOnlyMapSuite.scala --- @@ -83,6 +83,28 @@ class ExternalAppendOnlyMap

[GitHub] incubator-spark pull request: Fix ExternalMap on case of key's has...

2014-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/612#issuecomment-35621470 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12786/ --- If you

[GitHub] incubator-spark pull request: Fix ExternalMap on case of key's has...

2014-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/612#issuecomment-35621469 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top

[GitHub] incubator-spark pull request: Fix ExternalMap on case of key's has...

2014-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/612#issuecomment-35619100 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please t

[GitHub] incubator-spark pull request: Fix ExternalMap on case of key's has...

2014-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/612#issuecomment-35619101 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top-

[GitHub] incubator-spark pull request: Fix ExternalMap on case of key's has...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/612#issuecomment-35416708 One or more automated tests failed Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12765/ ---

[GitHub] incubator-spark pull request: Fix ExternalMap on case of key's has...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/612#issuecomment-35416706 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top

[GitHub] incubator-spark pull request: Fix ExternalMap on case of key's has...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/612#issuecomment-35413691 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please top-

[GitHub] incubator-spark pull request: Fix ExternalMap on case of key's has...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/612#issuecomment-35413690 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please t

[GitHub] incubator-spark pull request: Fix ExternalMap on case of key's has...

2014-02-18 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/incubator-spark/pull/612#issuecomment-35413258 Great find here! Thanks very much for submitting this patch. I just had a suggestion regarding the style of your check. It might be neat if we could add a test

[GitHub] incubator-spark pull request: Fix ExternalMap on case of key's has...

2014-02-18 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/incubator-spark/pull/612#issuecomment-35413065 Jenkins, this is ok to test. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To do so, please to

[GitHub] incubator-spark pull request: Fix ExternalMap on case of key's has...

2014-02-18 Thread aarondav
Github user aarondav commented on the pull request: https://github.com/apache/incubator-spark/pull/612#discussion_r9833685 This could be written pretty trivially using an Ordering, but I fear for the performance characteristics here, since this is a fast path. How does this look:

[GitHub] incubator-spark pull request: Fix ExternalMap on case of key's has...

2014-02-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/incubator-spark/pull/612#issuecomment-35372535 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. To

[GitHub] incubator-spark pull request: Fix ExternalMap on case of key's has...

2014-02-18 Thread guojc
GitHub user guojc opened a pull request: https://github.com/apache/incubator-spark/pull/612 Fix ExternalMap on case of key's hashCode equal to Int.Maxvalue Currently StreamBuffer use hash value to compare between buffers. And it want emmit buffer with lowest HashValue from Prio