GitHub user holdenk opened a pull request:
https://github.com/apache/spark/pull/242
[WIP] Spark 1271 (1320) cogroup and groupby should pass iterator[x]
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/holdenk/spark
spark-1320-cogroupandgroupshouldpassiterator
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/242.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #242
----
commit 4cd116f08d6c759e1a94ccac549012e42ea2e2cf
Author: Holden Karau <[email protected]>
Date: 2014-03-25T22:32:55Z
This is the it compiles point of replacing Seq with Iterator and JList with
JIterator in the groupby and cogroup signatures
commit 5ccb7ac8346c08a2bfdbf096ff8d423672959c2c
Author: Holden Karau <[email protected]>
Date: 2014-03-25T22:40:31Z
Calling size on an iterator is not so good if we want to use it after
commit 44b7259eeb78211e5dfb452b4e94be3e4e3c4239
Author: Holden Karau <[email protected]>
Date: 2014-03-25T22:40:37Z
Fix some tests
commit b676212847654f5e32125474b9f520da29760e2c
Author: Holden Karau <[email protected]>
Date: 2014-03-25T23:06:02Z
Fix groupbykey to actually give back an iterator
commit 0d395229da708d067fcb3e6ab0d6497e46279b0d
Author: Holden Karau <[email protected]>
Date: 2014-03-26T00:20:41Z
I think this might be a bad rabbit hole. Started work to make CoGroupedRDD
use iterator and then went crazy
commit 6576ba252288998f1861b5b55b81b276ef6f8458
Author: Holden Karau <[email protected]>
Date: 2014-03-26T00:21:28Z
Fix Java API suite
commit 75122e6e1ebc0dfe8854c3670064faacbb119689
Author: Holden Karau <[email protected]>
Date: 2014-03-26T00:21:45Z
hmmm try and fix up basic operation suite
commit a6e24ed0c3b1636ba19e2a59f349016e3283d675
Author: Holden Karau <[email protected]>
Date: 2014-03-26T00:22:20Z
Revert "I think this might be a bad rabbit hole. Started work to make
CoGroupedRDD use iterator and then went crazy"
This reverts commit df9afbec7e9fb558cf75d4e8dc94d8f44f101301.
commit e17ac9b28e660a4de6ba140d101cc55957023fbf
Author: Holden Karau <[email protected]>
Date: 2014-03-26T00:45:13Z
org.apache.spark.rdd.PairRDDFunctionsSuite passes
commit 0ec6bd701f4170f16a655f60ce372485bf9b2cae
Author: Holden Karau <[email protected]>
Date: 2014-03-26T01:29:18Z
core/tests now pass
commit cdbd16316aac7d702fbb67180b21eff00aa2c594
Author: Holden Karau <[email protected]>
Date: 2014-03-26T01:37:32Z
Fix some of the types in the streaming JavaAPI suite. Probably still needs
more work
commit 100b21ac2cef532df13824cd02e16c4d03d5a4fb
Author: Holden Karau <[email protected]>
Date: 2014-03-26T16:38:46Z
I really need to stop calling size on iterators, it is the path of sadness.
commit 31ebcf1cc1b599941f57d833f0cae4b8f5a19703
Author: Holden Karau <[email protected]>
Date: 2014-03-26T16:51:07Z
Fix style issues
commit 4b088062f9fcdaf419c44dca60c1e57aae2ef194
Author: Holden Karau <[email protected]>
Date: 2014-03-26T17:59:10Z
Revert this but for now put things in list pandas
commit b5e0849ad5ce7bba1f77c55da5b3164ed29f6558
Author: Holden Karau <[email protected]>
Date: 2014-03-26T20:21:41Z
Add a join based on the problem in SVD
commit 3247db74603e06292d83d166b02d35740482b395
Author: Holden Karau <[email protected]>
Date: 2014-03-26T20:35:20Z
Revert me, but we have some stuff to debug
more loggings
commit 315696172ca709f218018da78e409787e91633c2
Author: Holden Karau <[email protected]>
Date: 2014-03-26T21:11:33Z
Fix the bug
commit 33f17c93ecc1781e85ab09ad9e8b3f4f325d9b6f
Author: Holden Karau <[email protected]>
Date: 2014-03-26T22:12:19Z
Revert
commit 2992d93a10ca1f8917aebdcecb972203b6c90146
Author: Holden Karau <[email protected]>
Date: 2014-03-26T23:37:22Z
Revert "Revert this but for now put things in list pandas"
This reverts commit 4b088062f9fcdaf419c44dca60c1e57aae2ef194.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
