[GitHub] spark pull request: Clarify docstring for Pyspark's foreachPartiti...

2014-12-16 Thread tdhopper
Github user tdhopper commented on the pull request: https://github.com/apache/spark/pull/2895#issuecomment-67162827 @JoshRosen: Yup. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Clarify docstring for Pyspark's foreachPartiti...

2014-12-16 Thread tdhopper
Github user tdhopper closed the pull request at: https://github.com/apache/spark/pull/2895 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: Clarify docstring for Pyspark's foreachPartiti...

2014-12-15 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/2895#issuecomment-67102429 If you don't mind, could you close this PR since it has been subsumed by another commit? If we want to track the progress / backport status of a different fix, then

[GitHub] spark pull request: Clarify docstring for Pyspark's foreachPartiti...

2014-10-23 Thread tdhopper
Github user tdhopper commented on the pull request: https://github.com/apache/spark/pull/2895#issuecomment-60234425 Oh. Now that I look at master, @JoshRosen, I see that it's already been fixed by @davis

[GitHub] spark pull request: Clarify docstring for Pyspark's foreachPartiti...

2014-10-23 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/2895#issuecomment-60287652 Maybe we can backport SPARK-2871 to 1.1, since it looks like it also fixes a bunch of preservesPartitioning bugs. --- If your project is set up for it, you can reply

[GitHub] spark pull request: Clarify docstring for Pyspark's foreachPartiti...

2014-10-23 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/2895#issuecomment-60287875 @davies Do you think we should backport #2093 to branch-1.1 in order to fix this issue? --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: Clarify docstring for Pyspark's foreachPartiti...

2014-10-23 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/2895#issuecomment-60292102 @JoshRosen It will be better if we could easily backport them. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: Clarify docstring for Pyspark's foreachPartiti...

2014-10-23 Thread tdhopper
Github user tdhopper commented on the pull request: https://github.com/apache/spark/pull/2895#issuecomment-60304723 I'd love to see this happen. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: Clarify docstring for Pyspark's foreachPartiti...

2014-10-23 Thread davis
Github user davis commented on the pull request: https://github.com/apache/spark/pull/2895#issuecomment-60306345 hm why am i notified here? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Clarify docstring for Pyspark's foreachPartiti...

2014-10-23 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/2895#issuecomment-60306509 @davis I think someone mentioned you instead of @davies (an off-by-one-character error). --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: Clarify docstring for Pyspark's foreachPartiti...

2014-10-23 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/2895#issuecomment-60306682 (Imagine what `@Override`'s notification inbox must look like...) --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: Clarify docstring for Pyspark's foreachPartiti...

2014-10-22 Thread tdhopper
GitHub user tdhopper opened a pull request: https://github.com/apache/spark/pull/2895 Clarify docstring for Pyspark's foreachPartition Due to the underlying use of `mapPartitions` which requires a function that maps partitions to partitions, `foreachPartition` requires the function

[GitHub] spark pull request: Clarify docstring for Pyspark's foreachPartiti...

2014-10-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2895#issuecomment-60145810 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: Clarify docstring for Pyspark's foreachPartiti...

2014-10-22 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/2895#issuecomment-60159250 Actually, we might want to just fix this and allow `foreachPartition` and `foreach` to accept those types of UDFs, too. --- If your project is set up for it, you can