[GitHub] spark pull request: [SPARK-11393][SQL] CoGroupedIterator should re...

2015-10-30 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/9346 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-11393][SQL] CoGroupedIterator should re...

2015-10-30 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/9346#issuecomment-152498781 Okay, thanks. Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-11393][SQL] CoGroupedIterator should re...

2015-10-29 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/9346#issuecomment-152157478 Maybe "not idempotent" is not a proper word to describe this problem, `GroupedIterator` has a special

[GitHub] spark pull request: [SPARK-11393][SQL] CoGroupedIterator should re...

2015-10-29 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/9346#issuecomment-152158690 btw as https://github.com/apache/spark/pull/9330 has been merge, the problem is not generating an extra empty group but making the last group empty. --- If your

[GitHub] spark pull request: [SPARK-11393][SQL] CoGroupedIterator should re...

2015-10-29 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9346#issuecomment-152276815 If we use `grouped` directly and call `hasNext` right after we call `next`, the issue will be exposed. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-11393][SQL] CoGroupedIterator should re...

2015-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9346#issuecomment-152095218 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11393][SQL] CoGroupedIterator should re...

2015-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9346#issuecomment-152095222 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11393][SQL] CoGroupedIterator should re...

2015-10-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9346#issuecomment-152094846 **[Test build #44559 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44559/consoleFull)** for PR 9346 at commit

[GitHub] spark pull request: [SPARK-11393][SQL] CoGroupedIterator should re...

2015-10-29 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/9346#issuecomment-152149587 I'm a little confused. If `GroupedIterator.hasNext` isn't idempotent then we should fix that instead of working around it. However, as of master I wasn't able to

[GitHub] spark pull request: [SPARK-11393][SQL] CoGroupedIterator should re...

2015-10-28 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/9346 [SPARK-11393][SQL] CoGroupedIterator should respect the fact that GroupedIterator.hasNext is not idempotent When we cogroup 2 `GroupedIterator`s in `CoGroupedIterator`, if the right side is

[GitHub] spark pull request: [SPARK-11393][SQL] CoGroupedIterator should re...

2015-10-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9346#issuecomment-152072395 **[Test build #44559 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44559/consoleFull)** for PR 9346 at commit

[GitHub] spark pull request: [SPARK-11393][SQL] CoGroupedIterator should re...

2015-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9346#issuecomment-152071688 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11393][SQL] CoGroupedIterator should re...

2015-10-28 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9346#issuecomment-152071798 cc @marmbrus --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-11393][SQL] CoGroupedIterator should re...

2015-10-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9346#issuecomment-152071683 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not