Github user NarineK commented on the pull request:
https://github.com/apache/spark/pull/12836#issuecomment-219896062
Hi @sun-rui ,
The suggestion about 2.2) in previous comment is good:
https://github.com/apache/spark/pull/12836#issuecomment-219383918
however, in order to find the number of rows in the group we need to
somehow reduceByKey, use withZipIndex, or use iterator.duplicate etc., I've
tried different options but it is hard to count the number of elements without
exhausting the rowIter and without mutable variables.
Did you have something specific in your mind ?
https://github.com/NarineK/spark/blob/0b1b2558189a72903500500b96e22084120b96c0/sql/core/src/main/scala/org/apache/spark/sql/execution/objects.scala#L368
Thank you,
Narine
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
