Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/4533#issuecomment-73907037
Looking at the implementation, the driver does query one partition at a
time for the number of elements it thinks it needs and continues until it is
satisfied. I'd imagine this means querying just the first partition in almost
all cases, though it is done serially. Seems about right to me as opposed to
preemptively querying many partitions. The job can run locally on the driver.
So I don't know if the comment is categorically wrong, but also doesn't seem to
be clearly communicating something helpful. For that reason, IMHO it could be
removed. There's no "gotcha" I know of here.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
