Github user superbobry commented on a diff in the pull request:
https://github.com/apache/spark/pull/19369#discussion_r142392349
--- Diff:
core/src/main/scala/org/apache/spark/storage/BlockReplicationPolicy.scala ---
@@ -85,11 +65,9 @@ object BlockReplicationUtils {
* randomly shuffle elems
*/
def getRandomSample[T](elems: Seq[T], m: Int, r: Random): List[T] = {
- if (elems.size > m) {
- getSampleIds(elems.size, m, r).map(elems(_))
- } else {
- r.shuffle(elems).toList
- }
+ // This takes linear space, but is stable wrt m. That is for a fixed
--- End diff --
Sounds good to me. Will push an update.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
