Github user pkuwm commented on the issue:
https://github.com/apache/spark/pull/21386
For the tests, I was trying to do this:
1. assertEqualsInorgeOrder(shuffle(originSeq), originSeq)
But spark does not have assertEqualsInorgeOrder implemented. I was thinking
to check Multiset of shuffle(originSeq) and originSeq. But had trouble using
Multiset for expression and seq.
2. About the randomness, I was thinking to generate a Seq range(1, 501) and
shuffle it 30 times. And it should produce at least 80% distinct permutations.
Say using HashSet.add(shuffledResult). But I don't know how to implement this
idea in scala and codeGen for expressions.
This is my 1st time contributing to spark and codeGen. I hope committers
and contributors could help with tests, and also the shuffle function code.
Thanks a lot!
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
