Github user viirya commented on the pull request:
https://github.com/apache/spark/pull/11578#issuecomment-194654994
@davies I just benchmark that using generated code for sample() with #11517.
When `withReplacement = false`.
Without generated code for sample():
Intel(R) Core(TM) i7-5557U CPU @ 3.10GHz
range/sample/sum: Best/Avg Time(ms) Rate(M/s)
Per Row(ns) Relative
-------------------------------------------------------------------------------------------
range/sample/sum codegen=false 16460 / 17161 31.9
31.4 1.0X
range/sample/sum codegen=true 4081 / 5390 128.5
7.8 4.0X
With generated code for sample():
Intel(R) Core(TM) i7-5557U CPU @ 3.10GHz
range/sample/sum: Best/Avg Time(ms) Rate(M/s)
Per Row(ns) Relative
-------------------------------------------------------------------------------------------
range/sample/sum codegen=false 14018 / 16011 37.4
26.7 1.0X
range/sample/sum codegen=true 2476 / 2719 211.7
4.7 5.7X
Looks like it is faster with generated code.
I will benchmark for `withReplacement = true` later.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
