databricks-david-lewis commented on a change in pull request #25787:
[SPARK-29081][CORE] Replace calls to SerializationUtils.clone on properties
with a faster implementation
URL: https://github.com/apache/spark/pull/25787#discussion_r324437647
##########
File path: core/src/test/scala/org/apache/spark/benchmark/Benchmark.scala
##########
@@ -141,8 +141,9 @@ private[spark] class Benchmark(
val minIters = if (overrideNumIters != 0) overrideNumIters else minNumIters
val minDuration = if (overrideNumIters != 0) 0 else minTime.toNanos
val runTimes = ArrayBuffer[Long]()
+ val startTime = System.nanoTime()
var i = 0
- while (i < minIters || runTimes.sum < minDuration) {
+ while (i < minIters || (System.nanoTime() - startTime) < minDuration) {
Review comment:
I meant to call this out. My empty properties test case was taking way too
long. I found that the overhead of the loop was orders of magnitude longer than
the function itself, especially when the array buffer was needing to resize a
lot.
So instead of summing the time for each run I changed it to keep track of
the total time elapsed.
I doubt that it will affect many other benchmarks, but it is something to
note.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
