databricks-david-lewis commented on issue #25787: [SPARK-29081][CORE] Replace calls to SerializationUtils.clone on properties with a faster implementation URL: https://github.com/apache/spark/pull/25787#issuecomment-531502737 I made what I hope is a proper bench mark! Here are the results: ``` [info] Running org.apache.spark.util.PropertiesCloneBenchmark [info] Running benchmark: Empty Properties [info] Running case: SerializationUtils.clone [info] Stopped after 322992 iterations, 1965 ms [info] Running case: Utils.cloneProperties [info] Stopped after 14292083 iterations, 999 ms [info] [info] Java HotSpot(TM) 64-Bit Server VM 1.8.0_131-b11 on Mac OS X 10.14.6 [info] Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz [info] Empty Properties: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative [info] ------------------------------------------------------------------------------------------------------------------------ [info] SerializationUtils.clone 0 0 0 0.2 4216.0 1.0X [info] Utils.cloneProperties 0 0 0 90.9 11.0 383.3X [info] [info] Running benchmark: System Properties [info] Running case: SerializationUtils.clone [info] Stopped after 12051 iterations, 1998 ms [info] Running case: Utils.cloneProperties [info] Stopped after 1534682 iterations, 1891 ms [info] [info] Java HotSpot(TM) 64-Bit Server VM 1.8.0_131-b11 on Mac OS X 10.14.6 [info] Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz [info] System Properties: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative [info] ------------------------------------------------------------------------------------------------------------------------ [info] SerializationUtils.clone 0 0 0 0.0 121122.0 1.0X [info] Utils.cloneProperties 0 0 0 1.1 947.0 127.9X [info] [info] Running benchmark: Small Properties [info] Running case: SerializationUtils.clone [info] Stopped after 5149 iterations, 1999 ms [info] Running case: Utils.cloneProperties [info] Stopped after 1428075 iterations, 1902 ms [info] [info] Java HotSpot(TM) 64-Bit Server VM 1.8.0_131-b11 on Mac OS X 10.14.6 [info] Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz [info] Small Properties: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative [info] ------------------------------------------------------------------------------------------------------------------------ [info] SerializationUtils.clone 0 0 0 0.0 315474.0 1.0X [info] Utils.cloneProperties 0 0 0 0.9 1106.0 285.2X [info] [info] Running benchmark: Medium Properties [info] Running case: SerializationUtils.clone [info] Stopped after 1304 iterations, 2000 ms [info] Running case: Utils.cloneProperties [info] Stopped after 296214 iterations, 1979 ms [info] [info] Java HotSpot(TM) 64-Bit Server VM 1.8.0_131-b11 on Mac OS X 10.14.6 [info] Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz [info] Medium Properties: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative [info] ------------------------------------------------------------------------------------------------------------------------ [info] SerializationUtils.clone 1 2 0 0.0 1296861.0 1.0X [info] Utils.cloneProperties 0 0 0 0.2 5450.0 238.0X [info] [info] Running benchmark: Large Properties [info] Running case: SerializationUtils.clone [info] Stopped after 675 iterations, 2001 ms [info] Running case: Utils.cloneProperties [info] Stopped after 154225 iterations, 1989 ms [info] [info] Java HotSpot(TM) 64-Bit Server VM 1.8.0_131-b11 on Mac OS X 10.14.6 [info] Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz [info] Large Properties: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative [info] ------------------------------------------------------------------------------------------------------------------------ [info] SerializationUtils.clone 3 3 0 0.0 2587096.0 1.0X [info] Utils.cloneProperties 0 0 0 0.1 11077.0 233.6X ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
