dongjoon-hyun edited a comment on issue #22617: [SPARK-25484][SQL][TEST] Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark URL: https://github.com/apache/spark/pull/22617#issuecomment-452626356 I made a PR to you, @peter-toth . - https://github.com/peter-toth/spark/pull/3 (EC2 result). And, for the record, the following is Mac result. Most stuffs are consistent with this PR except `Array with 1000 rows`. ``` ================================================================================================ WITHOUT SPILL ================================================================================================ Java HotSpot(TM) 64-Bit Server VM 1.8.0_191-b12 on Mac OS X 10.14.2 Intel(R) Core(TM) i7-4770HQ CPU @ 2.20GHz Array with 100000 rows: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------ ArrayBuffer 5071 / 5178 20.2 49.5 1.0X ExternalAppendOnlyUnsafeRowArray 5564 / 5583 18.4 54.3 0.9X Java HotSpot(TM) 64-Bit Server VM 1.8.0_191-b12 on Mac OS X 10.14.2 Intel(R) Core(TM) i7-4770HQ CPU @ 2.20GHz Array with 1000 rows: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------ ArrayBuffer 10146 / 10169 25.8 38.7 1.0X ExternalAppendOnlyUnsafeRowArray 24099 / 24414 10.9 91.9 0.4X Java HotSpot(TM) 64-Bit Server VM 1.8.0_191-b12 on Mac OS X 10.14.2 Intel(R) Core(TM) i7-4770HQ CPU @ 2.20GHz Array with 30000 rows: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------ ArrayBuffer 22738 / 22748 21.6 46.3 1.0X ExternalAppendOnlyUnsafeRowArray 28000 / 28096 17.6 57.0 0.8X ================================================================================================ WITH SPILL ================================================================================================ Java HotSpot(TM) 64-Bit Server VM 1.8.0_191-b12 on Mac OS X 10.14.2 Intel(R) Core(TM) i7-4770HQ CPU @ 2.20GHz Spilling with 1000 rows: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------ UnsafeExternalSorter 18076 / 18179 14.5 69.0 1.0X ExternalAppendOnlyUnsafeRowArray 11861 / 11872 22.1 45.2 1.5X Java HotSpot(TM) 64-Bit Server VM 1.8.0_191-b12 on Mac OS X 10.14.2 Intel(R) Core(TM) i7-4770HQ CPU @ 2.20GHz Spilling with 10000 rows: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------ UnsafeExternalSorter 7 / 7 23.1 43.3 1.0X ExternalAppendOnlyUnsafeRowArray 7 / 8 22.3 44.8 1.0X ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
