aokolnychyi commented on issue #23171: [SPARK-26205][SQL] Optimize In for bytes, shorts, ints URL: https://github.com/apache/spark/pull/23171#issuecomment-446679580 @mgaido91 It won't be possible to apply the switch-based approach on timestamps as they are represented as longs. We can try dates as are represented as ints. Below is the result of that benchmark with this patch: ``` ================================================================================================ In Expression Benchmark ================================================================================================ Java HotSpot(TM) 64-Bit Server VM 1.8.0_192-b12 on Mac OS X 10.14 Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz 5 bytes: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------ In expression 45 / 56 223.3 4.5 1.0X InSet expression 58 / 61 173.4 5.8 0.8X Java HotSpot(TM) 64-Bit Server VM 1.8.0_192-b12 on Mac OS X 10.14 Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz 10 bytes: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------ In expression 37 / 40 268.8 3.7 1.0X InSet expression 63 / 67 158.2 6.3 0.6X Java HotSpot(TM) 64-Bit Server VM 1.8.0_192-b12 on Mac OS X 10.14 Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz 25 bytes: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------ In expression 51 / 55 194.4 5.1 1.0X InSet expression 87 / 92 114.3 8.7 0.6X Java HotSpot(TM) 64-Bit Server VM 1.8.0_192-b12 on Mac OS X 10.14 Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz 50 bytes: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------ In expression 74 / 79 135.4 7.4 1.0X InSet expression 128 / 138 78.1 12.8 0.6X Java HotSpot(TM) 64-Bit Server VM 1.8.0_192-b12 on Mac OS X 10.14 Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz 100 bytes: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------ In expression 123 / 128 81.1 12.3 1.0X InSet expression 197 / 212 50.7 19.7 0.6X Java HotSpot(TM) 64-Bit Server VM 1.8.0_192-b12 on Mac OS X 10.14 Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz 5 shorts: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------ In expression 29 / 31 342.2 2.9 1.0X InSet expression 110 / 114 90.8 11.0 0.3X Java HotSpot(TM) 64-Bit Server VM 1.8.0_192-b12 on Mac OS X 10.14 Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz 10 shorts: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------ In expression 25 / 27 407.3 2.5 1.0X InSet expression 122 / 127 82.1 12.2 0.2X Java HotSpot(TM) 64-Bit Server VM 1.8.0_192-b12 on Mac OS X 10.14 Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz 25 shorts: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------ In expression 27 / 29 367.2 2.7 1.0X InSet expression 124 / 127 80.9 12.4 0.2X Java HotSpot(TM) 64-Bit Server VM 1.8.0_192-b12 on Mac OS X 10.14 Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz 50 shorts: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------ In expression 26 / 27 386.5 2.6 1.0X InSet expression 158 / 162 63.2 15.8 0.2X Java HotSpot(TM) 64-Bit Server VM 1.8.0_192-b12 on Mac OS X 10.14 Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz 100 shorts: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------ In expression 28 / 30 353.5 2.8 1.0X InSet expression 136 / 141 73.4 13.6 0.2X Java HotSpot(TM) 64-Bit Server VM 1.8.0_192-b12 on Mac OS X 10.14 Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz 200 shorts: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------ In expression 33 / 36 307.0 3.3 1.0X InSet expression 136 / 141 73.5 13.6 0.2X Java HotSpot(TM) 64-Bit Server VM 1.8.0_192-b12 on Mac OS X 10.14 Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz 5 ints: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------ In expression 26 / 28 389.4 2.6 1.0X InSet expression 108 / 115 92.7 10.8 0.2X Java HotSpot(TM) 64-Bit Server VM 1.8.0_192-b12 on Mac OS X 10.14 Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz 10 ints: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------ In expression 23 / 25 431.0 2.3 1.0X InSet expression 119 / 124 84.1 11.9 0.2X Java HotSpot(TM) 64-Bit Server VM 1.8.0_192-b12 on Mac OS X 10.14 Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz 25 ints: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------ In expression 24 / 25 420.9 2.4 1.0X InSet expression 123 / 127 81.5 12.3 0.2X Java HotSpot(TM) 64-Bit Server VM 1.8.0_192-b12 on Mac OS X 10.14 Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz 50 ints: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------ In expression 25 / 26 406.6 2.5 1.0X InSet expression 153 / 157 65.4 15.3 0.2X Java HotSpot(TM) 64-Bit Server VM 1.8.0_192-b12 on Mac OS X 10.14 Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz 100 ints: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------ In expression 26 / 27 391.4 2.6 1.0X InSet expression 128 / 133 78.3 12.8 0.2X Java HotSpot(TM) 64-Bit Server VM 1.8.0_192-b12 on Mac OS X 10.14 Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz 200 ints: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------ In expression 28 / 30 354.9 2.8 1.0X InSet expression 126 / 131 79.7 12.6 0.2X ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
