Github user wangyum commented on the issue:
https://github.com/apache/spark/pull/21556
Benchmark results:
```
###############################[ Pushdown benchmark for Decimal
]################################
Java HotSpot(TM) 64-Bit Server VM 1.8.0_151-b12 on Mac OS X 10.12.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz
Select 1 decimal(9, 2) row (value = 7864320): Best/Avg Time(ms)
Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Parquet Vectorized 4004 / 5309 3.9
254.5 1.0X
Parquet Vectorized (Pushdown) 1401 / 1431 11.2
89.1 2.9X
Native ORC Vectorized 4499 / 4567 3.5
286.0 0.9X
Native ORC Vectorized (Pushdown) 899 / 961 17.5
57.2 4.5X
Select 10% decimal(9, 2) rows (value < 1572864): Best/Avg Time(ms)
Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Parquet Vectorized 5376 / 6437 2.9
341.8 1.0X
Parquet Vectorized (Pushdown) 2696 / 2754 5.8
171.4 2.0X
Native ORC Vectorized 5458 / 5623 2.9
347.0 1.0X
Native ORC Vectorized (Pushdown) 2230 / 2255 7.1
141.8 2.4X
Select 50% decimal(9, 2) rows (value < 7864320): Best/Avg Time(ms)
Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Parquet Vectorized 8280 / 8487 1.9
526.4 1.0X
Parquet Vectorized (Pushdown) 7716 / 7757 2.0
490.6 1.1X
Native ORC Vectorized 9144 / 9495 1.7
581.4 0.9X
Native ORC Vectorized (Pushdown) 7918 / 8118 2.0
503.4 1.0X
Select 90% decimal(9, 2) rows (value < 14155776): Best/Avg Time(ms)
Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Parquet Vectorized 9648 / 9676 1.6
613.4 1.0X
Parquet Vectorized (Pushdown) 9647 / 9778 1.6
613.3 1.0X
Native ORC Vectorized 10782 / 10867 1.5
685.5 0.9X
Native ORC Vectorized (Pushdown) 10108 / 10269 1.6
642.6 1.0X
Select 1 decimal(18, 2) row (value = 7864320): Best/Avg Time(ms)
Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Parquet Vectorized 4066 / 4147 3.9
258.5 1.0X
Parquet Vectorized (Pushdown) 84 / 89 188.0
5.3 48.6X
Native ORC Vectorized 5430 / 5512 2.9
345.3 0.7X
Native ORC Vectorized (Pushdown) 1054 / 1076 14.9
67.0 3.9X
Select 10% decimal(18, 2) rows (value < 1572864): Best/Avg Time(ms)
Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Parquet Vectorized 5028 / 5154 3.1
319.7 1.0X
Parquet Vectorized (Pushdown) 1360 / 1421 11.6
86.5 3.7X
Native ORC Vectorized 6266 / 6360 2.5
398.4 0.8X
Native ORC Vectorized (Pushdown) 2513 / 2550 6.3
159.8 2.0X
Select 50% decimal(18, 2) rows (value < 7864320): Best/Avg Time(ms)
Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Parquet Vectorized 8571 / 8600 1.8
544.9 1.0X
Parquet Vectorized (Pushdown) 6455 / 6713 2.4
410.4 1.3X
Native ORC Vectorized 10138 / 10353 1.6
644.5 0.8X
Native ORC Vectorized (Pushdown) 8166 / 8418 1.9
519.2 1.0X
Select 90% decimal(18, 2) rows (value < 14155776): Best/Avg Time(ms)
Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Parquet Vectorized 12184 / 12253 1.3
774.7 1.0X
Parquet Vectorized (Pushdown) 11720 / 11743 1.3
745.1 1.0X
Native ORC Vectorized 14024 / 14172 1.1
891.6 0.9X
Native ORC Vectorized (Pushdown) 13966 / 13996 1.1
887.9 0.9X
Select 1 decimal(38, 2) row (value = 7864320): Best/Avg Time(ms)
Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Parquet Vectorized 5650 / 6140 2.8
359.2 1.0X
Parquet Vectorized (Pushdown) 112 / 186 140.5
7.1 50.5X
Native ORC Vectorized 5890 / 6826 2.7
374.5 1.0X
Native ORC Vectorized (Pushdown) 1151 / 1396 13.7
73.2 4.9X
Select 10% decimal(38, 2) rows (value < 1572864): Best/Avg Time(ms)
Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Parquet Vectorized 6668 / 6863 2.4
424.0 1.0X
Parquet Vectorized (Pushdown) 1766 / 1781 8.9
112.3 3.8X
Native ORC Vectorized 6421 / 6448 2.4
408.2 1.0X
Native ORC Vectorized (Pushdown) 2648 / 2701 5.9
168.3 2.5X
Select 50% decimal(38, 2) rows (value < 7864320): Best/Avg Time(ms)
Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Parquet Vectorized 11268 / 11292 1.4
716.4 1.0X
Parquet Vectorized (Pushdown) 8544 / 8574 1.8
543.2 1.3X
Native ORC Vectorized 10975 / 11455 1.4
697.8 1.0X
Native ORC Vectorized (Pushdown) 9065 / 9166 1.7
576.3 1.2X
Select 90% decimal(38, 2) rows (value < 14155776): Best/Avg Time(ms)
Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Parquet Vectorized 15878 / 15972 1.0
1009.5 1.0X
Parquet Vectorized (Pushdown) 15141 / 15328 1.0
962.6 1.0X
Native ORC Vectorized 15407 / 16003 1.0
979.5 1.0X
Native ORC Vectorized (Pushdown) 15347 / 15414 1.0
975.7 1.0X
```
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
