Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/21288#discussion_r189120527
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/FilterPushdownBenchmark.scala ---
@@ -105,138 +128,306 @@ object FilterPushdownBenchmark {
}
/*
- Java HotSpot(TM) 64-Bit Server VM 1.8.0_152-b16 on Mac OS X 10.13.2
- Intel(R) Core(TM) i7-4770HQ CPU @ 2.20GHz
-
- Select 0 row (id IS NULL): Best/Avg Time(ms) Rate(M/s)
Per Row(ns) Relative
-
-----------------------------------------------------------------------------------------------
- Parquet Vectorized 7882 / 7957 2.0
501.1 1.0X
- Parquet Vectorized (Pushdown) 55 / 60 285.2
3.5 142.9X
- Native ORC Vectorized 5592 / 5627 2.8
355.5 1.4X
- Native ORC Vectorized (Pushdown) 66 / 70 237.2
4.2 118.9X
-
- Select 0 row (7864320 < id < 7864320): Best/Avg Time(ms) Rate(M/s)
Per Row(ns) Relative
-
-----------------------------------------------------------------------------------------------
- Parquet Vectorized 7884 / 7909 2.0
501.2 1.0X
- Parquet Vectorized (Pushdown) 739 / 752 21.3
47.0 10.7X
- Native ORC Vectorized 5614 / 5646 2.8
356.9 1.4X
- Native ORC Vectorized (Pushdown) 81 / 83 195.2
5.1 97.8X
-
- Select 1 row (id = 7864320): Best/Avg Time(ms) Rate(M/s)
Per Row(ns) Relative
-
-----------------------------------------------------------------------------------------------
- Parquet Vectorized 7905 / 8027 2.0
502.6 1.0X
- Parquet Vectorized (Pushdown) 740 / 766 21.2
47.1 10.7X
- Native ORC Vectorized 5684 / 5738 2.8
361.4 1.4X
- Native ORC Vectorized (Pushdown) 78 / 81 202.4
4.9 101.7X
-
- Select 1 row (id <=> 7864320): Best/Avg Time(ms) Rate(M/s)
Per Row(ns) Relative
-
-----------------------------------------------------------------------------------------------
- Parquet Vectorized 7928 / 7993 2.0
504.1 1.0X
- Parquet Vectorized (Pushdown) 747 / 772 21.0
47.5 10.6X
- Native ORC Vectorized 5728 / 5753 2.7
364.2 1.4X
- Native ORC Vectorized (Pushdown) 76 / 78 207.9
4.8 104.8X
-
- Select 1 row (7864320 <= id <= 7864320):Best/Avg Time(ms) Rate(M/s)
Per Row(ns) Relative
-
-----------------------------------------------------------------------------------------------
- Parquet Vectorized 7939 / 8021 2.0
504.8 1.0X
- Parquet Vectorized (Pushdown) 746 / 770 21.1
47.4 10.6X
- Native ORC Vectorized 5690 / 5734 2.8
361.7 1.4X
- Native ORC Vectorized (Pushdown) 76 / 79 206.7
4.8 104.3X
-
- Select 1 row (7864319 < id < 7864321): Best/Avg Time(ms) Rate(M/s)
Per Row(ns) Relative
-
-----------------------------------------------------------------------------------------------
- Parquet Vectorized 7972 / 8019 2.0
506.9 1.0X
- Parquet Vectorized (Pushdown) 742 / 764 21.2
47.2 10.7X
- Native ORC Vectorized 5704 / 5743 2.8
362.6 1.4X
- Native ORC Vectorized (Pushdown) 76 / 78 207.9
4.8 105.4X
-
- Select 10% rows (id < 1572864): Best/Avg Time(ms) Rate(M/s)
Per Row(ns) Relative
-
-----------------------------------------------------------------------------------------------
- Parquet Vectorized 8733 / 8808 1.8
555.2 1.0X
- Parquet Vectorized (Pushdown) 2213 / 2267 7.1
140.7 3.9X
- Native ORC Vectorized 6420 / 6463 2.4
408.2 1.4X
- Native ORC Vectorized (Pushdown) 1313 / 1331 12.0
83.5 6.7X
-
- Select 50% rows (id < 7864320): Best/Avg Time(ms) Rate(M/s)
Per Row(ns) Relative
-
-----------------------------------------------------------------------------------------------
- Parquet Vectorized 11518 / 11591 1.4
732.3 1.0X
- Parquet Vectorized (Pushdown) 7962 / 7991 2.0
506.2 1.4X
- Native ORC Vectorized 8927 / 8985 1.8
567.6 1.3X
- Native ORC Vectorized (Pushdown) 6102 / 6160 2.6
387.9 1.9X
-
- Select 90% rows (id < 14155776): Best/Avg Time(ms) Rate(M/s)
Per Row(ns) Relative
-
-----------------------------------------------------------------------------------------------
- Parquet Vectorized 14255 / 14389 1.1
906.3 1.0X
- Parquet Vectorized (Pushdown) 13564 / 13594 1.2
862.4 1.1X
- Native ORC Vectorized 11442 / 11608 1.4
727.5 1.2X
- Native ORC Vectorized (Pushdown) 10991 / 11029 1.4
698.8 1.3X
-
- Select all rows (id IS NOT NULL): Best/Avg Time(ms) Rate(M/s)
Per Row(ns) Relative
-
-----------------------------------------------------------------------------------------------
- Parquet Vectorized 14917 / 14938 1.1
948.4 1.0X
- Parquet Vectorized (Pushdown) 14910 / 14964 1.1
948.0 1.0X
- Native ORC Vectorized 11986 / 12069 1.3
762.0 1.2X
- Native ORC Vectorized (Pushdown) 12037 / 12123 1.3
765.3 1.2X
-
- Select all rows (id > -1): Best/Avg Time(ms) Rate(M/s)
Per Row(ns) Relative
-
-----------------------------------------------------------------------------------------------
- Parquet Vectorized 14951 / 14976 1.1
950.6 1.0X
- Parquet Vectorized (Pushdown) 14934 / 15016 1.1
949.5 1.0X
- Native ORC Vectorized 12000 / 12156 1.3
763.0 1.2X
- Native ORC Vectorized (Pushdown) 12079 / 12113 1.3
767.9 1.2X
-
- Select all rows (id != -1): Best/Avg Time(ms) Rate(M/s)
Per Row(ns) Relative
-
-----------------------------------------------------------------------------------------------
- Parquet Vectorized 14930 / 14972 1.1
949.3 1.0X
- Parquet Vectorized (Pushdown) 15015 / 15047 1.0
954.6 1.0X
- Native ORC Vectorized 12090 / 12259 1.3
768.7 1.2X
- Native ORC Vectorized (Pushdown) 12021 / 12096 1.3
764.2 1.2X
+ Intel(R) Xeon(R) CPU E5-2686 v4 @ 2.30GHz
--- End diff --
ok, I used `m4.2xlarge`.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
