Hello Michael Ho,

I'd like you to reexamine a change.  Please visit

    http://gerrit.cloudera.org:8080/4326

to look at the new patch set (#4).

Change subject: IMPALA-4008: don't bake in hash table and hash join pointers
......................................................................

IMPALA-4008: don't bake in hash table and hash join pointers

This fixes some of the cases where memory addresses are baked into
codegen'd code.

Testing:
Ran exhaustive build.

Perf:
Ran a local perf run. No significant changes. I was able to see some
small improvements on microbenchmarks.

    
+-----------+-----------------------+---------+------------+------------+----------------+
    | Workload  | File Format           | Avg (s) | Delta(Avg) | GeoMean(s) | 
Delta(GeoMean) |
    
+-----------+-----------------------+---------+------------+------------+----------------+
    | TPCH(_20) | parquet / none / none | 9.07    | +0.46%     | 5.88       | 
+0.34%         |
    
+-----------+-----------------------+---------+------------+------------+----------------+

    
+-----------+----------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+
    | Workload  | Query    | File Format           | Avg(s) | Base Avg(s) | 
Delta(Avg) | StdDev(%)  | Base StdDev(%) | Num Clients | Iters |
    
+-----------+----------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+
    | TPCH(_20) | TPCH-Q2  | parquet / none / none | 2.12   | 1.89        |   
+12.29%  | * 10.85% * | * 20.30% *     | 1           | 10    |
    | TPCH(_20) | TPCH-Q13 | parquet / none / none | 9.84   | 9.34        |   
+5.39%   |   9.01%    |   3.79%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q17 | parquet / none / none | 14.61  | 14.19       |   
+2.97%   |   2.15%    |   1.52%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q18 | parquet / none / none | 14.76  | 14.35       |   
+2.82%   |   3.20%    |   2.64%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q9  | parquet / none / none | 13.72  | 13.54       |   
+1.30%   |   1.75%    |   0.70%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q8  | parquet / none / none | 5.71   | 5.64        |   
+1.30%   |   1.21%    |   1.23%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q19 | parquet / none / none | 47.35  | 46.75       |   
+1.28%   |   2.39%    |   1.88%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q5  | parquet / none / none | 4.57   | 4.52        |   
+1.20%   |   1.30%    |   0.88%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q16 | parquet / none / none | 2.07   | 2.05        |   
+1.12%   |   2.59%    |   1.79%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q11 | parquet / none / none | 1.45   | 1.45        |   
+0.15%   |   2.69%    |   2.06%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q3  | parquet / none / none | 4.65   | 4.65        |   
-0.09%   |   2.12%    |   2.17%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q4  | parquet / none / none | 3.22   | 3.23        |   
-0.26%   |   1.03%    |   1.33%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q7  | parquet / none / none | 15.84  | 15.92       |   
-0.50%   |   0.91%    |   1.15%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q14 | parquet / none / none | 3.29   | 3.31        |   
-0.59%   |   3.31%    |   1.58%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q22 | parquet / none / none | 2.65   | 2.67        |   
-0.78%   |   3.03%    |   1.46%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q15 | parquet / none / none | 4.50   | 4.55        |   
-1.19%   |   2.87%    |   2.45%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q20 | parquet / none / none | 3.84   | 3.91        |   
-1.76%   |   2.20%    |   1.94%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q10 | parquet / none / none | 5.58   | 5.70        |   
-2.00%   |   1.01%    |   1.79%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q21 | parquet / none / none | 22.84  | 23.42       |   
-2.47%   |   0.68%    |   0.56%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q1  | parquet / none / none | 11.25  | 11.60       |   
-3.06%   |   0.48%    |   1.74%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q12 | parquet / none / none | 3.81   | 3.98        |   
-4.38%   |   1.62%    |   1.14%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q6  | parquet / none / none | 1.94   | 2.04        |   
-4.85%   |   2.40%    |   1.58%        | 1           | 10    |
    
+-----------+----------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+

    
+--------------------+-----------------------+---------+------------+------------+----------------+
    | Workload           | File Format           | Avg (s) | Delta(Avg) | 
GeoMean(s) | Delta(GeoMean) |
    
+--------------------+-----------------------+---------+------------+------------+----------------+
    | TARGETED-PERF(_20) | parquet / none / none | 8.17    | -1.66%     | 2.96  
     | -1.48%         |
    
+--------------------+-----------------------+---------+------------+------------+----------------+

    
+--------------------+--------------------------------------------------------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+
    | Workload           | Query                                                
  | File Format           | Avg(s) | Base Avg(s) | Delta(Avg) | StdDev(%)  | 
Base StdDev(%) | Num Clients | Iters |
    
+--------------------+--------------------------------------------------------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+
    | TARGETED-PERF(_20) | primitive_topn_bigint                                
  | parquet / none / none | 3.32   | 2.87        |   +15.31%  | * 17.22% * |   
1.64%        | 1           | 10    |
    | TARGETED-PERF(_20) | PERF_AGG-Q4                                          
  | parquet / none / none | 7.07   | 6.61        |   +6.93%   | * 15.59% * |   
5.08%        | 1           | 10    |
    | TARGETED-PERF(_20) | PERF_AGG-Q1                                          
  | parquet / none / none | 1.18   | 1.12        |   +5.57%   |   1.94%    |   
2.96%        | 1           | 10    |
    | TARGETED-PERF(_20) | primitive_empty_build_join_1                         
  | parquet / none / none | 10.75  | 10.47       |   +2.76%   |   1.15%    |   
0.94%        | 1           | 10    |
    | TARGETED-PERF(_20) | primitive_top-n_all                                  
  | parquet / none / none | 24.30  | 23.85       |   +1.87%   |   1.40%    |   
0.82%        | 1           | 10    |
    | TARGETED-PERF(_20) | primitive_broadcast_join_2                           
  | parquet / none / none | 2.45   | 2.42        |   +1.38%   |   1.93%    |   
1.33%        | 1           | 10    |
    | TARGETED-PERF(_20) | primitive_filter_bigint_selective                    
  | parquet / none / none | 0.57   | 0.57        |   +1.16%   |   3.66%    |   
4.19%        | 1           | 10    |
    | TARGETED-PERF(_20) | PERF_STRING-Q3                                       
  | parquet / none / none | 1.70   | 1.68        |   +1.06%   |   1.88%    |   
2.69%        | 1           | 10    |
    | TARGETED-PERF(_20) | primitive_broadcast_join_3                           
  | parquet / none / none | 4.15   | 4.13        |   +0.47%   |   1.27%    |   
1.28%        | 1           | 10    |
    | TARGETED-PERF(_20) | primitive_broadcast_join_1                           
  | parquet / none / none | 1.46   | 1.46        |   +0.32%   |   1.68%    |   
2.43%        | 1           | 10    |
    | TARGETED-PERF(_20) | primitive_orderby_all                                
  | parquet / none / none | 12.92  | 12.89       |   +0.22%   |   1.69%    |   
1.12%        | 1           | 10    |
    | TARGETED-PERF(_20) | PERF_STRING-Q4                                       
  | parquet / none / none | 1.68   | 1.67        |   +0.21%   |   2.31%    |   
2.44%        | 1           | 10    |
    | TARGETED-PERF(_20) | PERF_STRING-Q7                                       
  | parquet / none / none | 3.35   | 3.35        |   +0.14%   |   1.10%    |   
1.84%        | 1           | 10    |
    | TARGETED-PERF(_20) | primitive_conjunct_ordering_4                        
  | parquet / none / none | 0.46   | 0.46        |   +0.07%   |   0.29%    |   
0.27%        | 1           | 10    |
    | TARGETED-PERF(_20) | primitive_groupby_bigint_pk                          
  | parquet / none / none | 35.99  | 36.04       |   -0.13%   |   8.55%    |   
6.95%        | 1           | 10    |
    | TARGETED-PERF(_20) | PERF_AGG-Q6                                          
  | parquet / none / none | 0.97   | 0.97        |   -0.16%   |   2.68%    |   
2.66%        | 1           | 10    |
    | TARGETED-PERF(_20) | primitive_filter_decimal_selective                   
  | parquet / none / none | 0.84   | 0.84        |   -0.31%   |   2.85%    |   
3.58%        | 1           | 10    |
    | TARGETED-PERF(_20) | primitive_shuffle_join_union_all_with_groupby        
  | parquet / none / none | 20.46  | 20.53       |   -0.34%   |   0.60%    |   
0.60%        | 1           | 10    |
    | TARGETED-PERF(_20) | PERF_AGG-Q7                                          
  | parquet / none / none | 0.98   | 0.98        |   -0.44%   |   2.78%    |   
2.33%        | 1           | 10    |
    | TARGETED-PERF(_20) | primitive_exchange_shuffle                           
  | parquet / none / none | 25.21  | 25.34       |   -0.49%   |   1.94%    |   
1.30%        | 1           | 10    |
    | TARGETED-PERF(_20) | primitive_filter_string_like                         
  | parquet / none / none | 6.27   | 6.31        |   -0.56%   |   0.59%    |   
0.43%        | 1           | 10    |
    | TARGETED-PERF(_20) | PERF_STRING-Q5                                       
  | parquet / none / none | 1.96   | 1.97        |   -0.60%   |   1.98%    |   
1.25%        | 1           | 10    |
    | TARGETED-PERF(_20) | primitive_filter_bigint_non_selective                
  | parquet / none / none | 0.53   | 0.53        |   -0.71%   |   2.64%    |   
0.28%        | 1           | 10    |
    | TARGETED-PERF(_20) | primitive_exchange_broadcast                         
  | parquet / none / none | 14.77  | 14.97       |   -1.33%   |   2.59%    |   
2.39%        | 1           | 10    |
    | TARGETED-PERF(_20) | PERF_LIMIT-Q1                                        
  | parquet / none / none | 0.01   | 0.01        |   -1.40%   |   3.77%    |   
3.51%        | 1           | 10    |
    | TARGETED-PERF(_20) | primitive_orderby_bigint                             
  | parquet / none / none | 2.58   | 2.61        |   -1.44%   |   2.85%    |   
1.04%        | 1           | 10    |
    | TARGETED-PERF(_20) | PERF_STRING-Q1                                       
  | parquet / none / none | 1.47   | 1.49        |   -1.63%   |   2.79%    |   
1.62%        | 1           | 10    |
    | TARGETED-PERF(_20) | primitive_filter_decimal_non_selective               
  | parquet / none / none | 0.79   | 0.81        |   -1.65%   |   1.91%    |   
2.67%        | 1           | 10    |
    | TARGETED-PERF(_20) | PERF_STRING-Q2                                       
  | parquet / none / none | 1.56   | 1.58        |   -1.73%   |   2.86%    |   
2.57%        | 1           | 10    |
    | TARGETED-PERF(_20) | PERF_STRING-Q6                                       
  | parquet / none / none | 4.48   | 4.59        |   -2.22%   |   1.21%    |   
0.85%        | 1           | 10    |
    | TARGETED-PERF(_20) | primitive_conjunct_ordering_5                        
  | parquet / none / none | 11.80  | 12.10       |   -2.50%   |   3.06%    |   
2.09%        | 1           | 10    |
    | TARGETED-PERF(_20) | primitive_conjunct_ordering_1                        
  | parquet / none / none | 7.96   | 8.18        |   -2.58%   |   1.69%    |   
2.17%        | 1           | 10    |
    | TARGETED-PERF(_20) | 
primitive_shuffle_join_one_to_many_string_with_groupby | parquet / none / none 
| 93.11  | 95.78       |   -2.78%   |   1.07%    |   1.00%        | 1           
| 10    |
    | TARGETED-PERF(_20) | primitive_filter_string_selective                    
  | parquet / none / none | 0.88   | 0.91        |   -3.03%   |   4.68%    |   
5.34%        | 1           | 10    |
    | TARGETED-PERF(_20) | PERF_AGG-Q5                                          
  | parquet / none / none | 2.44   | 2.52        |   -3.35%   |   2.39%    |   
2.04%        | 1           | 10    |
    | TARGETED-PERF(_20) | primitive_groupby_bigint_highndv                     
  | parquet / none / none | 9.16   | 9.50        |   -3.59%   |   0.98%    |   
1.56%        | 1           | 10    |
    | TARGETED-PERF(_20) | primitive_conjunct_ordering_3                        
  | parquet / none / none | 1.44   | 1.50        |   -4.14%   |   1.10%    |   
1.08%        | 1           | 10    |
    | TARGETED-PERF(_20) | primitive_filter_string_non_selective                
  | parquet / none / none | 0.89   | 0.93        |   -4.31%   |   7.20%    |   
5.12%        | 1           | 10    |
    | TARGETED-PERF(_20) | primitive_groupby_decimal_highndv                    
  | parquet / none / none | 13.98  | 14.94       |   -6.40%   | * 11.24% * |   
8.66%        | 1           | 10    |
    | TARGETED-PERF(_20) | PERF_AGG-Q2                                          
  | parquet / none / none | 2.87   | 3.13        |   -8.36%   |   0.57%    |   
1.59%        | 1           | 10    |
    | TARGETED-PERF(_20) | primitive_groupby_bigint_lowndv                      
  | parquet / none / none | 1.48   | 1.62        |   -8.91%   |   1.62%    |   
1.28%        | 1           | 10    |
    | TARGETED-PERF(_20) | primitive_conjunct_ordering_2                        
  | parquet / none / none | 13.68  | 15.05       |   -9.14%   |   2.91%    |   
1.83%        | 1           | 10    |
    | TARGETED-PERF(_20) | primitive_groupby_decimal_lowndv.test                
  | parquet / none / none | 1.47   | 1.61        |   -9.26%   |   0.19%    |   
1.50%        | 1           | 10    |
    | TARGETED-PERF(_20) | PERF_AGG-Q3                                          
  | parquet / none / none | 4.25   | 4.82        |   -11.82%  |   0.79%    |   
0.84%        | 1           | 10    |
    
+--------------------+--------------------------------------------------------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+

Change-Id: Ie353666dbb5c958f0094d169306fe930ec3014c5
---
M be/src/exec/hash-table-test.cc
M be/src/exec/hash-table.cc
M be/src/exec/hash-table.h
M be/src/exec/hash-table.inline.h
M be/src/exec/partitioned-aggregation-node-ir.cc
M be/src/exec/partitioned-aggregation-node.cc
M be/src/exec/partitioned-hash-join-node-ir.cc
M be/src/exec/partitioned-hash-join-node.cc
8 files changed, 245 insertions(+), 265 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/26/4326/4
-- 
To view, visit http://gerrit.cloudera.org:8080/4326
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie353666dbb5c958f0094d169306fe930ec3014c5
Gerrit-PatchSet: 4
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong <tarmstr...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhe...@cloudera.com>
Gerrit-Reviewer: Michael Ho
Gerrit-Reviewer: Michael Ho <k...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>

Reply via email to