Modified: hive/trunk/ql/src/test/results/clientpositive/tez/vectorized_dynamic_partition_pruning.q.out URL: http://svn.apache.org/viewvc/hive/trunk/ql/src/test/results/clientpositive/tez/vectorized_dynamic_partition_pruning.q.out?rev=1641162&r1=1641161&r2=1641162&view=diff ============================================================================== --- hive/trunk/ql/src/test/results/clientpositive/tez/vectorized_dynamic_partition_pruning.q.out (original) +++ hive/trunk/ql/src/test/results/clientpositive/tez/vectorized_dynamic_partition_pruning.q.out Sun Nov 23 05:59:20 2014 @@ -441,8 +441,8 @@ STAGE PLANS: Stage: Stage-1 Tez Edges: - Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 6 (SIMPLE_EDGE) - Reducer 3 <- Map 5 (SIMPLE_EDGE), Reducer 2 (SIMPLE_EDGE) + Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE) + Reducer 3 <- Map 6 (SIMPLE_EDGE), Reducer 2 (SIMPLE_EDGE) Reducer 4 <- Reducer 3 (SIMPLE_EDGE) #### A masked pattern was here #### Vertices: @@ -460,19 +460,19 @@ STAGE PLANS: Map 5 Map Operator Tree: TableScan - alias: srcpart_hour - filterExpr: (hr is not null and (hour = 11)) (type: boolean) - Statistics: Num rows: 2 Data size: 344 Basic stats: COMPLETE Column stats: NONE + alias: srcpart_date + filterExpr: (ds is not null and (date = '2008-04-08')) (type: boolean) + Statistics: Num rows: 2 Data size: 376 Basic stats: COMPLETE Column stats: NONE Filter Operator - predicate: (hr is not null and (hour = 11)) (type: boolean) + predicate: (ds is not null and (date = '2008-04-08')) (type: boolean) Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Reduce Output Operator - key expressions: hr (type: string) + key expressions: ds (type: string) sort order: + - Map-reduce partition columns: hr (type: string) + Map-reduce partition columns: ds (type: string) Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Select Operator - expressions: hr (type: string) + expressions: ds (type: string) outputColumnNames: _col0 Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Group By Operator @@ -482,26 +482,27 @@ STAGE PLANS: Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Dynamic Partitioning Event Operator Target Input: srcpart - Partition key expr: hr + Partition key expr: ds Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE - Target column: hr + Target column: ds Target Vertex: Map 1 + Execution mode: vectorized Map 6 Map Operator Tree: TableScan - alias: srcpart_date - filterExpr: (ds is not null and (date = '2008-04-08')) (type: boolean) - Statistics: Num rows: 2 Data size: 376 Basic stats: COMPLETE Column stats: NONE + alias: srcpart_hour + filterExpr: (hr is not null and (hour = 11)) (type: boolean) + Statistics: Num rows: 2 Data size: 344 Basic stats: COMPLETE Column stats: NONE Filter Operator - predicate: (ds is not null and (date = '2008-04-08')) (type: boolean) + predicate: (hr is not null and (hour = 11)) (type: boolean) Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Reduce Output Operator - key expressions: ds (type: string) + key expressions: hr (type: string) sort order: + - Map-reduce partition columns: ds (type: string) + Map-reduce partition columns: hr (type: string) Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Select Operator - expressions: ds (type: string) + expressions: hr (type: string) outputColumnNames: _col0 Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Group By Operator @@ -511,11 +512,10 @@ STAGE PLANS: Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Dynamic Partitioning Event Operator Target Input: srcpart - Partition key expr: ds + Partition key expr: hr Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE - Target column: ds + Target column: hr Target Vertex: Map 1 - Execution mode: vectorized Reducer 2 Reduce Operator Tree: Merge Join Operator @@ -612,8 +612,8 @@ STAGE PLANS: Stage: Stage-1 Tez Edges: - Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 6 (SIMPLE_EDGE) - Reducer 3 <- Map 5 (SIMPLE_EDGE), Reducer 2 (SIMPLE_EDGE) + Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE) + Reducer 3 <- Map 6 (SIMPLE_EDGE), Reducer 2 (SIMPLE_EDGE) Reducer 4 <- Reducer 3 (SIMPLE_EDGE) #### A masked pattern was here #### Vertices: @@ -632,32 +632,32 @@ STAGE PLANS: Map 5 Map Operator Tree: TableScan - alias: srcpart_hour - filterExpr: (hr is not null and (hour = 11)) (type: boolean) - Statistics: Num rows: 2 Data size: 344 Basic stats: COMPLETE Column stats: NONE + alias: srcpart_date + filterExpr: (ds is not null and (date = '2008-04-08')) (type: boolean) + Statistics: Num rows: 2 Data size: 376 Basic stats: COMPLETE Column stats: NONE Filter Operator - predicate: (hr is not null and (hour = 11)) (type: boolean) + predicate: (ds is not null and (date = '2008-04-08')) (type: boolean) Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Reduce Output Operator - key expressions: hr (type: string) + key expressions: ds (type: string) sort order: + - Map-reduce partition columns: hr (type: string) + Map-reduce partition columns: ds (type: string) Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE + Execution mode: vectorized Map 6 Map Operator Tree: TableScan - alias: srcpart_date - filterExpr: (ds is not null and (date = '2008-04-08')) (type: boolean) - Statistics: Num rows: 2 Data size: 376 Basic stats: COMPLETE Column stats: NONE + alias: srcpart_hour + filterExpr: (hr is not null and (hour = 11)) (type: boolean) + Statistics: Num rows: 2 Data size: 344 Basic stats: COMPLETE Column stats: NONE Filter Operator - predicate: (ds is not null and (date = '2008-04-08')) (type: boolean) + predicate: (hr is not null and (hour = 11)) (type: boolean) Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Reduce Output Operator - key expressions: ds (type: string) + key expressions: hr (type: string) sort order: + - Map-reduce partition columns: ds (type: string) + Map-reduce partition columns: hr (type: string) Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE - Execution mode: vectorized Reducer 2 Reduce Operator Tree: Merge Join Operator @@ -1859,15 +1859,31 @@ STAGE PLANS: Stage: Stage-1 Tez Edges: - Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 5 (SIMPLE_EDGE) - Reducer 3 <- Reducer 2 (SIMPLE_EDGE) - Reducer 5 <- Map 4 (SIMPLE_EDGE) + Reducer 2 <- Map 1 (SIMPLE_EDGE) + Reducer 3 <- Map 5 (SIMPLE_EDGE), Reducer 2 (SIMPLE_EDGE) + Reducer 4 <- Reducer 3 (SIMPLE_EDGE) #### A masked pattern was here #### Vertices: Map 1 Map Operator Tree: TableScan alias: srcpart + filterExpr: (ds = '2008-04-08') (type: boolean) + Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE + Group By Operator + keys: ds (type: string) + mode: hash + outputColumnNames: _col0 + Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE + Reduce Output Operator + key expressions: _col0 (type: string) + sort order: + + Map-reduce partition columns: _col0 (type: string) + Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE + Map 5 + Map Operator Tree: + TableScan + alias: srcpart filterExpr: ds is not null (type: boolean) Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE Reduce Output Operator @@ -1889,25 +1905,22 @@ STAGE PLANS: Partition key expr: ds Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE Target column: ds - Target Vertex: Map 4 - Map 4 - Map Operator Tree: - TableScan - alias: srcpart - filterExpr: (ds = '2008-04-08') (type: boolean) - Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE - Group By Operator - keys: ds (type: string) - mode: hash - outputColumnNames: _col0 - Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: _col0 (type: string) - sort order: + - Map-reduce partition columns: _col0 (type: string) - Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE + Target Vertex: Map 1 Reducer 2 Reduce Operator Tree: + Group By Operator + keys: KEY._col0 (type: string) + mode: mergepartial + outputColumnNames: _col0 + Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE + Reduce Output Operator + key expressions: _col0 (type: string) + sort order: + + Map-reduce partition columns: _col0 (type: string) + Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE + Execution mode: vectorized + Reducer 3 + Reduce Operator Tree: Merge Join Operator condition map: Inner Join 0 to 1 @@ -1924,7 +1937,7 @@ STAGE PLANS: sort order: Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE value expressions: _col0 (type: bigint) - Reducer 3 + Reducer 4 Reduce Operator Tree: Group By Operator aggregations: count(VALUE._col0) @@ -1943,19 +1956,6 @@ STAGE PLANS: output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe Execution mode: vectorized - Reducer 5 - Reduce Operator Tree: - Group By Operator - keys: KEY._col0 (type: string) - mode: mergepartial - outputColumnNames: _col0 - Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: _col0 (type: string) - sort order: + - Map-reduce partition columns: _col0 (type: string) - Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE - Execution mode: vectorized Stage: Stage-0 Fetch Operator @@ -2354,16 +2354,6 @@ STAGE PLANS: Map 1 Map Operator Tree: TableScan - alias: srcpart - Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: ds (type: string) - sort order: + - Map-reduce partition columns: ds (type: string) - Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE - Map 4 - Map Operator Tree: - TableScan alias: srcpart_date filterExpr: (date = '2008-04-08') (type: boolean) Statistics: Num rows: 2 Data size: 376 Basic stats: COMPLETE Column stats: NONE @@ -2389,8 +2379,18 @@ STAGE PLANS: Partition key expr: ds Statistics: Num rows: 1 Data size: 188 Basic stats: COMPLETE Column stats: NONE Target column: ds - Target Vertex: Map 1 + Target Vertex: Map 4 Execution mode: vectorized + Map 4 + Map Operator Tree: + TableScan + alias: srcpart + Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE + Reduce Output Operator + key expressions: ds (type: string) + sort order: + + Map-reduce partition columns: ds (type: string) + Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE Reducer 2 Reduce Operator Tree: Merge Join Operator @@ -2541,8 +2541,8 @@ STAGE PLANS: Stage: Stage-1 Tez Edges: - Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 6 (SIMPLE_EDGE) - Reducer 3 <- Map 5 (SIMPLE_EDGE), Reducer 2 (SIMPLE_EDGE) + Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE) + Reducer 3 <- Map 6 (SIMPLE_EDGE), Reducer 2 (SIMPLE_EDGE) Reducer 4 <- Reducer 3 (SIMPLE_EDGE) #### A masked pattern was here #### Vertices: @@ -2560,18 +2560,19 @@ STAGE PLANS: Map 5 Map Operator Tree: TableScan - alias: srcpart_hour - filterExpr: ((hr is not null and (hour = 11)) and (hr = 11)) (type: boolean) - Statistics: Num rows: 2 Data size: 344 Basic stats: COMPLETE Column stats: NONE + alias: srcpart_date + filterExpr: (ds is not null and (date = '2008-04-08')) (type: boolean) + Statistics: Num rows: 2 Data size: 376 Basic stats: COMPLETE Column stats: NONE Filter Operator - predicate: ((hr is not null and (hour = 11)) and (hr = 11)) (type: boolean) + predicate: (ds is not null and (date = '2008-04-08')) (type: boolean) Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Reduce Output Operator - key expressions: '11' (type: string) + key expressions: ds (type: string) sort order: + + Map-reduce partition columns: ds (type: string) Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Select Operator - expressions: '11' (type: string) + expressions: ds (type: string) outputColumnNames: _col0 Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Group By Operator @@ -2581,26 +2582,26 @@ STAGE PLANS: Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Dynamic Partitioning Event Operator Target Input: srcpart - Partition key expr: hr + Partition key expr: ds Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE - Target column: hr + Target column: ds Target Vertex: Map 1 + Execution mode: vectorized Map 6 Map Operator Tree: TableScan - alias: srcpart_date - filterExpr: (ds is not null and (date = '2008-04-08')) (type: boolean) - Statistics: Num rows: 2 Data size: 376 Basic stats: COMPLETE Column stats: NONE + alias: srcpart_hour + filterExpr: ((hr is not null and (hour = 11)) and (hr = 11)) (type: boolean) + Statistics: Num rows: 2 Data size: 344 Basic stats: COMPLETE Column stats: NONE Filter Operator - predicate: (ds is not null and (date = '2008-04-08')) (type: boolean) + predicate: ((hr is not null and (hour = 11)) and (hr = 11)) (type: boolean) Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Reduce Output Operator - key expressions: ds (type: string) + key expressions: '11' (type: string) sort order: + - Map-reduce partition columns: ds (type: string) Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Select Operator - expressions: ds (type: string) + expressions: '11' (type: string) outputColumnNames: _col0 Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Group By Operator @@ -2610,11 +2611,10 @@ STAGE PLANS: Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Dynamic Partitioning Event Operator Target Input: srcpart - Partition key expr: ds + Partition key expr: hr Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE - Target column: ds + Target column: hr Target Vertex: Map 1 - Execution mode: vectorized Reducer 2 Reduce Operator Tree: Merge Join Operator @@ -2705,8 +2705,8 @@ STAGE PLANS: Stage: Stage-1 Tez Edges: - Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 6 (SIMPLE_EDGE) - Reducer 3 <- Map 5 (SIMPLE_EDGE), Reducer 2 (SIMPLE_EDGE) + Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE) + Reducer 3 <- Map 6 (SIMPLE_EDGE), Reducer 2 (SIMPLE_EDGE) Reducer 4 <- Reducer 3 (SIMPLE_EDGE) #### A masked pattern was here #### Vertices: @@ -2714,19 +2714,6 @@ STAGE PLANS: Map 5 Map Operator Tree: TableScan - alias: srcpart_hour - filterExpr: (hr = 13) (type: boolean) - Statistics: Num rows: 2 Data size: 344 Basic stats: COMPLETE Column stats: NONE - Filter Operator - predicate: (hr = 13) (type: boolean) - Statistics: Num rows: 1 Data size: 172 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: '13' (type: string) - sort order: + - Statistics: Num rows: 1 Data size: 172 Basic stats: COMPLETE Column stats: NONE - Map 6 - Map Operator Tree: - TableScan alias: srcpart_date filterExpr: (ds is not null and (date = '2008-04-08')) (type: boolean) Statistics: Num rows: 2 Data size: 376 Basic stats: COMPLETE Column stats: NONE @@ -2739,6 +2726,19 @@ STAGE PLANS: Map-reduce partition columns: ds (type: string) Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Execution mode: vectorized + Map 6 + Map Operator Tree: + TableScan + alias: srcpart_hour + filterExpr: (hr = 13) (type: boolean) + Statistics: Num rows: 2 Data size: 344 Basic stats: COMPLETE Column stats: NONE + Filter Operator + predicate: (hr = 13) (type: boolean) + Statistics: Num rows: 1 Data size: 172 Basic stats: COMPLETE Column stats: NONE + Reduce Output Operator + key expressions: '13' (type: string) + sort order: + + Statistics: Num rows: 1 Data size: 172 Basic stats: COMPLETE Column stats: NONE Reducer 2 Reduce Operator Tree: Merge Join Operator @@ -2825,23 +2825,34 @@ STAGE PLANS: Stage: Stage-1 Tez Edges: - Reducer 2 <- Map 1 (SIMPLE_EDGE), Union 3 (CONTAINS) - Reducer 4 <- Map 6 (SIMPLE_EDGE), Union 3 (SIMPLE_EDGE) - Reducer 5 <- Reducer 4 (SIMPLE_EDGE) - Reducer 8 <- Map 7 (SIMPLE_EDGE), Union 3 (CONTAINS) + Reducer 2 <- Map 1 (SIMPLE_EDGE), Union 6 (SIMPLE_EDGE) + Reducer 3 <- Reducer 2 (SIMPLE_EDGE) + Reducer 5 <- Map 4 (SIMPLE_EDGE), Union 6 (CONTAINS) + Reducer 8 <- Map 7 (SIMPLE_EDGE), Union 6 (CONTAINS) #### A masked pattern was here #### Vertices: Map 1 Map Operator Tree: TableScan alias: srcpart + filterExpr: ds is not null (type: boolean) + Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE + Reduce Output Operator + key expressions: ds (type: string) + sort order: + + Map-reduce partition columns: ds (type: string) + Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE + Map 4 + Map Operator Tree: + TableScan + alias: srcpart Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE Select Operator expressions: ds (type: string) outputColumnNames: ds Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE Group By Operator - aggregations: min(ds) + aggregations: max(ds) mode: hash outputColumnNames: _col0 Statistics: Num rows: 1 Data size: 84 Basic stats: COMPLETE Column stats: NONE @@ -2849,17 +2860,6 @@ STAGE PLANS: sort order: Statistics: Num rows: 1 Data size: 84 Basic stats: COMPLETE Column stats: NONE value expressions: _col0 (type: string) - Map 6 - Map Operator Tree: - TableScan - alias: srcpart - filterExpr: ds is not null (type: boolean) - Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: ds (type: string) - sort order: + - Map-reduce partition columns: ds (type: string) - Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE Map 7 Map Operator Tree: TableScan @@ -2870,7 +2870,7 @@ STAGE PLANS: outputColumnNames: ds Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE Group By Operator - aggregations: max(ds) + aggregations: min(ds) mode: hash outputColumnNames: _col0 Statistics: Num rows: 1 Data size: 84 Basic stats: COMPLETE Column stats: NONE @@ -2880,34 +2880,6 @@ STAGE PLANS: value expressions: _col0 (type: string) Reducer 2 Reduce Operator Tree: - Group By Operator - aggregations: min(VALUE._col0) - mode: mergepartial - outputColumnNames: _col0 - Filter Operator - predicate: _col0 is not null (type: boolean) - Group By Operator - keys: _col0 (type: string) - mode: hash - outputColumnNames: _col0 - Reduce Output Operator - key expressions: _col0 (type: string) - sort order: + - Map-reduce partition columns: _col0 (type: string) - Select Operator - expressions: _col0 (type: string) - outputColumnNames: _col0 - Group By Operator - keys: _col0 (type: string) - mode: hash - outputColumnNames: _col0 - Dynamic Partitioning Event Operator - Target Input: srcpart - Partition key expr: ds - Target column: ds - Target Vertex: Map 6 - Reducer 4 - Reduce Operator Tree: Merge Join Operator condition map: Left Semi Join 0 to 1 @@ -2924,7 +2896,7 @@ STAGE PLANS: sort order: Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE value expressions: _col0 (type: bigint) - Reducer 5 + Reducer 3 Reduce Operator Tree: Group By Operator aggregations: count(VALUE._col0) @@ -2943,7 +2915,7 @@ STAGE PLANS: output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe Execution mode: vectorized - Reducer 8 + Reducer 5 Reduce Operator Tree: Group By Operator aggregations: max(VALUE._col0) @@ -2970,9 +2942,37 @@ STAGE PLANS: Target Input: srcpart Partition key expr: ds Target column: ds - Target Vertex: Map 6 - Union 3 - Vertex: Union 3 + Target Vertex: Map 1 + Reducer 8 + Reduce Operator Tree: + Group By Operator + aggregations: min(VALUE._col0) + mode: mergepartial + outputColumnNames: _col0 + Filter Operator + predicate: _col0 is not null (type: boolean) + Group By Operator + keys: _col0 (type: string) + mode: hash + outputColumnNames: _col0 + Reduce Output Operator + key expressions: _col0 (type: string) + sort order: + + Map-reduce partition columns: _col0 (type: string) + Select Operator + expressions: _col0 (type: string) + outputColumnNames: _col0 + Group By Operator + keys: _col0 (type: string) + mode: hash + outputColumnNames: _col0 + Dynamic Partitioning Event Operator + Target Input: srcpart + Partition key expr: ds + Target column: ds + Target Vertex: Map 1 + Union 6 + Vertex: Union 6 Stage: Stage-0 Fetch Operator @@ -3009,23 +3009,34 @@ STAGE PLANS: Stage: Stage-1 Tez Edges: - Reducer 2 <- Map 1 (SIMPLE_EDGE), Union 3 (CONTAINS) - Reducer 4 <- Map 6 (SIMPLE_EDGE), Union 3 (SIMPLE_EDGE) - Reducer 5 <- Reducer 4 (SIMPLE_EDGE) - Reducer 8 <- Map 7 (SIMPLE_EDGE), Union 3 (CONTAINS) + Reducer 2 <- Map 1 (SIMPLE_EDGE), Union 6 (SIMPLE_EDGE) + Reducer 3 <- Reducer 2 (SIMPLE_EDGE) + Reducer 5 <- Map 4 (SIMPLE_EDGE), Union 6 (CONTAINS) + Reducer 8 <- Map 7 (SIMPLE_EDGE), Union 6 (CONTAINS) #### A masked pattern was here #### Vertices: Map 1 Map Operator Tree: TableScan alias: srcpart + filterExpr: ds is not null (type: boolean) + Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE + Reduce Output Operator + key expressions: ds (type: string) + sort order: + + Map-reduce partition columns: ds (type: string) + Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE + Map 4 + Map Operator Tree: + TableScan + alias: srcpart Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE Select Operator expressions: ds (type: string) outputColumnNames: ds Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE Group By Operator - aggregations: min(ds) + aggregations: max(ds) mode: hash outputColumnNames: _col0 Statistics: Num rows: 1 Data size: 84 Basic stats: COMPLETE Column stats: NONE @@ -3033,17 +3044,6 @@ STAGE PLANS: sort order: Statistics: Num rows: 1 Data size: 84 Basic stats: COMPLETE Column stats: NONE value expressions: _col0 (type: string) - Map 6 - Map Operator Tree: - TableScan - alias: srcpart - filterExpr: ds is not null (type: boolean) - Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: ds (type: string) - sort order: + - Map-reduce partition columns: ds (type: string) - Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE Map 7 Map Operator Tree: TableScan @@ -3054,7 +3054,7 @@ STAGE PLANS: outputColumnNames: ds Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE Group By Operator - aggregations: max(ds) + aggregations: min(ds) mode: hash outputColumnNames: _col0 Statistics: Num rows: 1 Data size: 84 Basic stats: COMPLETE Column stats: NONE @@ -3064,34 +3064,6 @@ STAGE PLANS: value expressions: _col0 (type: string) Reducer 2 Reduce Operator Tree: - Group By Operator - aggregations: min(VALUE._col0) - mode: mergepartial - outputColumnNames: _col0 - Filter Operator - predicate: _col0 is not null (type: boolean) - Group By Operator - keys: _col0 (type: string) - mode: hash - outputColumnNames: _col0 - Reduce Output Operator - key expressions: _col0 (type: string) - sort order: + - Map-reduce partition columns: _col0 (type: string) - Select Operator - expressions: _col0 (type: string) - outputColumnNames: _col0 - Group By Operator - keys: _col0 (type: string) - mode: hash - outputColumnNames: _col0 - Dynamic Partitioning Event Operator - Target Input: srcpart - Partition key expr: ds - Target column: ds - Target Vertex: Map 6 - Reducer 4 - Reduce Operator Tree: Merge Join Operator condition map: Left Semi Join 0 to 1 @@ -3110,7 +3082,7 @@ STAGE PLANS: sort order: + Map-reduce partition columns: _col0 (type: string) Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE - Reducer 5 + Reducer 3 Reduce Operator Tree: Group By Operator keys: KEY._col0 (type: string) @@ -3129,7 +3101,7 @@ STAGE PLANS: output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe Execution mode: vectorized - Reducer 8 + Reducer 5 Reduce Operator Tree: Group By Operator aggregations: max(VALUE._col0) @@ -3156,9 +3128,37 @@ STAGE PLANS: Target Input: srcpart Partition key expr: ds Target column: ds - Target Vertex: Map 6 - Union 3 - Vertex: Union 3 + Target Vertex: Map 1 + Reducer 8 + Reduce Operator Tree: + Group By Operator + aggregations: min(VALUE._col0) + mode: mergepartial + outputColumnNames: _col0 + Filter Operator + predicate: _col0 is not null (type: boolean) + Group By Operator + keys: _col0 (type: string) + mode: hash + outputColumnNames: _col0 + Reduce Output Operator + key expressions: _col0 (type: string) + sort order: + + Map-reduce partition columns: _col0 (type: string) + Select Operator + expressions: _col0 (type: string) + outputColumnNames: _col0 + Group By Operator + keys: _col0 (type: string) + mode: hash + outputColumnNames: _col0 + Dynamic Partitioning Event Operator + Target Input: srcpart + Partition key expr: ds + Target column: ds + Target Vertex: Map 1 + Union 6 + Vertex: Union 6 Stage: Stage-0 Fetch Operator @@ -3196,31 +3196,29 @@ STAGE PLANS: Stage: Stage-1 Tez Edges: - Reducer 11 <- Map 10 (SIMPLE_EDGE), Union 3 (CONTAINS) + Reducer 11 <- Map 10 (SIMPLE_EDGE), Union 9 (CONTAINS) Reducer 2 <- Map 1 (SIMPLE_EDGE), Union 3 (CONTAINS) - Reducer 4 <- Union 3 (SIMPLE_EDGE), Union 7 (SIMPLE_EDGE) - Reducer 6 <- Map 5 (SIMPLE_EDGE), Union 7 (CONTAINS) - Reducer 9 <- Map 8 (SIMPLE_EDGE), Union 7 (CONTAINS) + Reducer 4 <- Union 3 (SIMPLE_EDGE), Union 9 (SIMPLE_EDGE) + Reducer 6 <- Map 5 (SIMPLE_EDGE), Union 3 (CONTAINS) + Reducer 8 <- Map 7 (SIMPLE_EDGE), Union 9 (CONTAINS) #### A masked pattern was here #### Vertices: Map 1 Map Operator Tree: TableScan alias: srcpart + filterExpr: ds is not null (type: boolean) Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE - Select Operator - expressions: ds (type: string) - outputColumnNames: ds + Group By Operator + keys: ds (type: string) + mode: hash + outputColumnNames: _col0 Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE - Group By Operator - aggregations: min(ds) - mode: hash - outputColumnNames: _col0 - Statistics: Num rows: 1 Data size: 84 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - sort order: - Statistics: Num rows: 1 Data size: 84 Basic stats: COMPLETE Column stats: NONE - value expressions: _col0 (type: string) + Reduce Output Operator + key expressions: _col0 (type: string) + sort order: + + Map-reduce partition columns: _col0 (type: string) + Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE Map 10 Map Operator Tree: TableScan @@ -3231,7 +3229,7 @@ STAGE PLANS: outputColumnNames: ds Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE Group By Operator - aggregations: max(ds) + aggregations: min(ds) mode: hash outputColumnNames: _col0 Statistics: Num rows: 1 Data size: 84 Basic stats: COMPLETE Column stats: NONE @@ -3255,26 +3253,28 @@ STAGE PLANS: sort order: + Map-reduce partition columns: _col0 (type: string) Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE - Map 8 + Map 7 Map Operator Tree: TableScan alias: srcpart - filterExpr: ds is not null (type: boolean) Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE - Group By Operator - keys: ds (type: string) - mode: hash - outputColumnNames: _col0 + Select Operator + expressions: ds (type: string) + outputColumnNames: ds Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: _col0 (type: string) - sort order: + - Map-reduce partition columns: _col0 (type: string) - Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE + Group By Operator + aggregations: max(ds) + mode: hash + outputColumnNames: _col0 + Statistics: Num rows: 1 Data size: 84 Basic stats: COMPLETE Column stats: NONE + Reduce Output Operator + sort order: + Statistics: Num rows: 1 Data size: 84 Basic stats: COMPLETE Column stats: NONE + value expressions: _col0 (type: string) Reducer 11 Reduce Operator Tree: Group By Operator - aggregations: max(VALUE._col0) + aggregations: min(VALUE._col0) mode: mergepartial outputColumnNames: _col0 Filter Operator @@ -3298,7 +3298,7 @@ STAGE PLANS: Target Input: srcpart Partition key expr: ds Target column: ds - Target Vertex: Map 5 + Target Vertex: Map 1 Select Operator expressions: _col0 (type: string) outputColumnNames: _col0 @@ -3310,47 +3310,18 @@ STAGE PLANS: Target Input: srcpart Partition key expr: ds Target column: ds - Target Vertex: Map 8 + Target Vertex: Map 5 Reducer 2 Reduce Operator Tree: Group By Operator - aggregations: min(VALUE._col0) + keys: KEY._col0 (type: string) mode: mergepartial outputColumnNames: _col0 - Filter Operator - predicate: _col0 is not null (type: boolean) - Group By Operator - keys: _col0 (type: string) - mode: hash - outputColumnNames: _col0 - Reduce Output Operator - key expressions: _col0 (type: string) - sort order: + - Map-reduce partition columns: _col0 (type: string) - Select Operator - expressions: _col0 (type: string) - outputColumnNames: _col0 - Group By Operator - keys: _col0 (type: string) - mode: hash - outputColumnNames: _col0 - Dynamic Partitioning Event Operator - Target Input: srcpart - Partition key expr: ds - Target column: ds - Target Vertex: Map 5 - Select Operator - expressions: _col0 (type: string) - outputColumnNames: _col0 - Group By Operator - keys: _col0 (type: string) - mode: hash - outputColumnNames: _col0 - Dynamic Partitioning Event Operator - Target Input: srcpart - Partition key expr: ds - Target column: ds - Target Vertex: Map 8 + Reduce Output Operator + key expressions: _col0 (type: string) + sort order: + + Map-reduce partition columns: _col0 (type: string) + Execution mode: vectorized Reducer 4 Reduce Operator Tree: Merge Join Operator @@ -3383,21 +3354,50 @@ STAGE PLANS: sort order: + Map-reduce partition columns: _col0 (type: string) Execution mode: vectorized - Reducer 9 + Reducer 8 Reduce Operator Tree: Group By Operator - keys: KEY._col0 (type: string) + aggregations: max(VALUE._col0) mode: mergepartial outputColumnNames: _col0 - Reduce Output Operator - key expressions: _col0 (type: string) - sort order: + - Map-reduce partition columns: _col0 (type: string) - Execution mode: vectorized + Filter Operator + predicate: _col0 is not null (type: boolean) + Group By Operator + keys: _col0 (type: string) + mode: hash + outputColumnNames: _col0 + Reduce Output Operator + key expressions: _col0 (type: string) + sort order: + + Map-reduce partition columns: _col0 (type: string) + Select Operator + expressions: _col0 (type: string) + outputColumnNames: _col0 + Group By Operator + keys: _col0 (type: string) + mode: hash + outputColumnNames: _col0 + Dynamic Partitioning Event Operator + Target Input: srcpart + Partition key expr: ds + Target column: ds + Target Vertex: Map 1 + Select Operator + expressions: _col0 (type: string) + outputColumnNames: _col0 + Group By Operator + keys: _col0 (type: string) + mode: hash + outputColumnNames: _col0 + Dynamic Partitioning Event Operator + Target Input: srcpart + Partition key expr: ds + Target column: ds + Target Vertex: Map 5 Union 3 Vertex: Union 3 - Union 7 - Vertex: Union 7 + Union 9 + Vertex: Union 9 Stage: Stage-0 Fetch Operator @@ -3594,7 +3594,7 @@ STAGE PLANS: 1 ds (type: string) outputColumnNames: _col3 input vertices: - 1 Map 4 + 1 Map 3 Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE Map Join Operator condition map: @@ -3606,7 +3606,7 @@ STAGE PLANS: 0 _col3 (type: string) 1 hr (type: string) input vertices: - 1 Map 3 + 1 Map 4 Statistics: Num rows: 2420 Data size: 25709 Basic stats: COMPLETE Column stats: NONE Group By Operator aggregations: count() @@ -3620,19 +3620,19 @@ STAGE PLANS: Map 3 Map Operator Tree: TableScan - alias: srcpart_hour - filterExpr: (hr is not null and (hour = 11)) (type: boolean) - Statistics: Num rows: 2 Data size: 344 Basic stats: COMPLETE Column stats: NONE + alias: srcpart_date + filterExpr: (ds is not null and (date = '2008-04-08')) (type: boolean) + Statistics: Num rows: 2 Data size: 376 Basic stats: COMPLETE Column stats: NONE Filter Operator - predicate: (hr is not null and (hour = 11)) (type: boolean) + predicate: (ds is not null and (date = '2008-04-08')) (type: boolean) Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Reduce Output Operator - key expressions: hr (type: string) + key expressions: ds (type: string) sort order: + - Map-reduce partition columns: hr (type: string) + Map-reduce partition columns: ds (type: string) Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Select Operator - expressions: hr (type: string) + expressions: ds (type: string) outputColumnNames: _col0 Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Group By Operator @@ -3642,26 +3642,27 @@ STAGE PLANS: Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Dynamic Partitioning Event Operator Target Input: srcpart - Partition key expr: hr + Partition key expr: ds Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE - Target column: hr + Target column: ds Target Vertex: Map 1 + Execution mode: vectorized Map 4 Map Operator Tree: TableScan - alias: srcpart_date - filterExpr: (ds is not null and (date = '2008-04-08')) (type: boolean) - Statistics: Num rows: 2 Data size: 376 Basic stats: COMPLETE Column stats: NONE + alias: srcpart_hour + filterExpr: (hr is not null and (hour = 11)) (type: boolean) + Statistics: Num rows: 2 Data size: 344 Basic stats: COMPLETE Column stats: NONE Filter Operator - predicate: (ds is not null and (date = '2008-04-08')) (type: boolean) + predicate: (hr is not null and (hour = 11)) (type: boolean) Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Reduce Output Operator - key expressions: ds (type: string) + key expressions: hr (type: string) sort order: + - Map-reduce partition columns: ds (type: string) + Map-reduce partition columns: hr (type: string) Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Select Operator - expressions: ds (type: string) + expressions: hr (type: string) outputColumnNames: _col0 Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Group By Operator @@ -3671,11 +3672,10 @@ STAGE PLANS: Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Dynamic Partitioning Event Operator Target Input: srcpart - Partition key expr: ds + Partition key expr: hr Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE - Target column: ds + Target column: hr Target Vertex: Map 1 - Execution mode: vectorized Reducer 2 Reduce Operator Tree: Group By Operator @@ -4256,7 +4256,7 @@ STAGE PLANS: Stage: Stage-1 Tez Edges: - Map 1 <- Reducer 4 (BROADCAST_EDGE) + Map 3 <- Reducer 2 (BROADCAST_EDGE) Reducer 2 <- Map 1 (SIMPLE_EDGE) Reducer 4 <- Map 3 (SIMPLE_EDGE) #### A masked pattern was here #### @@ -4265,6 +4265,22 @@ STAGE PLANS: Map Operator Tree: TableScan alias: srcpart + filterExpr: (ds = '2008-04-08') (type: boolean) + Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE + Group By Operator + keys: ds (type: string) + mode: hash + outputColumnNames: _col0 + Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE + Reduce Output Operator + key expressions: _col0 (type: string) + sort order: + + Map-reduce partition columns: _col0 (type: string) + Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE + Map 3 + Map Operator Tree: + TableScan + alias: srcpart filterExpr: ds is not null (type: boolean) Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE Map Join Operator @@ -4277,7 +4293,7 @@ STAGE PLANS: 0 ds (type: string) 1 _col0 (type: string) input vertices: - 1 Reducer 4 + 1 Reducer 2 Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE Group By Operator aggregations: count() @@ -4288,44 +4304,9 @@ STAGE PLANS: sort order: Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE value expressions: _col0 (type: bigint) - Map 3 - Map Operator Tree: - TableScan - alias: srcpart - filterExpr: (ds = '2008-04-08') (type: boolean) - Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE - Group By Operator - keys: ds (type: string) - mode: hash - outputColumnNames: _col0 - Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: _col0 (type: string) - sort order: + - Map-reduce partition columns: _col0 (type: string) - Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE Reducer 2 Reduce Operator Tree: Group By Operator - aggregations: count(VALUE._col0) - mode: mergepartial - outputColumnNames: _col0 - Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE - Select Operator - expressions: _col0 (type: bigint) - outputColumnNames: _col0 - Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE - File Output Operator - compressed: false - Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE - table: - input format: org.apache.hadoop.mapred.TextInputFormat - output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat - serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe - Execution mode: vectorized - Reducer 4 - Reduce Operator Tree: - Group By Operator keys: KEY._col0 (type: string) mode: mergepartial outputColumnNames: _col0 @@ -4349,7 +4330,26 @@ STAGE PLANS: Partition key expr: ds Statistics: Num rows: 1000 Data size: 10624 Basic stats: COMPLETE Column stats: NONE Target column: ds - Target Vertex: Map 1 + Target Vertex: Map 3 + Reducer 4 + Reduce Operator Tree: + Group By Operator + aggregations: count(VALUE._col0) + mode: mergepartial + outputColumnNames: _col0 + Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE + Select Operator + expressions: _col0 (type: bigint) + outputColumnNames: _col0 + Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE + File Output Operator + compressed: false + Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE + table: + input format: org.apache.hadoop.mapred.TextInputFormat + output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat + serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe + Execution mode: vectorized Stage: Stage-0 Fetch Operator @@ -4487,23 +4487,13 @@ STAGE PLANS: Stage: Stage-1 Tez Edges: - Map 2 <- Map 1 (BROADCAST_EDGE) - Reducer 3 <- Map 2 (SIMPLE_EDGE) + Map 1 <- Map 3 (BROADCAST_EDGE) + Reducer 2 <- Map 1 (SIMPLE_EDGE) #### A masked pattern was here #### Vertices: Map 1 Map Operator Tree: TableScan - alias: srcpart - Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: ds (type: string) - sort order: + - Map-reduce partition columns: ds (type: string) - Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE - Map 2 - Map Operator Tree: - TableScan alias: srcpart_date filterExpr: (date = '2008-04-08') (type: boolean) Statistics: Num rows: 2 Data size: 376 Basic stats: COMPLETE Column stats: NONE @@ -4520,7 +4510,7 @@ STAGE PLANS: 0 ds (type: string) 1 ds (type: string) input vertices: - 1 Map 1 + 1 Map 3 Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE Group By Operator aggregations: count() @@ -4532,7 +4522,17 @@ STAGE PLANS: Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE value expressions: _col0 (type: bigint) Execution mode: vectorized - Reducer 3 + Map 3 + Map Operator Tree: + TableScan + alias: srcpart + Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE + Reduce Output Operator + key expressions: ds (type: string) + sort order: + + Map-reduce partition columns: ds (type: string) + Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE + Reducer 2 Reduce Operator Tree: Group By Operator aggregations: count(VALUE._col0) @@ -4684,7 +4684,7 @@ STAGE PLANS: 0 ds (type: string) 1 ds (type: string) input vertices: - 1 Map 4 + 1 Map 3 Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE Map Join Operator condition map: @@ -4696,7 +4696,7 @@ STAGE PLANS: 0 '11' (type: string) 1 '11' (type: string) input vertices: - 1 Map 3 + 1 Map 4 Statistics: Num rows: 1210 Data size: 12854 Basic stats: COMPLETE Column stats: NONE Group By Operator aggregations: count() @@ -4710,18 +4710,19 @@ STAGE PLANS: Map 3 Map Operator Tree: TableScan - alias: srcpart_hour - filterExpr: ((hr is not null and (hour = 11)) and (hr = 11)) (type: boolean) - Statistics: Num rows: 2 Data size: 344 Basic stats: COMPLETE Column stats: NONE + alias: srcpart_date + filterExpr: (ds is not null and (date = '2008-04-08')) (type: boolean) + Statistics: Num rows: 2 Data size: 376 Basic stats: COMPLETE Column stats: NONE Filter Operator - predicate: ((hr is not null and (hour = 11)) and (hr = 11)) (type: boolean) + predicate: (ds is not null and (date = '2008-04-08')) (type: boolean) Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Reduce Output Operator - key expressions: '11' (type: string) + key expressions: ds (type: string) sort order: + + Map-reduce partition columns: ds (type: string) Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Select Operator - expressions: '11' (type: string) + expressions: ds (type: string) outputColumnNames: _col0 Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Group By Operator @@ -4731,26 +4732,26 @@ STAGE PLANS: Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Dynamic Partitioning Event Operator Target Input: srcpart - Partition key expr: hr + Partition key expr: ds Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE - Target column: hr + Target column: ds Target Vertex: Map 1 + Execution mode: vectorized Map 4 Map Operator Tree: TableScan - alias: srcpart_date - filterExpr: (ds is not null and (date = '2008-04-08')) (type: boolean) - Statistics: Num rows: 2 Data size: 376 Basic stats: COMPLETE Column stats: NONE + alias: srcpart_hour + filterExpr: ((hr is not null and (hour = 11)) and (hr = 11)) (type: boolean) + Statistics: Num rows: 2 Data size: 344 Basic stats: COMPLETE Column stats: NONE Filter Operator - predicate: (ds is not null and (date = '2008-04-08')) (type: boolean) + predicate: ((hr is not null and (hour = 11)) and (hr = 11)) (type: boolean) Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Reduce Output Operator - key expressions: ds (type: string) + key expressions: '11' (type: string) sort order: + - Map-reduce partition columns: ds (type: string) Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Select Operator - expressions: ds (type: string) + expressions: '11' (type: string) outputColumnNames: _col0 Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Group By Operator @@ -4760,11 +4761,10 @@ STAGE PLANS: Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE Dynamic Partitioning Event Operator Target Input: srcpart - Partition key expr: ds + Partition key expr: hr Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE - Target column: ds + Target column: hr Target Vertex: Map 1 - Execution mode: vectorized Reducer 2 Reduce Operator Tree: Group By Operator @@ -4824,15 +4824,30 @@ STAGE PLANS: Stage: Stage-1 Tez Edges: - Map 1 <- Map 4 (BROADCAST_EDGE) - Map 2 <- Map 1 (BROADCAST_EDGE) - Reducer 3 <- Map 2 (SIMPLE_EDGE) + Map 1 <- Map 2 (BROADCAST_EDGE) + Map 3 <- Map 1 (BROADCAST_EDGE) + Reducer 4 <- Map 3 (SIMPLE_EDGE) #### A masked pattern was here #### Vertices: Map 1 Map 2 Map Operator Tree: TableScan + alias: srcpart_date + filterExpr: (ds is not null and (date = '2008-04-08')) (type: boolean) + Statistics: Num rows: 2 Data size: 376 Basic stats: COMPLETE Column stats: NONE + Filter Operator + predicate: (ds is not null and (date = '2008-04-08')) (type: boolean) + Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE + Reduce Output Operator + key expressions: ds (type: string) + sort order: + + Map-reduce partition columns: ds (type: string) + Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE + Execution mode: vectorized + Map 3 + Map Operator Tree: + TableScan alias: srcpart_hour filterExpr: (hr = 13) (type: boolean) Statistics: Num rows: 2 Data size: 344 Basic stats: COMPLETE Column stats: NONE @@ -4860,22 +4875,7 @@ STAGE PLANS: sort order: Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE value expressions: _col0 (type: bigint) - Map 4 - Map Operator Tree: - TableScan - alias: srcpart_date - filterExpr: (ds is not null and (date = '2008-04-08')) (type: boolean) - Statistics: Num rows: 2 Data size: 376 Basic stats: COMPLETE Column stats: NONE - Filter Operator - predicate: (ds is not null and (date = '2008-04-08')) (type: boolean) - Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE - Reduce Output Operator - key expressions: ds (type: string) - sort order: + - Map-reduce partition columns: ds (type: string) - Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE - Execution mode: vectorized - Reducer 3 + Reducer 4 Reduce Operator Tree: Group By Operator aggregations: count(VALUE._col0) @@ -4923,34 +4923,16 @@ STAGE PLANS: Stage: Stage-1 Tez Edges: - Map 4 <- Union 3 (BROADCAST_EDGE) - Reducer 2 <- Map 1 (SIMPLE_EDGE), Union 3 (CONTAINS) - Reducer 5 <- Map 4 (SIMPLE_EDGE) - Reducer 7 <- Map 6 (SIMPLE_EDGE), Union 3 (CONTAINS) + Map 1 <- Union 5 (BROADCAST_EDGE) + Reducer 2 <- Map 1 (SIMPLE_EDGE) + Reducer 4 <- Map 3 (SIMPLE_EDGE), Union 5 (CONTAINS) + Reducer 7 <- Map 6 (SIMPLE_EDGE), Union 5 (CONTAINS) #### A masked pattern was here #### Vertices: Map 1 Map Operator Tree: TableScan alias: srcpart - Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE - Select Operator - expressions: ds (type: string) - outputColumnNames: ds - Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE - Group By Operator - aggregations: min(ds) - mode: hash - outputColumnNames: _col0 - Statistics: Num rows: 1 Data size: 84 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - sort order: - Statistics: Num rows: 1 Data size: 84 Basic stats: COMPLETE Column stats: NONE - value expressions: _col0 (type: string) - Map 4 - Map Operator Tree: - TableScan - alias: srcpart filterExpr: ds is not null (type: boolean) Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE Map Join Operator @@ -4964,7 +4946,7 @@ STAGE PLANS: 1 _col0 (type: string) outputColumnNames: _col2 input vertices: - 1 Union 3 + 1 Union 5 Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE Group By Operator keys: _col2 (type: string) @@ -4976,7 +4958,7 @@ STAGE PLANS: sort order: + Map-reduce partition columns: _col0 (type: string) Statistics: Num rows: 2200 Data size: 23372 Basic stats: COMPLETE Column stats: NONE - Map 6 + Map 3 Map Operator Tree: TableScan alias: srcpart @@ -4994,10 +4976,47 @@ STAGE PLANS: sort order: Statistics: Num rows: 1 Data size: 84 Basic stats: COMPLETE Column stats: NONE value expressions: _col0 (type: string) + Map 6 + Map Operator Tree: + TableScan + alias: srcpart + Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE + Select Operator + expressions: ds (type: string) + outputColumnNames: ds + Statistics: Num rows: 2000 Data size: 21248 Basic stats: COMPLETE Column stats: NONE + Group By Operator + aggregations: min(ds) + mode: hash + outputColumnNames: _col0 + Statistics: Num rows: 1 Data size: 84 Basic stats: COMPLETE Column stats: NONE + Reduce Output Operator + sort order: + Statistics: Num rows: 1 Data size: 84 Basic stats: COMPLETE Column stats: NONE + value expressions: _col0 (type: string) Reducer 2 Reduce Operator Tree: Group By Operator - aggregations: min(VALUE._col0) + keys: KEY._col0 (type: string) + mode: mergepartial + outputColumnNames: _col0 + Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE + Select Operator + expressions: _col0 (type: string) + outputColumnNames: _col0 + Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE + File Output Operator + compressed: false + Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE + table: + input format: org.apache.hadoop.mapred.TextInputFormat + output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat + serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe + Execution mode: vectorized + Reducer 4 + Reduce Operator Tree: + Group By Operator + aggregations: max(VALUE._col0) mode: mergepartial outputColumnNames: _col0 Filter Operator @@ -5021,30 +5040,11 @@ STAGE PLANS: Target Input: srcpart Partition key expr: ds Target column: ds - Target Vertex: Map 4 - Reducer 5 - Reduce Operator Tree: - Group By Operator - keys: KEY._col0 (type: string) - mode: mergepartial - outputColumnNames: _col0 - Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE - Select Operator - expressions: _col0 (type: string) - outputColumnNames: _col0 - Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE - File Output Operator - compressed: false - Statistics: Num rows: 1100 Data size: 11686 Basic stats: COMPLETE Column stats: NONE - table: - input format: org.apache.hadoop.mapred.TextInputFormat - output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat - serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe - Execution mode: vectorized + Target Vertex: Map 1 Reducer 7 Reduce Operator Tree: Group By Operator - aggregations: max(VALUE._col0) + aggregations: min(VALUE._col0) mode: mergepartial outputColumnNames: _col0 Filter Operator @@ -5068,9 +5068,9 @@ STAGE PLANS: Target Input: srcpart Partition key expr: ds Target column: ds - Target Vertex: Map 4 - Union 3 - Vertex: Union 3 + Target Vertex: Map 1 + Union 5 + Vertex: Union 5 Stage: Stage-0 Fetch Operator
Modified: hive/trunk/ql/src/test/results/clientpositive/tez/vectorized_mapjoin.q.out URL: http://svn.apache.org/viewvc/hive/trunk/ql/src/test/results/clientpositive/tez/vectorized_mapjoin.q.out?rev=1641162&r1=1641161&r2=1641162&view=diff ============================================================================== --- hive/trunk/ql/src/test/results/clientpositive/tez/vectorized_mapjoin.q.out (original) +++ hive/trunk/ql/src/test/results/clientpositive/tez/vectorized_mapjoin.q.out Sun Nov 23 05:59:20 2014 @@ -14,27 +14,13 @@ STAGE PLANS: Stage: Stage-1 Tez Edges: - Map 2 <- Map 1 (BROADCAST_EDGE) - Reducer 3 <- Map 2 (SIMPLE_EDGE) + Map 1 <- Map 3 (BROADCAST_EDGE) + Reducer 2 <- Map 1 (SIMPLE_EDGE) #### A masked pattern was here #### Vertices: Map 1 Map Operator Tree: TableScan - alias: t2 - Statistics: Num rows: 12288 Data size: 377237 Basic stats: COMPLETE Column stats: NONE - Filter Operator - predicate: cint is not null (type: boolean) - Statistics: Num rows: 6144 Data size: 188618 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - key expressions: cint (type: int) - sort order: + - Map-reduce partition columns: cint (type: int) - Statistics: Num rows: 6144 Data size: 188618 Basic stats: COMPLETE Column stats: NONE - Execution mode: vectorized - Map 2 - Map Operator Tree: - TableScan alias: t1 Statistics: Num rows: 12288 Data size: 377237 Basic stats: COMPLETE Column stats: NONE Filter Operator @@ -51,7 +37,7 @@ STAGE PLANS: 1 cint (type: int) outputColumnNames: _col2, _col17 input vertices: - 1 Map 1 + 1 Map 3 Statistics: Num rows: 6758 Data size: 207479 Basic stats: COMPLETE Column stats: NONE Group By Operator aggregations: count(_col2), max(_col17), min(_col2), avg((_col2 + _col17)) @@ -63,7 +49,21 @@ STAGE PLANS: Statistics: Num rows: 1 Data size: 16 Basic stats: COMPLETE Column stats: NONE value expressions: _col0 (type: bigint), _col1 (type: int), _col2 (type: int), _col3 (type: struct<count:bigint,sum:double,input:int>) Execution mode: vectorized - Reducer 3 + Map 3 + Map Operator Tree: + TableScan + alias: t2 + Statistics: Num rows: 12288 Data size: 377237 Basic stats: COMPLETE Column stats: NONE + Filter Operator + predicate: cint is not null (type: boolean) + Statistics: Num rows: 6144 Data size: 188618 Basic stats: COMPLETE Column stats: NONE + Reduce Output Operator + key expressions: cint (type: int) + sort order: + + Map-reduce partition columns: cint (type: int) + Statistics: Num rows: 6144 Data size: 188618 Basic stats: COMPLETE Column stats: NONE + Execution mode: vectorized + Reducer 2 Reduce Operator Tree: Group By Operator aggregations: count(VALUE._col0), max(VALUE._col1), min(VALUE._col2), avg(VALUE._col3) Modified: hive/trunk/ql/src/test/results/clientpositive/tez/vectorized_nested_mapjoin.q.out URL: http://svn.apache.org/viewvc/hive/trunk/ql/src/test/results/clientpositive/tez/vectorized_nested_mapjoin.q.out?rev=1641162&r1=1641161&r2=1641162&view=diff ============================================================================== --- hive/trunk/ql/src/test/results/clientpositive/tez/vectorized_nested_mapjoin.q.out (original) +++ hive/trunk/ql/src/test/results/clientpositive/tez/vectorized_nested_mapjoin.q.out Sun Nov 23 05:59:20 2014 @@ -10,45 +10,14 @@ STAGE PLANS: Stage: Stage-1 Tez Edges: - Map 1 <- Map 4 (BROADCAST_EDGE) - Map 4 <- Map 3 (BROADCAST_EDGE) - Reducer 2 <- Map 1 (SIMPLE_EDGE) + Map 2 <- Map 1 (BROADCAST_EDGE) + Map 3 <- Map 2 (BROADCAST_EDGE) + Reducer 4 <- Map 3 (SIMPLE_EDGE) #### A masked pattern was here #### Vertices: Map 1 Map Operator Tree: TableScan - alias: v3 - Statistics: Num rows: 12288 Data size: 377237 Basic stats: COMPLETE Column stats: NONE - Filter Operator - predicate: csmallint is not null (type: boolean) - Statistics: Num rows: 6144 Data size: 188618 Basic stats: COMPLETE Column stats: NONE - Map Join Operator - condition map: - Inner Join 0 to 1 - condition expressions: - 0 {_col1} - 1 - keys: - 0 _col0 (type: smallint) - 1 csmallint (type: smallint) - outputColumnNames: _col1 - input vertices: - 0 Map 4 - Statistics: Num rows: 6758 Data size: 207479 Basic stats: COMPLETE Column stats: NONE - Group By Operator - aggregations: sum(_col1) - mode: hash - outputColumnNames: _col0 - Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE - Reduce Output Operator - sort order: - Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE - value expressions: _col0 (type: double) - Execution mode: vectorized - Map 3 - Map Operator Tree: - TableScan alias: v1 Statistics: Num rows: 12288 Data size: 377237 Basic stats: COMPLETE Column stats: NONE Filter Operator @@ -61,7 +30,7 @@ STAGE PLANS: Statistics: Num rows: 3072 Data size: 94309 Basic stats: COMPLETE Column stats: NONE value expressions: csmallint (type: smallint), cdouble (type: double) Execution mode: vectorized - Map 4 + Map 2 Map Operator Tree: TableScan alias: v2 @@ -80,7 +49,7 @@ STAGE PLANS: 1 ctinyint (type: tinyint) outputColumnNames: _col0, _col1, _col5, _col15 input vertices: - 0 Map 3 + 0 Map 1 Statistics: Num rows: 6758 Data size: 207479 Basic stats: COMPLETE Column stats: NONE Filter Operator predicate: (_col0 = _col15) (type: boolean) @@ -96,7 +65,38 @@ STAGE PLANS: Statistics: Num rows: 3379 Data size: 103739 Basic stats: COMPLETE Column stats: NONE value expressions: _col1 (type: double) Execution mode: vectorized - Reducer 2 + Map 3 + Map Operator Tree: + TableScan + alias: v3 + Statistics: Num rows: 12288 Data size: 377237 Basic stats: COMPLETE Column stats: NONE + Filter Operator + predicate: csmallint is not null (type: boolean) + Statistics: Num rows: 6144 Data size: 188618 Basic stats: COMPLETE Column stats: NONE + Map Join Operator + condition map: + Inner Join 0 to 1 + condition expressions: + 0 {_col1} + 1 + keys: + 0 _col0 (type: smallint) + 1 csmallint (type: smallint) + outputColumnNames: _col1 + input vertices: + 0 Map 2 + Statistics: Num rows: 6758 Data size: 207479 Basic stats: COMPLETE Column stats: NONE + Group By Operator + aggregations: sum(_col1) + mode: hash + outputColumnNames: _col0 + Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE + Reduce Output Operator + sort order: + Statistics: Num rows: 1 Data size: 8 Basic stats: COMPLETE Column stats: NONE + value expressions: _col0 (type: double) + Execution mode: vectorized + Reducer 4 Reduce Operator Tree: Group By Operator aggregations: sum(VALUE._col0) Modified: hive/trunk/ql/src/test/results/clientpositive/tez/vectorized_ptf.q.out URL: http://svn.apache.org/viewvc/hive/trunk/ql/src/test/results/clientpositive/tez/vectorized_ptf.q.out?rev=1641162&r1=1641161&r2=1641162&view=diff ============================================================================== Files hive/trunk/ql/src/test/results/clientpositive/tez/vectorized_ptf.q.out (original) and hive/trunk/ql/src/test/results/clientpositive/tez/vectorized_ptf.q.out Sun Nov 23 05:59:20 2014 differ Modified: hive/trunk/ql/src/test/results/clientpositive/tez/vectorized_shufflejoin.q.out URL: http://svn.apache.org/viewvc/hive/trunk/ql/src/test/results/clientpositive/tez/vectorized_shufflejoin.q.out?rev=1641162&r1=1641161&r2=1641162&view=diff ============================================================================== --- hive/trunk/ql/src/test/results/clientpositive/tez/vectorized_shufflejoin.q.out (original) +++ hive/trunk/ql/src/test/results/clientpositive/tez/vectorized_shufflejoin.q.out Sun Nov 23 05:59:20 2014 @@ -21,7 +21,7 @@ STAGE PLANS: Map 1 Map Operator Tree: TableScan - alias: t2 + alias: t1 Statistics: Num rows: 12288 Data size: 377237 Basic stats: COMPLETE Column stats: NONE Filter Operator predicate: cint is not null (type: boolean) @@ -35,7 +35,7 @@ STAGE PLANS: Map 4 Map Operator Tree: TableScan - alias: t1 + alias: t2 Statistics: Num rows: 12288 Data size: 377237 Basic stats: COMPLETE Column stats: NONE Filter Operator predicate: cint is not null (type: boolean) Modified: hive/trunk/ql/src/test/results/clientpositive/udf_case_column_pruning.q.out URL: http://svn.apache.org/viewvc/hive/trunk/ql/src/test/results/clientpositive/udf_case_column_pruning.q.out?rev=1641162&r1=1641161&r2=1641162&view=diff ============================================================================== --- hive/trunk/ql/src/test/results/clientpositive/udf_case_column_pruning.q.out (original) +++ hive/trunk/ql/src/test/results/clientpositive/udf_case_column_pruning.q.out Sun Nov 23 05:59:20 2014 @@ -28,7 +28,7 @@ STAGE PLANS: Map Reduce Map Operator Tree: TableScan - alias: b + alias: a Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE Filter Operator predicate: key is not null (type: boolean) @@ -39,7 +39,7 @@ STAGE PLANS: Map-reduce partition columns: key (type: string) Statistics: Num rows: 250 Data size: 2656 Basic stats: COMPLETE Column stats: NONE TableScan - alias: a + alias: b Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE Column stats: NONE Filter Operator predicate: key is not null (type: boolean)
