[ https://issues.apache.org/jira/browse/HIVE-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131223#comment-14131223 ]
Gopal V commented on HIVE-7156: ------------------------------- I just re-ran my test with this patch applied & the group-by still increases the row-count when applied? {code} hive> explain select distinct L_SHIPDATE from lineitem; STAGE PLANS: Stage: Stage-1 Tez Edges: Reducer 2 <- Map 1 (SIMPLE_EDGE) DagName: gopal_20140912004141_1f23f948-7852-4882-9f3e-1810904988b8:1 Vertices: Map 1 Map Operator Tree: TableScan alias: lineitem Statistics: Num rows: 5999989709 Data size: 4833087637230 Basic stats: COMPLETE Column stats: COMPLETE Select Operator expressions: l_shipdate (type: string) outputColumnNames: l_shipdate Statistics: Num rows: 5999989709 Data size: 4833087637230 Basic stats: COMPLETE Column stats: COMPLETE Group By Operator keys: l_shipdate (type: string) mode: hash outputColumnNames: _col0 Statistics: Num rows: 113279805705920 Data size: 10648301736356480 Basic stats: COMPLETE Column stats: COMPLETE Reduce Output Operator key expressions: _col0 (type: string) sort order: + Map-reduce partition columns: _col0 (type: string) Statistics: Num rows: 113279805705920 Data size: 10648301736356480 Basic stats: COMPLETE Column stats: COMPLETE Execution mode: vectorized {code} > Group-By operator stat-annotation only uses distinct approx to generate > rollups > ------------------------------------------------------------------------------- > > Key: HIVE-7156 > URL: https://issues.apache.org/jira/browse/HIVE-7156 > Project: Hive > Issue Type: Sub-task > Affects Versions: 0.14.0 > Reporter: Gopal V > Assignee: Prasanth J > Attachments: HIVE-7156.1.patch, HIVE-7156.2.patch, HIVE-7156.3.patch, > HIVE-7156.4.patch > > > The stats annotation for a group-by only annotates the reduce-side row-count > with the distinct values. > The map-side gets the row-count as the rows output instead of distinct * > parallelism, while the reducer side gets the correct parallelism. > {code} > hive> explain select distinct L_SHIPDATE from lineitem; > Vertices: > Map 1 > Map Operator Tree: > TableScan > alias: lineitem > Statistics: Num rows: 5999989709 Data size: 4745677733354 > Basic stats: COMPLETE Column stats: COMPLETE > Select Operator > expressions: l_shipdate (type: string) > outputColumnNames: l_shipdate > Statistics: Num rows: 5999989709 Data size: 4745677733354 > Basic stats: COMPLETE Column stats: COMPLETE > Group By Operator > keys: l_shipdate (type: string) > mode: hash > outputColumnNames: _col0 > Statistics: Num rows: 5999989709 Data size: > 563999032646 Basic stats: COMPLETE Column stats: COMPLETE > Reduce Output Operator > key expressions: _col0 (type: string) > sort order: + > Map-reduce partition columns: _col0 (type: string) > Statistics: Num rows: 5999989709 Data size: > 563999032646 Basic stats: COMPLETE Column stats: COMPLETE > Execution mode: vectorized > Reducer 2 > Reduce Operator Tree: > Group By Operator > keys: KEY._col0 (type: string) > mode: mergepartial > outputColumnNames: _col0 > Statistics: Num rows: 1955 Data size: 183770 Basic stats: > COMPLETE Column stats: COMPLETE > Select Operator > expressions: _col0 (type: string) > outputColumnNames: _col0 > Statistics: Num rows: 1955 Data size: 183770 Basic stats: > COMPLETE Column stats: COMPLETE > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)