[
https://issues.apache.org/jira/browse/HIVE-22538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16992156#comment-16992156
]
Jesus Camacho Rodriguez commented on HIVE-22538:
------------------------------------------------
[~pvary], I did not have time to check this further. There is an enum
{{ReducerTraits}} in {{ReduceSinkDesc}} and an {{EnumSet reduceTraits}}. My
initial take would be to try to set that enum correctly whenever we create a RS
to enforce bucketing for ACID tables (we already pass {{writeType}} in the
constructor hence maybe it is not so difficult to set the traits at that time?
Or maybe from SemanticAnalyzer when the RS is created?). [~gopalv], does that
make sense? Other thoughts?
> RS deduplication does not always enforce
> hive.optimize.reducededuplication.min.reducer
> --------------------------------------------------------------------------------------
>
> Key: HIVE-22538
> URL: https://issues.apache.org/jira/browse/HIVE-22538
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
> Reporter: Jesus Camacho Rodriguez
> Assignee: Jesus Camacho Rodriguez
> Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22538.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> For transactional tables, that property might be overriden to 1, which can
> lead to merging final aggregation into a single stage (hence leading to
> performance degradation). For instance, when autogather column stats is
> enabled, this can happen for the following query:
> {code}
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> EXPLAIN
> CREATE TABLE x STORED AS ORC TBLPROPERTIES('transactional'='true') AS
> SELECT * FROM SRC x CLUSTER BY x.key;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
