[
https://issues.apache.org/jira/browse/HIVE-12808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15166508#comment-15166508
]
Lefty Leverenz commented on HIVE-12808:
---------------------------------------
Doc note: This adds *hive.optimize.ppd.windowing* to HiveConf.java, so it will
need to be documented in Configuration Properties for the 2.1.0 release.
* [Configuration Properties -- Query and DDL Execution |
https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-QueryandDDLExecution]
I added a TODOC2.1 label.
> Logical PPD: Push filter clauses through PTF(Windowing) into TS
> ---------------------------------------------------------------
>
> Key: HIVE-12808
> URL: https://issues.apache.org/jira/browse/HIVE-12808
> Project: Hive
> Issue Type: Bug
> Components: CBO, Logical Optimizer
> Affects Versions: 1.2.1, 2.0.0
> Reporter: Gopal V
> Assignee: Laljo John Pullokkaran
> Labels: TODOC2.1
> Fix For: 2.1.0
>
> Attachments: HIVE-12808.01.patch, HIVE-12808.02.patch,
> HIVE-12808.03.patch, HIVE-12808.04.patch, HIVE-12808.05.patch
>
>
> Simplified repro case of [HCC
> #8880|https://community.hortonworks.com/questions/8880/hive-on-tez-pushdown-predicate-doesnt-work-in-part.html],
> with the slow query showing the push-down miss.
> And the manually rewritten query to indicate the expected one.
> Part of the problem could be the window range not being split apart for PPD,
> but the FIL is not pushed down even if the rownum filter is removed.
> {code}
> create temporary table positions (regionid string, id bigint, deviceid
> string, ts string);
> insert into positions values('1d6a0be1-6366-4692-9597-ebd5cd0f01d1',
> 1422792010, '6c5d1a30-2331-448b-a726-a380d6b3a432', '2016-01-01'),
> ('1d6a0be1-6366-4692-9597-ebd5cd0f01d1', 1422792010,
> '6c5d1a30-2331-448b-a726-a380d6b3a432', '2016-01-01'),
> ('1d6a0be1-6366-4692-9597-ebd5cd0f01d1', 1422792010,
> '6c5d1a30-2331-448b-a726-a380d6b3a432', '2016-01-02'),
> ('1d6a0be1-6366-4692-9597-ebd5cd0f01d1', 1422792010,
> '6c5d1a30-2331-448b-a726-a380d6b3a432', '2016-01-02');
> -- slow query
> explain
> WITH t1 AS
> (
> SELECT *,
> Row_number() over ( PARTITION BY regionid, id, deviceid
> ORDER BY ts DESC) AS rownos
> FROM positions ),
> latestposition as (
> SELECT *
> FROM t1
> WHERE rownos = 1)
> SELECT *
> FROM latestposition
> WHERE regionid='1d6a0be1-6366-4692-9597-ebd5cd0f01d1'
> AND id=1422792010
> AND deviceid='6c5d1a30-2331-448b-a726-a380d6b3a432';
> -- fast query
> explain
> WITH t1 AS
> (
> SELECT *,
> Row_number() over ( PARTITION BY regionid, id, deviceid
> ORDER BY ts DESC) AS rownos
> FROM positions
> WHERE regionid='1d6a0be1-6366-4692-9597-ebd5cd0f01d1'
> AND id=1422792010
> AND deviceid='6c5d1a30-2331-448b-a726-a380d6b3a432'
> ),latestposition as (
> SELECT *
> FROM t1
> WHERE rownos = 1)
> SELECT *
> FROM latestposition
> ;
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
