[
https://issues.apache.org/jira/browse/HIVE-10419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15346621#comment-15346621
]
Neil Best commented on HIVE-10419:
----------------------------------
It appears to me that using an analytic function in a view defeats partition
pruning of the underlying table somehow. Consider the following which may be
helpful:
{noformat}
0: jdbc:hive2://pco-ph-gbdm-03:10000> set hive.mapred.mode;
+-----------------------------+--+
| set |
+-----------------------------+--+
| hive.mapred.mode=nonstrict |
+-----------------------------+--+
0: jdbc:hive2://pco-ph-gbdm-03:10000> drop view if exists sla.minute_order_v;
No rows affected (0.105 seconds)
0: jdbc:hive2://pco-ph-gbdm-03:10000>
0: jdbc:hive2://pco-ph-gbdm-03:10000> create view if not exists
sla.minute_order_v
0: jdbc:hive2://pco-ph-gbdm-03:10000> as select
0: jdbc:hive2://pco-ph-gbdm-03:10000> flight_id,
0: jdbc:hive2://pco-ph-gbdm-03:10000> tail,
0: jdbc:hive2://pco-ph-gbdm-03:10000> acpu_time,
0: jdbc:hive2://pco-ph-gbdm-03:10000> device,
0: jdbc:hive2://pco-ph-gbdm-03:10000> alt_m,
0: jdbc:hive2://pco-ph-gbdm-03:10000> agl_ft,
0: jdbc:hive2://pco-ph-gbdm-03:10000> row_number() over (
0: jdbc:hive2://pco-ph-gbdm-03:10000> partition by flight_source,
flight_id, device
0: jdbc:hive2://pco-ph-gbdm-03:10000> order by acpu_time) asc_order,
0: jdbc:hive2://pco-ph-gbdm-03:10000> row_number() over (
0: jdbc:hive2://pco-ph-gbdm-03:10000> partition by flight_source,
flight_id, device
0: jdbc:hive2://pco-ph-gbdm-03:10000> order by acpu_time desc) desc_order,
0: jdbc:hive2://pco-ph-gbdm-03:10000> year,
0: jdbc:hive2://pco-ph-gbdm-03:10000> month,
0: jdbc:hive2://pco-ph-gbdm-03:10000> day,
0: jdbc:hive2://pco-ph-gbdm-03:10000> flight_source
0: jdbc:hive2://pco-ph-gbdm-03:10000> from
0: jdbc:hive2://pco-ph-gbdm-03:10000> sla.minute
0: jdbc:hive2://pco-ph-gbdm-03:10000> where
0: jdbc:hive2://pco-ph-gbdm-03:10000> agl_ft >= 10000
0: jdbc:hive2://pco-ph-gbdm-03:10000> and device = 'sla';
No rows affected (0.103 seconds)
0: jdbc:hive2://pco-ph-gbdm-03:10000> explain authorization select * from
minute_order_v where year = 2016 and month = 5 and day = 23 and flight_source =
'gdw';
+--------------------------------------------------------------------------------------------------------------------------------------------+--+
| Explain
|
+--------------------------------------------------------------------------------------------------------------------------------------------+--+
| INPUTS:
|
| sla@minute_order_v
|
| sla@minute
|
| sla@minute@year=2016/month=5/day=23/flight_source=gdw
|
| sla@minute@year=2016/month=5/day=23/flight_source=periodic
|
| sla@minute@year=2016/month=5/day=24/flight_source=gdw
|
| sla@minute@year=2016/month=5/day=24/flight_source=periodic
|
| OUTPUTS:
|
|
hdfs://nameservice1/tmp/hive/etl-user/0d2d8c59-1dbe-4fb0-8cde-478ff33841de/hive_2016-06-23_15-02-16_343_5980077610772883275-6/-mr-10000
|
| CURRENT_USER:
|
| etl-user
|
| OPERATION:
|
| CREATEVIEW
|
| AUTHORIZATION_FAILURES:
|
| No privilege 'Select' found for inputs { database:sla,
table:minute_order_v}
|
| No privilege 'Select' found for inputs { database:sla, table:minute}
|
+--------------------------------------------------------------------------------------------------------------------------------------------+--+
16 rows selected (0.269 seconds)
0: jdbc:hive2://pco-ph-gbdm-03:10000> drop view if exists sla.minute_order_v;
No rows affected (0.07 seconds)
0: jdbc:hive2://pco-ph-gbdm-03:10000>
0: jdbc:hive2://pco-ph-gbdm-03:10000> create view if not exists
sla.minute_order_v
0: jdbc:hive2://pco-ph-gbdm-03:10000> as select
0: jdbc:hive2://pco-ph-gbdm-03:10000> flight_id,
0: jdbc:hive2://pco-ph-gbdm-03:10000> tail,
0: jdbc:hive2://pco-ph-gbdm-03:10000> acpu_time,
0: jdbc:hive2://pco-ph-gbdm-03:10000> device,
0: jdbc:hive2://pco-ph-gbdm-03:10000> alt_m,
0: jdbc:hive2://pco-ph-gbdm-03:10000> agl_ft,
0: jdbc:hive2://pco-ph-gbdm-03:10000> -- row_number() over (
0: jdbc:hive2://pco-ph-gbdm-03:10000> -- partition by flight_source,
flight_id, device
0: jdbc:hive2://pco-ph-gbdm-03:10000> -- order by acpu_time) asc_order,
0: jdbc:hive2://pco-ph-gbdm-03:10000> -- row_number() over (
0: jdbc:hive2://pco-ph-gbdm-03:10000> -- partition by flight_source,
flight_id, device
0: jdbc:hive2://pco-ph-gbdm-03:10000> -- order by acpu_time desc)
desc_order,
0: jdbc:hive2://pco-ph-gbdm-03:10000> year,
0: jdbc:hive2://pco-ph-gbdm-03:10000> month,
0: jdbc:hive2://pco-ph-gbdm-03:10000> day,
0: jdbc:hive2://pco-ph-gbdm-03:10000> flight_source
0: jdbc:hive2://pco-ph-gbdm-03:10000> from
0: jdbc:hive2://pco-ph-gbdm-03:10000> sla.minute
0: jdbc:hive2://pco-ph-gbdm-03:10000> where
0: jdbc:hive2://pco-ph-gbdm-03:10000> agl_ft >= 10000
0: jdbc:hive2://pco-ph-gbdm-03:10000> and device = 'sla';
No rows affected (0.097 seconds)
0: jdbc:hive2://pco-ph-gbdm-03:10000> explain authorization select * from
minute_order_v where year = 2016 and month = 5 and day = 23 and flight_source =
'gdw';
+--------------------------------------------------------------------------------------------------------------------------------------------+--+
| Explain
|
+--------------------------------------------------------------------------------------------------------------------------------------------+--+
| INPUTS:
|
| sla@minute_order_v
|
| sla@minute
|
| sla@minute@year=2016/month=5/day=23/flight_source=gdw
|
| OUTPUTS:
|
|
hdfs://nameservice1/tmp/hive/etl-user/0d2d8c59-1dbe-4fb0-8cde-478ff33841de/hive_2016-06-23_15-02-51_565_9081433377804806084-6/-mr-10000
|
| CURRENT_USER:
|
| etl-user
|
| OPERATION:
|
| CREATEVIEW
|
| AUTHORIZATION_FAILURES:
|
| No privilege 'Select' found for inputs { database:sla,
table:minute_order_v}
|
| No privilege 'Select' found for inputs { database:sla, table:minute}
|
+--------------------------------------------------------------------------------------------------------------------------------------------+--+
13 rows selected (0.233 seconds)
{noformat}
This was the best place I could find to report this. Please advise if you know
of something better.
> can't do query on partitioned view with analytic function in strictmode
> -----------------------------------------------------------------------
>
> Key: HIVE-10419
> URL: https://issues.apache.org/jira/browse/HIVE-10419
> Project: Hive
> Issue Type: Bug
> Components: Hive, Views
> Affects Versions: 0.13.0, 0.14.0, 1.0.0
> Environment: Cloudera 5.3.x.
> Reporter: Hector Lagos
>
> Hey Guys,
> I created the following table:
> CREATE TABLE t1 (id int, key string, value string) partitioned by (dt int);
> And after that i created a view on that table as follow:
> create view v1 PARTITIONED ON (dt)
> as
> SELECT * FROM (
> SELECT row_number() over (partition by key order by value asc) as row_n, *
> FROM t1
> ) t WHERE row_n = 1;
> We are working with hive.mapred.mode=strict and when I try to do the query
> select * from v1 where dt = 2 , I'm getting the following error:
> FAILED: SemanticException [Error 10041]: No partition predicate found for
> Alias "v1:t:t1" Table "t1"
> Is this a bug or a limitation of Hive when you use analytic functions in
> partitioned views? If i remove the row_number function it works without
> problems.
> Thanks in advance, any help will be appreciated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
