[jira] [Updated] (HIVE-13807) Extend metadata provider to pull up predicates through Union
[
https://issues.apache.org/jira/browse/HIVE-13807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-13807:
---
Resolution: Fixed
Fix Version/s: 2.1.0
Status: Resolved (was: Patch Available)
Pushed to master, branch-2.1. Thanks for reviewing [~ashutoshc]!
> Extend metadata provider to pull up predicates through Union
>
>
> Key: HIVE-13807
> URL: https://issues.apache.org/jira/browse/HIVE-13807
> Project: Hive
> Issue Type: Sub-task
> Components: CBO
>Affects Versions: 2.1.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Fix For: 2.1.0
>
> Attachments: HIVE-13807.01.patch, HIVE-13807.patch
>
>
> Follow-up of HIVE-13068.
> Currently, when we pull up predicates through Union operation with Calcite
> metadata provider, we just create a single disjunction of pulled up
> predicates.
> E.g. Assume operators _I1, I2, I3_ with predicates {{P1, P2, P3}} that can be
> pulled up through them, respectively.
> For an operation _Union (I1, I2, I3)_ we infer a new predicate {{Pu}}, such
> that {{Pu = P1 OR P2 OR P3}}.
> While this is correct, we miss some chances for simplification e.g. if there
> are common factors in P1, P2, and P3. Further, this inference differs
> slightly from the way that the metadata provider pulls up predicates for
> other operators (a list of predicates), thus breaking some assumptions and
> missing some optimization opportunities.
> Ex. ql/src/test/results/clientpositive/input26.q.out
> {noformat}
> explain
> select * from (
> select * from (select * from srcpart a where a.ds = '2008-04-08' and a.hr =
> '11' order by a.key limit 5)pa
> union all
> select * from (select * from srcpart b where b.ds = '2008-04-08' and b.hr =
> '14' limit 5)pb
> )subq
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13807) Extend metadata provider to pull up predicates through Union
[
https://issues.apache.org/jira/browse/HIVE-13807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-13807:
---
Attachment: HIVE-13807.01.patch
> Extend metadata provider to pull up predicates through Union
>
>
> Key: HIVE-13807
> URL: https://issues.apache.org/jira/browse/HIVE-13807
> Project: Hive
> Issue Type: Sub-task
> Components: CBO
>Affects Versions: 2.1.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Fix For: 2.1.0
>
> Attachments: HIVE-13807.01.patch, HIVE-13807.patch
>
>
> Follow-up of HIVE-13068.
> Currently, when we pull up predicates through Union operation with Calcite
> metadata provider, we just create a single disjunction of pulled up
> predicates.
> E.g. Assume operators _I1, I2, I3_ with predicates {{P1, P2, P3}} that can be
> pulled up through them, respectively.
> For an operation _Union (I1, I2, I3)_ we infer a new predicate {{Pu}}, such
> that {{Pu = P1 OR P2 OR P3}}.
> While this is correct, we miss some chances for simplification e.g. if there
> are common factors in P1, P2, and P3. Further, this inference differs
> slightly from the way that the metadata provider pulls up predicates for
> other operators (a list of predicates), thus breaking some assumptions and
> missing some optimization opportunities.
> Ex. ql/src/test/results/clientpositive/input26.q.out
> {noformat}
> explain
> select * from (
> select * from (select * from srcpart a where a.ds = '2008-04-08' and a.hr =
> '11' order by a.key limit 5)pa
> union all
> select * from (select * from srcpart b where b.ds = '2008-04-08' and b.hr =
> '14' limit 5)pb
> )subq
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13807) Extend metadata provider to pull up predicates through Union
[
https://issues.apache.org/jira/browse/HIVE-13807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-13807:
---
Description:
Follow-up of HIVE-13068.
Currently, when we pull up predicates through Union operation with Calcite
metadata provider, we just create a single disjunction of pulled up predicates.
E.g. Assume operators _I1, I2, I3_ with predicates {{P1, P2, P3}} that can be
pulled up through them, respectively.
For an operation _Union (I1, I2, I3)_ we infer a new predicate {{Pu}}, such
that {{Pu = P1 OR P2 OR P3}}.
While this is correct, we miss some chances for simplification e.g. if there
are common factors in P1, P2, and P3. Further, this inference differs slightly
from the way that the metadata provider pulls up predicates for other operators
(a list of predicates), thus breaking some assumptions and missing some
optimization opportunities.
Ex. ql/src/test/results/clientpositive/input26.q.out
{noformat}
explain
select * from (
select * from (select * from srcpart a where a.ds = '2008-04-08' and a.hr =
'11' order by a.key limit 5)pa
union all
select * from (select * from srcpart b where b.ds = '2008-04-08' and b.hr =
'14' limit 5)pb
)subq
{noformat}
was:
Follow-up of HIVE-13068.
Currently, when we pull up predicates through Union operation with Calcite
metadata provider, we just create a single conjunction of pulled up predicates.
E.g. Assume operators _I1, I2, I3_ with predicates {{P1, P2, P3}} that can be
pulled up through them, respectively.
For an operation _Union (I1, I2, I3)_ we infer a new predicate {{Pu}}, such
that {{Pu = P1 OR P2 OR P3}}.
While this is correct, we miss some chances for simplification e.g. if there
are common factors in P1, P2, and P3. Further, this inference differs slightly
from the way that the metadata provider pulls up predicates for other
operators, thus breaking some assumptions and missing some optimization
opportunities.
Ex. ql/src/test/results/clientpositive/input26.q.out
{noformat}
explain
select * from (
select * from (select * from srcpart a where a.ds = '2008-04-08' and a.hr =
'11' order by a.key limit 5)pa
union all
select * from (select * from srcpart b where b.ds = '2008-04-08' and b.hr =
'14' limit 5)pb
)subq
{noformat}
> Extend metadata provider to pull up predicates through Union
>
>
> Key: HIVE-13807
> URL: https://issues.apache.org/jira/browse/HIVE-13807
> Project: Hive
> Issue Type: Sub-task
> Components: CBO
>Affects Versions: 2.1.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-13807.patch
>
>
> Follow-up of HIVE-13068.
> Currently, when we pull up predicates through Union operation with Calcite
> metadata provider, we just create a single disjunction of pulled up
> predicates.
> E.g. Assume operators _I1, I2, I3_ with predicates {{P1, P2, P3}} that can be
> pulled up through them, respectively.
> For an operation _Union (I1, I2, I3)_ we infer a new predicate {{Pu}}, such
> that {{Pu = P1 OR P2 OR P3}}.
> While this is correct, we miss some chances for simplification e.g. if there
> are common factors in P1, P2, and P3. Further, this inference differs
> slightly from the way that the metadata provider pulls up predicates for
> other operators (a list of predicates), thus breaking some assumptions and
> missing some optimization opportunities.
> Ex. ql/src/test/results/clientpositive/input26.q.out
> {noformat}
> explain
> select * from (
> select * from (select * from srcpart a where a.ds = '2008-04-08' and a.hr =
> '11' order by a.key limit 5)pa
> union all
> select * from (select * from srcpart b where b.ds = '2008-04-08' and b.hr =
> '14' limit 5)pb
> )subq
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13807) Extend metadata provider to pull up predicates through Union
[
https://issues.apache.org/jira/browse/HIVE-13807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-13807:
---
Attachment: HIVE-13807.patch
> Extend metadata provider to pull up predicates through Union
>
>
> Key: HIVE-13807
> URL: https://issues.apache.org/jira/browse/HIVE-13807
> Project: Hive
> Issue Type: Sub-task
> Components: CBO
>Affects Versions: 2.1.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-13807.patch
>
>
> Follow-up of HIVE-13068.
> Currently, when we pull up predicates through Union operation with Calcite
> metadata provider, we just create a single conjunction of pulled up
> predicates.
> E.g. Assume operators _I1, I2, I3_ with predicates {{P1, P2, P3}} that can be
> pulled up through them, respectively.
> For an operation _Union (I1, I2, I3)_ we infer a new predicate {{Pu}}, such
> that {{Pu = P1 OR P2 OR P3}}.
> While this is correct, we miss some chances for simplification e.g. if there
> are common factors in P1, P2, and P3. Further, this inference differs
> slightly from the way that the metadata provider pulls up predicates for
> other operators, thus breaking some assumptions and missing some optimization
> opportunities.
> Ex. ql/src/test/results/clientpositive/input26.q.out
> {noformat}
> explain
> select * from (
> select * from (select * from srcpart a where a.ds = '2008-04-08' and a.hr =
> '11' order by a.key limit 5)pa
> union all
> select * from (select * from srcpart b where b.ds = '2008-04-08' and b.hr =
> '14' limit 5)pb
> )subq
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13807) Extend metadata provider to pull up predicates through Union
[
https://issues.apache.org/jira/browse/HIVE-13807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-13807:
---
Status: Patch Available (was: In Progress)
> Extend metadata provider to pull up predicates through Union
>
>
> Key: HIVE-13807
> URL: https://issues.apache.org/jira/browse/HIVE-13807
> Project: Hive
> Issue Type: Sub-task
> Components: CBO
>Affects Versions: 2.1.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>
> Follow-up of HIVE-13068.
> Currently, when we pull up predicates through Union operation with Calcite
> metadata provider, we just create a single conjunction of pulled up
> predicates.
> E.g. Assume operators _I1, I2, I3_ with predicates {{P1, P2, P3}} that can be
> pulled up through them, respectively.
> For an operation _Union (I1, I2, I3)_ we infer a new predicate {{Pu}}, such
> that {{Pu = P1 OR P2 OR P3}}.
> While this is correct, we miss some chances for simplification e.g. if there
> are common factors in P1, P2, and P3. Further, this inference differs
> slightly from the way that the metadata provider pulls up predicates for
> other operators, thus breaking some assumptions and missing some optimization
> opportunities.
> Ex. ql/src/test/results/clientpositive/input26.q.out
> {noformat}
> explain
> select * from (
> select * from (select * from srcpart a where a.ds = '2008-04-08' and a.hr =
> '11' order by a.key limit 5)pa
> union all
> select * from (select * from srcpart b where b.ds = '2008-04-08' and b.hr =
> '14' limit 5)pb
> )subq
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13807) Extend metadata provider to pull up predicates through Union
[
https://issues.apache.org/jira/browse/HIVE-13807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-13807:
---
Description:
Follow-up of HIVE-13068.
Currently, when we pull up predicates through Union operation with Calcite
metadata provider, we just create a single conjunction of pulled up predicates.
E.g. Assume operators _I1, I2, I3_ with predicates {{P1, P2, P3}} that can be
pulled up through them, respectively.
For an operation _Union (I1, I2, I3)_ we infer a new predicate {{Pu}}, such
that {{Pu = P1 OR P2 OR P3}}.
While this is correct, we miss some chances for simplification e.g. if there
are common factors in P1, P2, and P3. Further, this inference differs slightly
from the way that the metadata provider pulls up predicates for other
operators, thus breaking some assumptions and missing some optimization
opportunities.
Ex. ql/src/test/results/clientpositive/input26.q.out
{noformat}
explain
select * from (
select * from (select * from srcpart a where a.ds = '2008-04-08' and a.hr =
'11' order by a.key limit 5)pa
union all
select * from (select * from srcpart b where b.ds = '2008-04-08' and b.hr =
'14' limit 5)pb
)subq
{noformat}
was:
Follow-up of HIVE-13068.
Currently, when we pull up predicates through Union operation with Calcite
metadata provider, we just create a single conjunction of pulled up predicates.
E.g. Assume operators {{I1, I2, I3}} with predicates {{P1, P2, P3}} that can be
pulled up through them, respectively.
For an operation _Union (I1, I2, I3)_ we infer a new predicate {{Pu}}, such
that {{Pu = P1 OR P2 OR P3}}.
While this is correct, we miss some chances for simplification e.g. if there
are common factors in P1, P2, and P3. Further, this inference differs slightly
from the way that the metadata provider pulls up predicates for other
operators, thus breaking some assumptions and missing some optimization
opportunities.
Ex. ql/src/test/results/clientpositive/input26.q.out
{noformat}
explain
select * from (
select * from (select * from srcpart a where a.ds = '2008-04-08' and a.hr =
'11' order by a.key limit 5)pa
union all
select * from (select * from srcpart b where b.ds = '2008-04-08' and b.hr =
'14' limit 5)pb
)subq
{noformat}
> Extend metadata provider to pull up predicates through Union
>
>
> Key: HIVE-13807
> URL: https://issues.apache.org/jira/browse/HIVE-13807
> Project: Hive
> Issue Type: Sub-task
> Components: CBO
>Affects Versions: 2.1.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>
> Follow-up of HIVE-13068.
> Currently, when we pull up predicates through Union operation with Calcite
> metadata provider, we just create a single conjunction of pulled up
> predicates.
> E.g. Assume operators _I1, I2, I3_ with predicates {{P1, P2, P3}} that can be
> pulled up through them, respectively.
> For an operation _Union (I1, I2, I3)_ we infer a new predicate {{Pu}}, such
> that {{Pu = P1 OR P2 OR P3}}.
> While this is correct, we miss some chances for simplification e.g. if there
> are common factors in P1, P2, and P3. Further, this inference differs
> slightly from the way that the metadata provider pulls up predicates for
> other operators, thus breaking some assumptions and missing some optimization
> opportunities.
> Ex. ql/src/test/results/clientpositive/input26.q.out
> {noformat}
> explain
> select * from (
> select * from (select * from srcpart a where a.ds = '2008-04-08' and a.hr =
> '11' order by a.key limit 5)pa
> union all
> select * from (select * from srcpart b where b.ds = '2008-04-08' and b.hr =
> '14' limit 5)pb
> )subq
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
