[jira] [Commented] (CALCITE-3273) Materialization matching rules that consider compensating MutableProject
[
https://issues.apache.org/jira/browse/CALCITE-3273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911945#comment-16911945
]
jin xing commented on CALCITE-3273:
---
As far as I know, {{AggregateOnProjectToAggregateUnifyRule}} is also an
important case where compensating {{MutableProject}} is considered.
> Materialization matching rules that consider compensating MutableProject
>
>
> Key: CALCITE-3273
> URL: https://issues.apache.org/jira/browse/CALCITE-3273
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: jin xing
>Priority: Major
>
> In current implementation of rules in SubstitutionVisitor.java &
> MaterializedViewSubstitutionVisitor.java, it's quite common to add a
> compensating Project on top of child node of target(MV-rel) during matching.
> But afterwards the next round matching should be able to handle such a
> compensated Project and match upward along the plan tree. Otherwise we fail
> the matching. After all, the goal of matching is to transform the query and
> let a complete 'target' show up in the transformed query plan.
> I opened this as a parent JIRA to add MV matching rules that consider
> compensating MutableProject.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Assigned] (CALCITE-3274) Add FilterOnProjectToFilterUnifyRule for materialization matching
[
https://issues.apache.org/jira/browse/CALCITE-3274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jin xing reassigned CALCITE-3274:
-
Assignee: jin xing
> Add FilterOnProjectToFilterUnifyRule for materialization matching
> -
>
> Key: CALCITE-3274
> URL: https://issues.apache.org/jira/browse/CALCITE-3274
> Project: Calcite
> Issue Type: Sub-task
> Components: core
>Reporter: jin xing
>Assignee: jin xing
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> With current implementation of MV matching, below case fails
> {code:java}
> @Test public void testFilterOnProject() {
> String deduplicated =
> "(select \"empid\", \"deptno\", \"name\", \"salary\",
> \"commission\"\n"
> + "from \"emps\"\n"
> + "group by \"empid\", \"deptno\", \"name\", \"salary\",
> \"commission\")";
> String mv =
> "select * from\n"
> + "(select \"deptno\", sum(\"salary\") as \"sum_salary\",
> sum(\"commission\")\n"
> + "from " + deduplicated + "\n"
> + "group by \"deptno\")\n"
> + "where \"sum_salary\" > 10";
> String query =
> "select * from\n"
> + "(select \"deptno\", sum(\"salary\") as \"sum_salary\"\n"
> + "from " + deduplicated + "\n"
> + "group by \"deptno\")\n"
> + "where \"sum_salary\" > 10";
> checkMaterialize(mv, query);
> }
> {code}
> {{Reason:}}
> After matching of the Aggregates, a compensating Project is added, but
> afterwards matching of filter fails to handle it.
> This issue proposes to handle such case where query and target are Filters
> and query has a compensating Project child node
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (CALCITE-3274) Add FilterOnProjectToFilterUnifyRule for materialization matching
[
https://issues.apache.org/jira/browse/CALCITE-3274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CALCITE-3274:
Labels: pull-request-available (was: )
> Add FilterOnProjectToFilterUnifyRule for materialization matching
> -
>
> Key: CALCITE-3274
> URL: https://issues.apache.org/jira/browse/CALCITE-3274
> Project: Calcite
> Issue Type: Sub-task
> Components: core
>Reporter: jin xing
>Priority: Major
> Labels: pull-request-available
>
> With current implementation of MV matching, below case fails
> {code:java}
> @Test public void testFilterOnProject() {
> String deduplicated =
> "(select \"empid\", \"deptno\", \"name\", \"salary\",
> \"commission\"\n"
> + "from \"emps\"\n"
> + "group by \"empid\", \"deptno\", \"name\", \"salary\",
> \"commission\")";
> String mv =
> "select * from\n"
> + "(select \"deptno\", sum(\"salary\") as \"sum_salary\",
> sum(\"commission\")\n"
> + "from " + deduplicated + "\n"
> + "group by \"deptno\")\n"
> + "where \"sum_salary\" > 10";
> String query =
> "select * from\n"
> + "(select \"deptno\", sum(\"salary\") as \"sum_salary\"\n"
> + "from " + deduplicated + "\n"
> + "group by \"deptno\")\n"
> + "where \"sum_salary\" > 10";
> checkMaterialize(mv, query);
> }
> {code}
> {{Reason:}}
> After matching of the Aggregates, a compensating Project is added, but
> afterwards matching of filter fails to handle it.
> This issue proposes to handle such case where query and target are Filters
> and query has a compensating Project child node
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (CALCITE-3274) Add FilterOnProjectToFilterUnifyRule for materialization matching
[
https://issues.apache.org/jira/browse/CALCITE-3274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jin xing updated CALCITE-3274:
--
Description:
With current implementation of MV matching, below case fails
{code:java}
@Test public void testFilterOnProject() {
String deduplicated =
"(select \"empid\", \"deptno\", \"name\", \"salary\", \"commission\"\n"
+ "from \"emps\"\n"
+ "group by \"empid\", \"deptno\", \"name\", \"salary\",
\"commission\")";
String mv =
"select * from\n"
+ "(select \"deptno\", sum(\"salary\") as \"sum_salary\",
sum(\"commission\")\n"
+ "from " + deduplicated + "\n"
+ "group by \"deptno\")\n"
+ "where \"sum_salary\" > 10";
String query =
"select * from\n"
+ "(select \"deptno\", sum(\"salary\") as \"sum_salary\"\n"
+ "from " + deduplicated + "\n"
+ "group by \"deptno\")\n"
+ "where \"sum_salary\" > 10";
checkMaterialize(mv, query);
}
{code}
{{Reason:}}
After matching of the Aggregates, a compensating Project is added, but
afterwards matching of filter fails to handle it.
This issue proposes to handle such case where query and target are Filters and
query has a compensating Project child node
was:
With current implementation of MV matching, below case fails
{code:java}
@Test public void testFilterOnProject() {
String deduplicated =
"(select \"empid\", \"deptno\", \"name\", \"salary\", \"commission\"\n"
+ "from \"emps\"\n"
+ "group by \"empid\", \"deptno\", \"name\", \"salary\",
\"commission\")";
String mv =
"select * from\n"
+ "(select \"deptno\", sum(\"salary\") as \"sum_salary\",
sum(\"commission\")\n"
+ "from " + deduplicated + "\n"
+ "group by \"deptno\")\n"
+ "where \"sum_salary\" > 10";
String query =
"select * from\n"
+ "(select \"deptno\", sum(\"salary\") as \"sum_salary\"\n"
+ "from " + deduplicated + "\n"
+ "group by \"deptno\")\n"
+ "where \"sum_salary\" > 10";
checkMaterialize(mv, query);
}
{code}
{{Reason:}}
After matching of the Aggregates, a compensating Project is added, but
afterwards matching of filter fails to handle it.
This issue proposes to handle such case.
> Add FilterOnProjectToFilterUnifyRule for materialization matching
> -
>
> Key: CALCITE-3274
> URL: https://issues.apache.org/jira/browse/CALCITE-3274
> Project: Calcite
> Issue Type: Sub-task
> Components: core
>Reporter: jin xing
>Priority: Major
>
> With current implementation of MV matching, below case fails
> {code:java}
> @Test public void testFilterOnProject() {
> String deduplicated =
> "(select \"empid\", \"deptno\", \"name\", \"salary\",
> \"commission\"\n"
> + "from \"emps\"\n"
> + "group by \"empid\", \"deptno\", \"name\", \"salary\",
> \"commission\")";
> String mv =
> "select * from\n"
> + "(select \"deptno\", sum(\"salary\") as \"sum_salary\",
> sum(\"commission\")\n"
> + "from " + deduplicated + "\n"
> + "group by \"deptno\")\n"
> + "where \"sum_salary\" > 10";
> String query =
> "select * from\n"
> + "(select \"deptno\", sum(\"salary\") as \"sum_salary\"\n"
> + "from " + deduplicated + "\n"
> + "group by \"deptno\")\n"
> + "where \"sum_salary\" > 10";
> checkMaterialize(mv, query);
> }
> {code}
> {{Reason:}}
> After matching of the Aggregates, a compensating Project is added, but
> afterwards matching of filter fails to handle it.
> This issue proposes to handle such case where query and target are Filters
> and query has a compensating Project child node
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (CALCITE-3274) Add FilterOnProjectToFilterUnifyRule for materialization matching
[
https://issues.apache.org/jira/browse/CALCITE-3274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jin xing updated CALCITE-3274:
--
Description:
With current implementation of MV matching, below case fails
{code:java}
@Test public void testFilterOnProject() {
String deduplicated =
"(select \"empid\", \"deptno\", \"name\", \"salary\", \"commission\"\n"
+ "from \"emps\"\n"
+ "group by \"empid\", \"deptno\", \"name\", \"salary\",
\"commission\")";
String mv =
"select * from\n"
+ "(select \"deptno\", sum(\"salary\") as \"sum_salary\",
sum(\"commission\")\n"
+ "from " + deduplicated + "\n"
+ "group by \"deptno\")\n"
+ "where \"sum_salary\" > 10";
String query =
"select * from\n"
+ "(select \"deptno\", sum(\"salary\") as \"sum_salary\"\n"
+ "from " + deduplicated + "\n"
+ "group by \"deptno\")\n"
+ "where \"sum_salary\" > 10";
checkMaterialize(mv, query);
}
{code}
{{Reason:}}
After matching of the Aggregates, a compensating Project is added, but
afterwards matching of filter fails to handle it.
This issue proposes to handle such case.
> Add FilterOnProjectToFilterUnifyRule for materialization matching
> -
>
> Key: CALCITE-3274
> URL: https://issues.apache.org/jira/browse/CALCITE-3274
> Project: Calcite
> Issue Type: Sub-task
> Components: core
>Reporter: jin xing
>Priority: Major
>
> With current implementation of MV matching, below case fails
> {code:java}
> @Test public void testFilterOnProject() {
> String deduplicated =
> "(select \"empid\", \"deptno\", \"name\", \"salary\",
> \"commission\"\n"
> + "from \"emps\"\n"
> + "group by \"empid\", \"deptno\", \"name\", \"salary\",
> \"commission\")";
> String mv =
> "select * from\n"
> + "(select \"deptno\", sum(\"salary\") as \"sum_salary\",
> sum(\"commission\")\n"
> + "from " + deduplicated + "\n"
> + "group by \"deptno\")\n"
> + "where \"sum_salary\" > 10";
> String query =
> "select * from\n"
> + "(select \"deptno\", sum(\"salary\") as \"sum_salary\"\n"
> + "from " + deduplicated + "\n"
> + "group by \"deptno\")\n"
> + "where \"sum_salary\" > 10";
> checkMaterialize(mv, query);
> }
> {code}
> {{Reason:}}
> After matching of the Aggregates, a compensating Project is added, but
> afterwards matching of filter fails to handle it.
> This issue proposes to handle such case.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Created] (CALCITE-3274) Add FilterOnProjectToFilterUnifyRule for materialization matching
jin xing created CALCITE-3274: - Summary: Add FilterOnProjectToFilterUnifyRule for materialization matching Key: CALCITE-3274 URL: https://issues.apache.org/jira/browse/CALCITE-3274 Project: Calcite Issue Type: Sub-task Components: core Reporter: jin xing -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (CALCITE-3256) Add ProjectOnProjectToProjectUnifyRule for materialization matching.
[
https://issues.apache.org/jira/browse/CALCITE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jin xing updated CALCITE-3256:
--
Description:
In current code, below matching fails:
{code:java}
MV:
select deptno, sum(salary) + 2, sum(commission)
from emps
group by deptno
Query:
select deptno, sum(salary) + 2
from emps
group by deptno
{code}
The reason is that -- after matching of the Aggregates, a compensating Project
is added, but afterwards matching fails to handle it.
This issue proposes to add a rule that match when query and target are both
Project and query has a compensating Project as a child. After this issue below
case can be handled:
{code:java}
query: Project(projects: [$0, +($1, 2)])
Project(projects: [$1, $3, $4])
Rel-A
target: Project(projects: [$1, +($3, 2)])
Rel-A
{code}
was:
This issue proposes to add a rule to match a project
{code:java}
MV:
select deptno, sum(salary) + 2, sum(commission)
from emps
group by deptno
Query:
select deptno, sum(salary) + 2
from emps
group by deptno
{code}
After matching of the Aggregates, a compensating Project is added, but
afterwards matching fails to handle it.
> Add ProjectOnProjectToProjectUnifyRule for materialization matching.
>
>
> Key: CALCITE-3256
> URL: https://issues.apache.org/jira/browse/CALCITE-3256
> Project: Calcite
> Issue Type: Sub-task
> Components: core
>Reporter: jin xing
>Assignee: jin xing
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In current code, below matching fails:
> {code:java}
> MV:
> select deptno, sum(salary) + 2, sum(commission)
> from emps
> group by deptno
> Query:
> select deptno, sum(salary) + 2
> from emps
> group by deptno
> {code}
> The reason is that -- after matching of the Aggregates, a compensating
> Project is added, but afterwards matching fails to handle it.
> This issue proposes to add a rule that match when query and target are both
> Project and query has a compensating Project as a child. After this issue
> below case can be handled:
> {code:java}
>query: Project(projects: [$0, +($1, 2)])
> Project(projects: [$1, $3, $4])
>Rel-A
>target: Project(projects: [$1, +($3, 2)])
> Rel-A
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (CALCITE-3256) Add ProjectOnProjectToProjectUnifyRule for materialization matching.
[
https://issues.apache.org/jira/browse/CALCITE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jin xing updated CALCITE-3256:
--
Description:
In current code, below matching fails:
{code:java}
MV:
select deptno, sum(salary) + 2, sum(commission)
from emps
group by deptno
Query:
select deptno, sum(salary) + 2
from emps
group by deptno
{code}
The reason is that -- after matching of the Aggregates, a compensating Project
is added, but afterwards matching fails to handle it.
This issue proposes to add a rule that match when query and target are both
Project and query has a compensating Project as a child. After this issue below
case can be handled:
{code:java}
Query:
Project(projects: [$0, +($1, 2)])
Project(projects: [$1, $3, $4])
Rel-A
Target:
Project(projects: [$1, +($3, 2)])
Rel-A
{code}
was:
In current code, below matching fails:
{code:java}
MV:
select deptno, sum(salary) + 2, sum(commission)
from emps
group by deptno
Query:
select deptno, sum(salary) + 2
from emps
group by deptno
{code}
The reason is that -- after matching of the Aggregates, a compensating Project
is added, but afterwards matching fails to handle it.
This issue proposes to add a rule that match when query and target are both
Project and query has a compensating Project as a child. After this issue below
case can be handled:
{code:java}
Query:
Project(projects: [$0, +($1, 2)])
Project(projects: [$1, $3, $4])
Rel-A
Target:
Project(projects: [$1, +($3, 2)])
Rel-A
{code}
> Add ProjectOnProjectToProjectUnifyRule for materialization matching.
>
>
> Key: CALCITE-3256
> URL: https://issues.apache.org/jira/browse/CALCITE-3256
> Project: Calcite
> Issue Type: Sub-task
> Components: core
>Reporter: jin xing
>Assignee: jin xing
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In current code, below matching fails:
> {code:java}
> MV:
> select deptno, sum(salary) + 2, sum(commission)
> from emps
> group by deptno
> Query:
> select deptno, sum(salary) + 2
> from emps
> group by deptno
> {code}
> The reason is that -- after matching of the Aggregates, a compensating
> Project is added, but afterwards matching fails to handle it.
> This issue proposes to add a rule that match when query and target are both
> Project and query has a compensating Project as a child. After this issue
> below case can be handled:
> {code:java}
> Query:
> Project(projects: [$0, +($1, 2)])
> Project(projects: [$1, $3, $4])
> Rel-A
> Target:
> Project(projects: [$1, +($3, 2)])
> Rel-A
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (CALCITE-3256) Add ProjectOnProjectToProjectUnifyRule for materialization matching.
[
https://issues.apache.org/jira/browse/CALCITE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jin xing updated CALCITE-3256:
--
Description:
In current code, below matching fails:
{code:java}
MV:
select deptno, sum(salary) + 2, sum(commission)
from emps
group by deptno
Query:
select deptno, sum(salary) + 2
from emps
group by deptno
{code}
The reason is that -- after matching of the Aggregates, a compensating Project
is added, but afterwards matching fails to handle it.
This issue proposes to add a rule that match when query and target are both
Project and query has a compensating Project as a child. After this issue below
case can be handled:
{code:java}
Query:
Project(projects: [$0, +($1, 2)])
Project(projects: [$1, $3, $4])
Rel-A
Target:
Project(projects: [$1, +($3, 2)])
Rel-A
{code}
was:
In current code, below matching fails:
{code:java}
MV:
select deptno, sum(salary) + 2, sum(commission)
from emps
group by deptno
Query:
select deptno, sum(salary) + 2
from emps
group by deptno
{code}
The reason is that -- after matching of the Aggregates, a compensating Project
is added, but afterwards matching fails to handle it.
This issue proposes to add a rule that match when query and target are both
Project and query has a compensating Project as a child. After this issue below
case can be handled:
{code:java}
query: Project(projects: [$0, +($1, 2)])
Project(projects: [$1, $3, $4])
Rel-A
target: Project(projects: [$1, +($3, 2)])
Rel-A
{code}
> Add ProjectOnProjectToProjectUnifyRule for materialization matching.
>
>
> Key: CALCITE-3256
> URL: https://issues.apache.org/jira/browse/CALCITE-3256
> Project: Calcite
> Issue Type: Sub-task
> Components: core
>Reporter: jin xing
>Assignee: jin xing
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In current code, below matching fails:
> {code:java}
> MV:
> select deptno, sum(salary) + 2, sum(commission)
> from emps
> group by deptno
> Query:
> select deptno, sum(salary) + 2
> from emps
> group by deptno
> {code}
> The reason is that -- after matching of the Aggregates, a compensating
> Project is added, but afterwards matching fails to handle it.
> This issue proposes to add a rule that match when query and target are both
> Project and query has a compensating Project as a child. After this issue
> below case can be handled:
> {code:java}
> Query:
> Project(projects: [$0, +($1, 2)])
> Project(projects: [$1, $3, $4])
> Rel-A
> Target:
> Project(projects: [$1, +($3, 2)])
> Rel-A
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (CALCITE-3256) Add ProjectOnProjectToProjectUnifyRule for materialization matching.
[
https://issues.apache.org/jira/browse/CALCITE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jin xing updated CALCITE-3256:
--
Description:
This issue proposes to add a rule to match a project
{code:java}
MV:
select deptno, sum(salary) + 2, sum(commission)
from emps
group by deptno
Query:
select deptno, sum(salary) + 2
from emps
group by deptno
{code}
After matching of the Aggregates, a compensating Project is added, but
afterwards matching fails to handle it.
was:
In current implementation of rules in SubstitutionVisitor.java &
MaterializedViewSubstitutionVisitor.java, it's quite common to add a
compensating Project on top of child node of target(MV-rel) during matching.
But afterwards the next round matching should be able to handle such a
compensated Project and match upward along the plan tree. Otherwise we fail the
matching. After all, the goal of matching is to transform the query and let a
complete 'target' show up in the transformed query plan.
I found cases where the compensated Project cannot be properly handled.
{code:java}
MV:
select deptno, sum(salary) + 2, sum(commission)
from emps
group by deptno
Query:
select deptno, sum(salary) + 2
from emps
group by deptno
{code}
After matching of the Aggregates, a compensating Project is added, but
afterwards matching fails to handle it.
> Add ProjectOnProjectToProjectUnifyRule for materialization matching.
>
>
> Key: CALCITE-3256
> URL: https://issues.apache.org/jira/browse/CALCITE-3256
> Project: Calcite
> Issue Type: Sub-task
> Components: core
>Reporter: jin xing
>Assignee: jin xing
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This issue proposes to add a rule to match a project
> {code:java}
> MV:
> select deptno, sum(salary) + 2, sum(commission)
> from emps
> group by deptno
> Query:
> select deptno, sum(salary) + 2
> from emps
> group by deptno
> {code}
> After matching of the Aggregates, a compensating Project is added, but
> afterwards matching fails to handle it.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (CALCITE-3256) Add ProjectOnProjectToProjectUnifyRule for materialization matching.
[
https://issues.apache.org/jira/browse/CALCITE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jin xing updated CALCITE-3256:
--
Parent: CALCITE-3273
Issue Type: Sub-task (was: Improvement)
> Add ProjectOnProjectToProjectUnifyRule for materialization matching.
>
>
> Key: CALCITE-3256
> URL: https://issues.apache.org/jira/browse/CALCITE-3256
> Project: Calcite
> Issue Type: Sub-task
> Components: core
>Reporter: jin xing
>Assignee: jin xing
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In current implementation of rules in SubstitutionVisitor.java &
> MaterializedViewSubstitutionVisitor.java, it's quite common to add a
> compensating Project on top of child node of target(MV-rel) during matching.
> But afterwards the next round matching should be able to handle such a
> compensated Project and match upward along the plan tree. Otherwise we fail
> the matching. After all, the goal of matching is to transform the query and
> let a complete 'target' show up in the transformed query plan.
> I found cases where the compensated Project cannot be properly handled.
> {code:java}
> MV:
> select deptno, sum(salary) + 2, sum(commission)
> from emps
> group by deptno
> Query:
> select deptno, sum(salary) + 2
> from emps
> group by deptno
> {code}
> After matching of the Aggregates, a compensating Project is added, but
> afterwards matching fails to handle it.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (CALCITE-3273) Materialization matching rules that consider compensating MutableProject
[ https://issues.apache.org/jira/browse/CALCITE-3273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jin xing updated CALCITE-3273: -- Description: In current implementation of rules in SubstitutionVisitor.java & MaterializedViewSubstitutionVisitor.java, it's quite common to add a compensating Project on top of child node of target(MV-rel) during matching. But afterwards the next round matching should be able to handle such a compensated Project and match upward along the plan tree. Otherwise we fail the matching. After all, the goal of matching is to transform the query and let a complete 'target' show up in the transformed query plan. I opened this as a parent JIRA to add MV matching rules that consider compensating MutableProject. > Materialization matching rules that consider compensating MutableProject > > > Key: CALCITE-3273 > URL: https://issues.apache.org/jira/browse/CALCITE-3273 > Project: Calcite > Issue Type: Improvement > Components: core >Reporter: jin xing >Priority: Major > > In current implementation of rules in SubstitutionVisitor.java & > MaterializedViewSubstitutionVisitor.java, it's quite common to add a > compensating Project on top of child node of target(MV-rel) during matching. > But afterwards the next round matching should be able to handle such a > compensated Project and match upward along the plan tree. Otherwise we fail > the matching. After all, the goal of matching is to transform the query and > let a complete 'target' show up in the transformed query plan. > I opened this as a parent JIRA to add MV matching rules that consider > compensating MutableProject. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (CALCITE-3273) Materialization matching rules that consider compensating MutableProject
jin xing created CALCITE-3273: - Summary: Materialization matching rules that consider compensating MutableProject Key: CALCITE-3273 URL: https://issues.apache.org/jira/browse/CALCITE-3273 Project: Calcite Issue Type: Improvement Components: core Reporter: jin xing -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (CALCITE-1581) UDTF like in hive
[
https://issues.apache.org/jira/browse/CALCITE-1581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911931#comment-16911931
]
Chunwei Lei commented on CALCITE-1581:
--
I am +1 on the patch, overall. Thanks for the effort!
> UDTF like in hive
> -
>
> Key: CALCITE-1581
> URL: https://issues.apache.org/jira/browse/CALCITE-1581
> Project: Calcite
> Issue Type: New Feature
>Reporter: Xiaoyong Deng
>Assignee: pengzhiwei
>Priority: Major
> Labels: pull-request-available, udtf
> Fix For: 1.21.0
>
> Time Spent: 7.5h
> Remaining Estimate: 0h
>
> Support one row in and multi-column/multi-row out(one-to-many mapping), just
> like udtf in hive.
> The query would like this:
> {code}
> select
> func(c0, c1) as (f0, f1, f2)
> from table_name;
> {code}
> c0 and c1 are 'table_name' columns. f0, f1 and f2 are new generated columns.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Created] (CALCITE-3272) TUMBLE Table Value Function
Rui Wang created CALCITE-3272: - Summary: TUMBLE Table Value Function Key: CALCITE-3272 URL: https://issues.apache.org/jira/browse/CALCITE-3272 Project: Calcite Issue Type: Sub-task Reporter: Rui Wang Define a builtin TVF: Tumble (data , timecol , dur, [ offset ]) The return value of Tumble is a relation that includes all columns of data as well as additional event time columns wstart and wend. Examples of TUMBLE TVF are (from https://s.apache.org/streaming-beam-sql): 8:21> SELECT * FROM Bid; -- | bidtime | price | item | -- | 8:07| $2| A| | 8:11| $3| B| | 8:05| $4| C| | 8:09| $5| D| | 8:13| $1| E| | 8:17| $6| F| -- 8:21> SELECT * FROM Tumble ( data=> TABLE Bid , timecol => DESCRIPTOR ( bidtime ) , dur => INTERVAL '10' MINUTES , offset => INTERVAL '0' MINUTES ); -- | wstart | wend | bidtime | price | item | -- | 8:00 | 8:10 | 8:07| $2| A| | 8:10 | 8:20 | 8:11| $3| B| | 8:00 | 8:10 | 8:05| $4| C| | 8:00 | 8:10 | 8:09| $5| D| | 8:10 | 8:20 | 8:13| $1| E| | 8:10 | 8:20 | 8:17| $6| F| -- 8:21> SELECT MAX ( wstart ) , wend , SUM ( price ) FROM Tumble ( data=> TABLE ( Bid ) , timecol => DESCRIPTOR ( bidtime ) , dur => INTERVAL '10 ' MINUTES ) GROUP BY wend; - | wstart | wend | price | - | 8:00 | 8:10 | $11 | | 8:10 | 8:20 | $10 | - -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (CALCITE-2302) Implicit type cast support
[
https://issues.apache.org/jira/browse/CALCITE-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911921#comment-16911921
]
Haisheng Yuan commented on CALCITE-2302:
+1 on the patch, overall. Given the default behavior change for queries running
on Postgres, ('9'/2 returns 4.5 instead of 4), we'd better underscore or warn
the important default behavior change for implicit cast and integer division in
the release notes.
Thanks for the great work!
> Implicit type cast support
> --
>
> Key: CALCITE-2302
> URL: https://issues.apache.org/jira/browse/CALCITE-2302
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.17.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 6h 40m
> Remaining Estimate: 0h
>
> Now many DBs have support implicit type cast, eg: SqlServer, Oracle, Hive.
> Implicit type cast is an useful function for many cases, So we should support
> this.
> I checkout Calcite code and found that:
> # Now we use a validator to validate our operands types[ through kinds of
> namespaces and scopes ]
> # Most of the validations will finally goes to
> {code:java}
> SqlOperator.validateOperands
> {code}
> # which will use validation logic defined in corresponding
> SqlOperandTypeChecker
> What i'm confused about is where should i put the implicit type cast logic
> in? I figured out 2 ways:
> # Supply a tool class/rules to add casts into a parsed SqlNode tree which
> will then go through the validation logic later on.
> # Unleash the validation logic in kinds of SqlOperandTypeChecker, then
> modify the RelNode/RexNodes tree converted from a validated SqlNode tree to
> add in casts through custom RelOptRules.
> So guys, which of the 2 ways should i go, or if there are better way to do
> this?
> I need your help.
>
> Updated 18-05-30:
> Hi guys, i have made a PR in
> [CALCITE-2302|https://github.com/apache/calcite/pull/706]
> This is design doc: [Calcite Implicit Type Cast
> Design|https://docs.google.com/document/d/1g2RUnLXyp_LjUlO-wbblKuP5hqEu3a_2Mt2k4dh6RwU/edit?usp=sharing].
> This is the conversion types mapping: [Conversion Types
> Mapping|https://docs.google.com/spreadsheets/d/1GhleX5h5W8-kJKh7NMJ4vtoE78pwfaZRJl88ULX_MgU/edit?usp=sharing].
> I really appreciate your suggestions, thx.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Resolved] (CALCITE-3213) Support complex type expressions for SqlDataTypeSpec
[
https://issues.apache.org/jira/browse/CALCITE-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chan resolved CALCITE-3213.
-
Resolution: Fixed
> Support complex type expressions for SqlDataTypeSpec
> -
>
> Key: CALCITE-3213
> URL: https://issues.apache.org/jira/browse/CALCITE-3213
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.20.0
>Reporter: Danny Chan
>Assignee: Danny Chan
>Priority: Major
> Fix For: 1.21.0
>
>
> We should support nested struct type like:
> {code:sql}
> ROW(
> foo NUMBER(5, 2) NOT NULL,
> rec ROW(b BOOLEAN, i MyUDT NOT NULL))
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (CALCITE-3271) TVF windowing and EMIT syntax support in Calcite
[ https://issues.apache.org/jira/browse/CALCITE-3271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chan updated CALCITE-3271: Issue Type: Improvement (was: Bug) > TVF windowing and EMIT syntax support in Calcite > > > Key: CALCITE-3271 > URL: https://issues.apache.org/jira/browse/CALCITE-3271 > Project: Calcite > Issue Type: Improvement > Components: core >Affects Versions: 1.20.0 >Reporter: Danny Chan >Assignee: Danny Chan >Priority: Major > > Copied from the mailing list: > Calcite has not implemented the syntax in that paper. I would support an > effort to add it (unsurprising, since I am a co-author of that paper). > EMIT STREAM is equivalent to the current SELECT STREAM syntax. > There is no equivalent in current Calcite of the EMIT AFTER WATERMARK, or > EMIT STREAM AFTER DELAY. > HOP, TUMBLE and SESSION are supported in Calcite’s SQL parser, but following > the paper would be replaced with a table function call. We could need to add > HOP, TUMBLE and SESSION table functions. We would also need to make the > system aware of how watermarks flow through these table functions (an area > that the paper does not go into). > Julian -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (CALCITE-3271) TVF windowing and EMIT syntax support in Calcite
Danny Chan created CALCITE-3271: --- Summary: TVF windowing and EMIT syntax support in Calcite Key: CALCITE-3271 URL: https://issues.apache.org/jira/browse/CALCITE-3271 Project: Calcite Issue Type: Bug Components: core Affects Versions: 1.20.0 Reporter: Danny Chan Assignee: Danny Chan Copied from the mailing list: Calcite has not implemented the syntax in that paper. I would support an effort to add it (unsurprising, since I am a co-author of that paper). EMIT STREAM is equivalent to the current SELECT STREAM syntax. There is no equivalent in current Calcite of the EMIT AFTER WATERMARK, or EMIT STREAM AFTER DELAY. HOP, TUMBLE and SESSION are supported in Calcite’s SQL parser, but following the paper would be replaced with a table function call. We could need to add HOP, TUMBLE and SESSION table functions. We would also need to make the system aware of how watermarks flow through these table functions (an area that the paper does not go into). Julian -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (CALCITE-3167) Remove overriding equals and hashCode methods in EnumerableTableScan
[
https://issues.apache.org/jira/browse/CALCITE-3167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jin xing resolved CALCITE-3167.
---
Resolution: Fixed
> Remove overriding equals and hashCode methods in EnumerableTableScan
>
>
> Key: CALCITE-3167
> URL: https://issues.apache.org/jira/browse/CALCITE-3167
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.19.0
>Reporter: jin xing
>Assignee: Julian Hyde
>Priority: Minor
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> In current code of {{EnumerableTableScan}}, methods of equals and hashCode
> are overridden for matching of {{EnumerableTableScan}}s.
> While after optimizing with the same HEP planner, {{EnumerableTableScan}}s
> from two plans but with the same digest will the share the same Java object.
> See
> [RelOptMaterializations|https://github.com/apache/calcite/blob/adf4cc4dc5cdb9f5e49c85d10f46a2fdcd831ccf/core/src/main/java/org/apache/calcite/plan/RelOptMaterializations.java#L192].
> I think it's ok to remove the redundant overriding methods in
> {{EnumerableTableScan}}.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (CALCITE-3167) Remove overriding equals and hashCode methods in EnumerableTableScan
[
https://issues.apache.org/jira/browse/CALCITE-3167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911878#comment-16911878
]
jin xing commented on CALCITE-3167:
---
Fixed in
https://github.com/apache/calcite/commit/3f550c3f28d75d0f75b5815a22395dbecf992f03
> Remove overriding equals and hashCode methods in EnumerableTableScan
>
>
> Key: CALCITE-3167
> URL: https://issues.apache.org/jira/browse/CALCITE-3167
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.19.0
>Reporter: jin xing
>Assignee: Julian Hyde
>Priority: Minor
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> In current code of {{EnumerableTableScan}}, methods of equals and hashCode
> are overridden for matching of {{EnumerableTableScan}}s.
> While after optimizing with the same HEP planner, {{EnumerableTableScan}}s
> from two plans but with the same digest will the share the same Java object.
> See
> [RelOptMaterializations|https://github.com/apache/calcite/blob/adf4cc4dc5cdb9f5e49c85d10f46a2fdcd831ccf/core/src/main/java/org/apache/calcite/plan/RelOptMaterializations.java#L192].
> I think it's ok to remove the redundant overriding methods in
> {{EnumerableTableScan}}.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (CALCITE-3167) Remove overriding equals and hashCode methods in EnumerableTableScan
[
https://issues.apache.org/jira/browse/CALCITE-3167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911877#comment-16911877
]
jin xing commented on CALCITE-3167:
---
[~julianhyde]
Thanks a lot for merging !
> Remove overriding equals and hashCode methods in EnumerableTableScan
>
>
> Key: CALCITE-3167
> URL: https://issues.apache.org/jira/browse/CALCITE-3167
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.19.0
>Reporter: jin xing
>Assignee: Julian Hyde
>Priority: Minor
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> In current code of {{EnumerableTableScan}}, methods of equals and hashCode
> are overridden for matching of {{EnumerableTableScan}}s.
> While after optimizing with the same HEP planner, {{EnumerableTableScan}}s
> from two plans but with the same digest will the share the same Java object.
> See
> [RelOptMaterializations|https://github.com/apache/calcite/blob/adf4cc4dc5cdb9f5e49c85d10f46a2fdcd831ccf/core/src/main/java/org/apache/calcite/plan/RelOptMaterializations.java#L192].
> I think it's ok to remove the redundant overriding methods in
> {{EnumerableTableScan}}.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (CALCITE-3251) BinaryExpression evaluate method support Long type.
[ https://issues.apache.org/jira/browse/CALCITE-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chan updated CALCITE-3251: Component/s: core Affects Version/s: 1.19.0 1.20.0 > BinaryExpression evaluate method support Long type. > --- > > Key: CALCITE-3251 > URL: https://issues.apache.org/jira/browse/CALCITE-3251 > Project: Calcite > Issue Type: Bug > Components: core >Affects Versions: 1.19.0, 1.20.0 >Reporter: xzh_dz >Assignee: Danny Chan >Priority: Minor > Labels: pull-request-available > Fix For: 1.21.0 > > Time Spent: 4h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (CALCITE-3265) Remove useless code in ExpressionTest
[ https://issues.apache.org/jira/browse/CALCITE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chan updated CALCITE-3265: Fix Version/s: 1.21.0 Affects Version/s: 1.20.0 > Remove useless code in ExpressionTest > - > > Key: CALCITE-3265 > URL: https://issues.apache.org/jira/browse/CALCITE-3265 > Project: Calcite > Issue Type: Wish >Affects Versions: 1.20.0 >Reporter: xzh_dz >Assignee: Danny Chan >Priority: Minor > Labels: pull-request-available > Fix For: 1.21.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (CALCITE-3265) Remove useless code in ExpressionTest
[ https://issues.apache.org/jira/browse/CALCITE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chan updated CALCITE-3265: Summary: Remove useless code in ExpressionTest (was: remove useless code) > Remove useless code in ExpressionTest > - > > Key: CALCITE-3265 > URL: https://issues.apache.org/jira/browse/CALCITE-3265 > Project: Calcite > Issue Type: Wish >Reporter: xzh_dz >Priority: Minor > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (CALCITE-3265) Remove useless code in ExpressionTest
[ https://issues.apache.org/jira/browse/CALCITE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chan resolved CALCITE-3265. - Assignee: Danny Chan Resolution: Fixed Fixed in [5a42812|https://github.com/apache/calcite/commit/5a42812832e51a7551a793ac75160188e06668e6], thanks for your PR, [~xzh_dz] ! > Remove useless code in ExpressionTest > - > > Key: CALCITE-3265 > URL: https://issues.apache.org/jira/browse/CALCITE-3265 > Project: Calcite > Issue Type: Wish >Reporter: xzh_dz >Assignee: Danny Chan >Priority: Minor > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (CALCITE-3251) BinaryExpression evaluate method support Long type.
[ https://issues.apache.org/jira/browse/CALCITE-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chan resolved CALCITE-3251. - Fix Version/s: 1.21.0 Assignee: Danny Chan Resolution: Fixed Fixed in [5a42812|https://github.com/apache/calcite/commit/5a42812832e51a7551a793ac75160188e06668e6], thanks for your PR, [~xzh_dz] ! > BinaryExpression evaluate method support Long type. > --- > > Key: CALCITE-3251 > URL: https://issues.apache.org/jira/browse/CALCITE-3251 > Project: Calcite > Issue Type: Bug >Reporter: xzh_dz >Assignee: Danny Chan >Priority: Minor > Labels: pull-request-available > Fix For: 1.21.0 > > Time Spent: 4h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (CALCITE-3259) Align 'Property' in the serialized xml string of RelXmlWriter.
[
https://issues.apache.org/jira/browse/CALCITE-3259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Haisheng Yuan resolved CALCITE-3259.
Fix Version/s: 1.21.0
Resolution: Fixed
Fixed in
https://github.com/apache/calcite/commit/beeb012c994642e2db46c30e6059f1f1c22287c9,
thanks for the PR, [~yanlin-Lynn]!
> Align 'Property' in the serialized xml string of RelXmlWriter.
> --
>
> Key: CALCITE-3259
> URL: https://issues.apache.org/jira/browse/CALCITE-3259
> Project: Calcite
> Issue Type: Improvement
>Reporter: Wang Yanlin
>Priority: Minor
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> In the serialized xml string of RelXmlWriter, the 'Property' label is
> aligned.
> For the sql below
> {noformat}
> select 1 + 2, 3 from (values (true))
> {noformat}
> the serialized xml string is like this:
> {noformat}
>
>
> +(1, 2)
>
> 3
>
>
>
> [{ true }]
>
>
>
>
> {noformat}
> It's better to make 'Property' aligned.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (CALCITE-2166) Cumulative cost of RelSubset.best RelNode is increased after calling RelSubset.propagateCostImprovements() for input RelNodes
[
https://issues.apache.org/jira/browse/CALCITE-2166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911714#comment-16911714
]
Xiening Dai commented on CALCITE-2166:
--
hi [~danny0405][~vvysotskyi], not sure what is the progress of this issue. I
spend some time looking into this and have a couple of proposals as below
(ranking from the easier to the hardiest) -
1. In propagateCostImprovements0() if we find bestrel cost is increased, we
just updated RelSubset.bestCost
This is the most trivial fix we can do. With that we can guarantee the internal
state of memo is consistent, but the plan generated can be sub-optimal.
2. In propagateCostImprovements0() if we find bestrel cost is increased, we
recompute cost for all rels within current subset, and choose the cheapest one
as the best.
This goes one step further than #1 to look for a potentially better plan when
the original bestrel cost is increased. Although this may generate better plans
under some cases, it is still not able to guarantee the optimal result.
3. Redesign the whole bestrel logic, so a given subset could have more than one
bestrel, and each best candidate would map to one or more parents.
The issue here is the fundamental assumption of our DP search model is broken
by exception case like this bug. The best of current subset does not
necessarliy uses the best of the input subset, because although the best of
input subset grantees the minimal of input cost, it doesn't necessarily
minimize the self cost of parent node since the row count could increase. With
this in mind, in order to find the optimal plan, we would need to have multiple
"best" candiates and take input rows into account when calculate parent's best.
Theoretically this would grantee an optimal plan, but it's fairly complex and
could incur significant overhead on runtime performance.
At this moment, I'd prefer proposal #1. #3 is too complex and requires drastic
change on the core framework which I don't see very feasible. #2 adds overhead
but again, same as #1, it doesn't grantee an optimal plan.
Let me know your thoughts. I would love to see this resolved so we can move on
to CALCITE-2018 which I believe is fairly important to the functionality of
core framework.
> Cumulative cost of RelSubset.best RelNode is increased after calling
> RelSubset.propagateCostImprovements() for input RelNodes
> -
>
> Key: CALCITE-2166
> URL: https://issues.apache.org/jira/browse/CALCITE-2166
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.15.0
>Reporter: Volodymyr Vysotskyi
>Assignee: Danny Chan
>Priority: Critical
>
> After calling {{RelSubset.propagateCostImprovements()}} cumulative cost of
> {{RelSubset.best}} {{RelNode}} may be increased due to the increase of the
> non-cumulative cost caused by changing of input best {{RelNode}}.
> To observe this issue, add this code:
> {code:java}
> if (subset.best != null) {
> RelOptCost bestCost = getCost(subset.best,
> RelMetadataQuery.instance());
> if (!subset.bestCost.equals(bestCost)) {
> throw new AssertionError(
> "relSubset [" + subset.getDescription()
> + "] has wrong best cost "
> + subset.bestCost + ". Correct cost is " + bestCost);
> }
> }
> {code}
> into {{VolcanoPlanner.validate()}} method (line 907).
> List of unit tests which fail with this check:
> {noformat}
> Failed tests:
>
> MaterializationTest.testJoinMaterializationUKFK9:1823->checkMaterialize:198->checkMaterialize:205->checkThatMaterialize:233
> relSubset [rel#226287:Subset#8.ENUMERABLE.[]] has wrong best cost {221.5
> rows, 128.25 cpu, 0.0 io}. Correct cost is {233.0 rows, 178.0 cpu, 0.0 io}
> ScannableTableTest.testPFPushDownProjectFilterAggregateNested:279 relSubset
> [rel#12950:Subset#5.ENUMERABLE.[]] has wrong best cost {63.8 rows, 62.308
> cpu, 0.0 io}. Correct cost is {70.4 rows, 60.404 cpu, 0.0 io}
> ScannableTableTest.testPFTableRefusesFilterCooperative:221 relSubset
> [rel#13382:Subset#2.ENUMERABLE.[]] has wrong best cost {81.0 rows, 181.01
> cpu, 0.0 io}. Correct cost is {150.5 rows, 250.505 cpu, 0.0 io}
> ScannableTableTest.testProjectableFilterableCooperative:148 relSubset
> [rel#13611:Subset#2.ENUMERABLE.[]] has wrong best cost {81.0 rows, 181.01
> cpu, 0.0 io}. Correct cost is {150.5 rows, 250.505 cpu, 0.0 io}
> ScannableTableTest.testProjectableFilterableNonCooperative:165 relSubset
> [rel#13754:Subset#2.ENUMERABLE.[]] has wrong best cost {81.0 rows, 181.01
> cpu, 0.0 io}. Correct cost is {150.5 rows, 250.505 cpu, 0.0 io}
> FrameworksTest.testUpdate:336->executeQuery:367
[jira] [Commented] (CALCITE-1917) Support column reference in "FOR SYSTEM_TIME AS OF"
[
https://issues.apache.org/jira/browse/CALCITE-1917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911631#comment-16911631
]
Julian Hyde commented on CALCITE-1917:
--
{{orders.rowtime}} is a system column. I'd feel more confident if we allowed
non-system columns. And also if we had some positive and negative validator
tests. Lastly, make clear that this applies to parse, validate, sql2rel but we
still cannot execute.
> Support column reference in "FOR SYSTEM_TIME AS OF"
> ---
>
> Key: CALCITE-1917
> URL: https://issues.apache.org/jira/browse/CALCITE-1917
> Project: Calcite
> Issue Type: New Feature
>Reporter: Jark Wu
>Priority: Major
>
> As discussed in mailing list[1], the standard says QSTPS can’t contain a
> column reference. So when joining the Orders to the Products table for the
> price as of the time the order was placed is impossible using "FOR
> SYSTEM_TIME AS OF". But can be expressed using a subquery, such as:
> {code}
> SELECT *
> FROM Orders AS o
> JOIN LATERAL (SELECT * FROM ProductPrices WHERE sysStart <= O.orderTime
> AND sysEnd > O.orderTime) AS P
> ON o.productId = p.productId
> {code}
> But subquery is too complex for users. We know that the standard says it
> can’t contain a column reference. We initialize this discuss as we would like
> to "extend" the standard to simplify such query:
> {code}
> SELECT *
> FROM Orders AS o
> JOIN LATERAL ProductPrices FOR SYSTEM_TIME AS OF O.orderTime AS P
> ON o.productId = p.productId
> {code}
> [1]
> https://lists.apache.org/thread.html/f877f356a8365bf74ea7d8e4a171224104d653cf73861afb2901a58f@%3Cdev.calcite.apache.org%3E
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (CALCITE-3013) support CeilFuncBigInt TO
[ https://issues.apache.org/jira/browse/CALCITE-3013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911627#comment-16911627 ] Julian Hyde commented on CALCITE-3013: -- I'd be surprised if Oracle can cast integer to date (either explicitly or implicitly). Oracle's dates (like SQL dates) are not based on the Unix Epoch, or any epoch. > support CeilFuncBigInt TO > -- > > Key: CALCITE-3013 > URL: https://issues.apache.org/jira/browse/CALCITE-3013 > Project: Calcite > Issue Type: Test >Reporter: xzh_dz >Priority: Trivial > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (CALCITE-3270) Close resources
[ https://issues.apache.org/jira/browse/CALCITE-3270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911625#comment-16911625 ] Julian Hyde commented on CALCITE-3270: -- Please add more detail. As this stands I could legitimately close as "cannot reproduce". > Close resources > --- > > Key: CALCITE-3270 > URL: https://issues.apache.org/jira/browse/CALCITE-3270 > Project: Calcite > Issue Type: Task >Reporter: Fokko Driesprong >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (CALCITE-3261) Make JDBC output column labels more human readable
[
https://issues.apache.org/jira/browse/CALCITE-3261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911617#comment-16911617
]
Julian Hyde commented on CALCITE-3261:
--
I agree, it might be nice.
We must not change the column aliases automatically assigned for sub-queries
and other relational algebra nodes. The "EXPR$n" logic is called widely, and
appears in a lot of test output.
At the top level of a statement you could derive the labels for JDBC based on
the SqlParserPos of the items in the SELECT clause.
What does MySQL do if the expression has an alias? if the expression has an
alias that contains a space? if the expression is of the form
"tableAlias.column"? if the SQL text is too long? if it contains spaces? if it
contains back-ticks or double-quotes (which are difficult to escape)? if the
SQL text is not unique? if the expression contains two or more consecutive
spaces? if the expression is "null"? It's worth writing down all of these
examples, and more, as use cases.
> Make JDBC output column labels more human readable
> --
>
> Key: CALCITE-3261
> URL: https://issues.apache.org/jira/browse/CALCITE-3261
> Project: Calcite
> Issue Type: Improvement
> Components: core, jdbc-adapter
>Reporter: Xiaoguang zhang
>Assignee: Xiaoguang Zhang
>Priority: Major
>
> After implementing the missing `print(ResultSet)` method from the examples in
> Calcite [tutorial|[https://calcite.apache.org/docs/index.html]], I found that
> the output field names are incorrect:
> {code:java}
> [deptno, EXPR$1]
> [1, 1]
> [2, 2]
> {code}
> The field name `EXPR$1` can be improved to be more human readable like MySQL
> client:
> {code:java}
> ++--+
> | deptno | min(e.empid) |
> ++--+
> | 1 |1 |
> | 2 |2 |
> ++--+
> {code}
>
> PS: the sql statement generating the above outputs:
> {code:sql}
> select d.deptno, min(e.empid) from hr.emps as e join hr.depts as d on
> e.deptno = d.deptno group by d.deptno having count(*) > 1
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (CALCITE-3260) Add support of evaluate method with default Evaluator.
[
https://issues.apache.org/jira/browse/CALCITE-3260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911605#comment-16911605
]
Julian Hyde commented on CALCITE-3260:
--
Instead, in {{Expressions}}, add a method {{public static Object
evaluate(Expression)}}. It's better if everything isn't public.
> Add support of evaluate method with default Evaluator.
> --
>
> Key: CALCITE-3260
> URL: https://issues.apache.org/jira/browse/CALCITE-3260
> Project: Calcite
> Issue Type: New Feature
>Reporter: Wang Yanlin
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently, the public method *evaluate* of AbstractNode need a Evaluator
> object as parameter, but Evaluator class has default access control. This
> limit the access of *evaluate* method. So may be we can add an overload
> *evaluate* method with default Evaluator, thus, allow to evaluate Expression
> directly.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (CALCITE-3243) Incomplete validation of operands in JSON functions
[
https://issues.apache.org/jira/browse/CALCITE-3243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911477#comment-16911477
]
Matt Wang commented on CALCITE-3243:
[~zabetak] I will work on it.
> Incomplete validation of operands in JSON functions
> -
>
> Key: CALCITE-3243
> URL: https://issues.apache.org/jira/browse/CALCITE-3243
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.20.0
>Reporter: Stamatis Zampetakis
>Assignee: Matt Wang
>Priority: Major
>
> The operands of various JSON functions are not validated correctly.
> Consider for instance the {{JSON_VALUE}} function and the following calls:
> {code:sql}
> json_value('{\"foo\":100}', 'strict $.foo')"
> json_value(1, 'strict $.foo')"
> json_value(TRUE, 'strict $.foo')"
> {code}
> The first call is legal but the next are not; if I am not wrong the operands
> should be always CHAR or VARCHAR literals which is not the case above.
> Queries involving such calls fail at runtime when compiling generated code. I
> guess that such kind of problems should be captured by the validator and we
> should not even arrive at the code generation step.
> The problems can be easily reproduced by adding such calls in the
> {{SqlOperatorBaseTest}}.
> The problem does not only affect {{JSON_VALUE}} but other JSON functions
> (such as {{JSON_QUERY}} etc.) as well.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (CALCITE-2589) VolcanoPlanner#fireRules and VolcanoRuleCall#matchRecurse should ignore known-to-be-unimportant relations
[
https://issues.apache.org/jira/browse/CALCITE-2589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stamatis Zampetakis updated CALCITE-2589:
-
Fix Version/s: (was: 1.21.0)
1.22.0
> VolcanoPlanner#fireRules and VolcanoRuleCall#matchRecurse should ignore
> known-to-be-unimportant relations
> -
>
> Key: CALCITE-2589
> URL: https://issues.apache.org/jira/browse/CALCITE-2589
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Affects Versions: 1.17.0
>Reporter: Vladimir Sitnikov
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.22.0
>
>
> {{call.getPlanner().setImportance}} is used to avoid use of
> known-to-be-inefficient relation, however the check of importance is
> performed very late.
> The check is performed in org.apache.calcite.plan.volcano.RuleQueue#skipMatch
> when ruleCalls have already been created.
> I suggest to move the check into VolcanoPlanner#fireRules and
> VolcanoRuleCall#matchRecurse
> It would reduce amount of "possible" rule executions.
> Note: calling setImportance BEFORE transformTo would would help as well to
> filter out unimportant rule calls early.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Resolved] (CALCITE-1882) Can't obtain the user defined aggregate function such as sum,avg by calcite
[
https://issues.apache.org/jira/browse/CALCITE-1882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stamatis Zampetakis resolved CALCITE-1882.
--
Resolution: Cannot Reproduce
I am marking this as resolved since after CALCITE-2282 the issue should no
longer appear.
> Can't obtain the user defined aggregate function such as sum,avg by calcite
> ---
>
> Key: CALCITE-1882
> URL: https://issues.apache.org/jira/browse/CALCITE-1882
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.12.0
>Reporter: yuemeng
>Assignee: yuemeng
>Priority: Critical
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> If we want to register a sum or avg aggregate function to deal with different
> data type such as sum(double) ,we implement a SqlUserDefinedAggFunction and
> register with name sum,but when we execute a sql like:
> {code}
> select id,sum(payment) from table test group by id
> {code}
> in fact,it always give the SqlSumAggFunction function which builtin by
> calcite,never find the exactly function which we register by ourself.
> During sql parse process,method createCall in SqlAbstractParserImpl will be
> called,
> {code}
> protected SqlCall createCall(
> SqlIdentifier funName,
> SqlParserPos pos,
> SqlFunctionCategory funcType,
> SqlLiteral functionQualifier,
> SqlNode[] operands) {
> SqlOperator fun = null;
> // First, try a half-hearted resolution as a builtin function.
> // If we find one, use it; this will guarantee that we
> // preserve the correct syntax (i.e. don't quote builtin function
> /// name when regenerating SQL).
> if (funName.isSimple()) {
> final List list = Lists.newArrayList();
> opTab.lookupOperatorOverloads(funName, funcType, SqlSyntax.FUNCTION,
> list);//we lookup in SqlStdOperatorTable,and always find buidin function for
> sum ,avg .eg
> if (list.size() == 1) {
> fun = list.get(0);
> }
> }
> // Otherwise, just create a placeholder function. Later, during
> // validation, it will be resolved into a real function reference.
> if (fun == null) {
> fun = new SqlUnresolvedFunction(funName, null, null, null, null,
> funcType);
> }
> return fun.createCall(functionQualifier, pos, operands);
> }
> {code}
> but the problem will be appear in deriveType,because of we get the
> SqlSumAggFunction previously,and the sqlKind of SqlSumAggFunction is AVG,but
> the sqlKind of sql user defined agg function (sum) which we register by
> ourself is OTHER_FUNCTION,so it filter out our sum function.
> {code}
> private static Iterator
> filterOperatorRoutinesByKind(Iterator routines,
> final SqlKind sqlKind) {
> return Iterators.filter(routines,
> new PredicateImpl() {
> public boolean test(SqlOperator input) {
> return input.getKind() == sqlKind;
> }
> });
> }
> {code}
> that cause we never obtain sum function which registered by user .
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (CALCITE-3140) Multiple failures when executing slow tests
[
https://issues.apache.org/jira/browse/CALCITE-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stamatis Zampetakis updated CALCITE-3140:
-
Fix Version/s: (was: 1.21.0)
1.22.0
> Multiple failures when executing slow tests
> ---
>
> Key: CALCITE-3140
> URL: https://issues.apache.org/jira/browse/CALCITE-3140
> Project: Calcite
> Issue Type: Bug
>Affects Versions: 1.18.0, 1.19.0, 1.20.0
> Environment: Windows 10 Pro, jdk1.8.0_202, maven 3.6.1 with 8G memory
> allocated to the VM (Xmx).
>Reporter: Stamatis Zampetakis
>Assignee: Stamatis Zampetakis
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.22.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Running slow tests using the following command
> {noformat}
> mvn clean install -Dcalcite.test.slow -fn
> {noformat}
> finishes with the following errors.
> {noformat}
> [ERROR] Tests run: 12620, Failures: 0, Errors: 6, Skipped: 6310, Time
> elapsed: 2,522.456 s <<< FAILURE! - in org.apache.calcite.test.FoodmartTest
> (SOE + Timeouts)
> [ERROR] Tests run: 256, Failures: 0, Errors: 1, Skipped: 1, Time elapsed:
> 10.448 s <<< FAILURE! - in org.apache.calcite.sql.test.SqlOperatorTest
> [ERROR] Tests run: 17, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 49.711 s <<< FAILURE! - in org.apache.calcite.test.CoreQuidemTest
> (Intermittent failures/Plan comparison)
> [ERROR] Tests run: 256, Failures: 0, Errors: 1, Skipped: 1, Time elapsed:
> 19.759 s <<< FAILURE! - in org.apache.calcite.test.CalciteSqlOperatorTest
> [ERROR] Tests run: 11, Failures: 1, Errors: 0, Skipped: 7, Time elapsed:
> 8.132 s <<< FAILURE! - in org.apache.calcite.adapter.tpcds.TpcdsTest (Stale
> plans)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (CALCITE-3141) Slow tests are not run in continuous integration
[ https://issues.apache.org/jira/browse/CALCITE-3141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stamatis Zampetakis updated CALCITE-3141: - Fix Version/s: (was: 1.21.0) 1.22.0 > Slow tests are not run in continuous integration > > > Key: CALCITE-3141 > URL: https://issues.apache.org/jira/browse/CALCITE-3141 > Project: Calcite > Issue Type: Bug >Affects Versions: 1.19.0, 1.20.0 >Reporter: Stamatis Zampetakis >Assignee: Stamatis Zampetakis >Priority: Major > Fix For: 1.22.0 > > > Trying to execute slow tests locally always ends up with failures and these > failures are not reflected in continuous integration (Jenkins, AppVeyor, > Travis). > > From the CI configuration below (extract from the logs) this appears normal > for Jenkins and AppVeyor but not for Travis. > > *Jenkins:* > /home/jenkins/tools/maven/apache-maven-3.3.3/bin/mvn -f ws/pom.xml > "-Dmaven.repo.local=/home/jenkins/jenkins-slave/workspace/Calcite-Master/jdk/JDK > 1.8 (latest)/label_exp/ubuntu&&!cloud-slave&&!H27/.repository" clean install > javadoc:javadoc -DskipITs -B > > *Travis:* > $ if [ $SLOW_TESTS = "Y" ]; then export > TESTS="-Dgroups=org.apache.calcite.test.SlowTests --projects :calcite-core"; > else export TESTS=-DskipSlowTests; fi > The command "if [ $SLOW_TESTS = "Y" ]; then export > TESTS="-Dgroups=org.apache.calcite.test.SlowTests --projects :calcite-core"; > else export TESTS=-DskipSlowTests; fi" exited with 0. > > *AppVeyor:* > mvn -Dcheckstyle.skip -Dsurefire.useFile=false -Dsurefire.threadCount=1 > -Dsurefire.perCoreThreadCount=false > -Djavax.net.ssl.trustStorePassword=changeit test -Djna.nosys=true -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Comment Edited] (CALCITE-3260) Add support of evaluate method with default Evaluator.
[ https://issues.apache.org/jira/browse/CALCITE-3260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911351#comment-16911351 ] Wang Yanlin edited comment on CALCITE-3260 at 8/20/19 1:31 PM: --- Well, then can we make class *Evaluator* public, so that, we can also be able to evaluate Expression directly. [~julianhyde] was (Author: yanlin-lynn): Well, then can we make class *Evaluator* public, so that, we can also be able to evaluate Expression directly. > Add support of evaluate method with default Evaluator. > -- > > Key: CALCITE-3260 > URL: https://issues.apache.org/jira/browse/CALCITE-3260 > Project: Calcite > Issue Type: New Feature >Reporter: Wang Yanlin >Priority: Minor > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Currently, the public method *evaluate* of AbstractNode need a Evaluator > object as parameter, but Evaluator class has default access control. This > limit the access of *evaluate* method. So may be we can add an overload > *evaluate* method with default Evaluator, thus, allow to evaluate Expression > directly. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (CALCITE-3260) Add support of evaluate method with default Evaluator.
[ https://issues.apache.org/jira/browse/CALCITE-3260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911351#comment-16911351 ] Wang Yanlin commented on CALCITE-3260: -- Well, then can we make class *Evaluator* public, so that, we can also be able to evaluate Expression directly. > Add support of evaluate method with default Evaluator. > -- > > Key: CALCITE-3260 > URL: https://issues.apache.org/jira/browse/CALCITE-3260 > Project: Calcite > Issue Type: New Feature >Reporter: Wang Yanlin >Priority: Minor > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Currently, the public method *evaluate* of AbstractNode need a Evaluator > object as parameter, but Evaluator class has default access control. This > limit the access of *evaluate* method. So may be we can add an overload > *evaluate* method with default Evaluator, thus, allow to evaluate Expression > directly. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (CALCITE-3261) Make JDBC output column labels more human readable
[
https://issues.apache.org/jira/browse/CALCITE-3261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaoguang Zhang reassigned CALCITE-3261:
Assignee: Xiaoguang Zhang
> Make JDBC output column labels more human readable
> --
>
> Key: CALCITE-3261
> URL: https://issues.apache.org/jira/browse/CALCITE-3261
> Project: Calcite
> Issue Type: Improvement
> Components: core, jdbc-adapter
>Reporter: Xiaoguang zhang
>Assignee: Xiaoguang Zhang
>Priority: Major
>
> After implementing the missing `print(ResultSet)` method from the examples in
> Calcite [tutorial|[https://calcite.apache.org/docs/index.html]], I found that
> the output field names are incorrect:
> {code:java}
> [deptno, EXPR$1]
> [1, 1]
> [2, 2]
> {code}
> The field name `EXPR$1` can be improved to be more human readable like MySQL
> client:
> {code:java}
> ++--+
> | deptno | min(e.empid) |
> ++--+
> | 1 |1 |
> | 2 |2 |
> ++--+
> {code}
>
> PS: the sql statement generating the above outputs:
> {code:sql}
> select d.deptno, min(e.empid) from hr.emps as e join hr.depts as d on
> e.deptno = d.deptno group by d.deptno having count(*) > 1
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Comment Edited] (CALCITE-3261) Make JDBC output column labels more human readable
[
https://issues.apache.org/jira/browse/CALCITE-3261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911323#comment-16911323
]
Xiaoguang Zhang edited comment on CALCITE-3261 at 8/20/19 1:14 PM:
---
Thanks [~danny0405], [~julianhyde] for replaying.
I agree with [~julianhyde] that this is indeed not an incorrect behavior, it's
more of a potential improvement. The issue type is marked as "Improvement" but
the title is misleading, I'll change it later.
I've did more tests and it shows that the column labels are `min(e.empid)` from
both MySQL CLI and JDBC driver. Besides, from a user's perspective, I think
`min(e.empid)` is more friendly. What do you think?
Seems like this is a nice-to-have issue, I would like to give it a try if you
don't mind.
was (Author: zhxiaog):
Thanks [~danny0405] [~julianhyde] for replaying.
I agree with [~julianhyde] that this is indeed not a incorrect behavior, it's
more of a potential improvement. The issue type is marked as "Improvement" but
the title is misleading, I'll change it later.
I've did more tests and it shows that the column labels are `min(e.empid)` from
both MySQL CLI and JDBC driver. Besides, from a user's perspective, I think
`min(e.empid)` is more friendly. What do you think?
Seems like this is a nice-to-have issue, I would like to give it a try if you
don't mind.
> Make JDBC output column labels more human readable
> --
>
> Key: CALCITE-3261
> URL: https://issues.apache.org/jira/browse/CALCITE-3261
> Project: Calcite
> Issue Type: Improvement
> Components: core, jdbc-adapter
>Reporter: Xiaoguang zhang
>Priority: Major
>
> After implementing the missing `print(ResultSet)` method from the examples in
> Calcite [tutorial|[https://calcite.apache.org/docs/index.html]], I found that
> the output field names are incorrect:
> {code:java}
> [deptno, EXPR$1]
> [1, 1]
> [2, 2]
> {code}
> The field name `EXPR$1` can be improved to be more human readable like MySQL
> client:
> {code:java}
> ++--+
> | deptno | min(e.empid) |
> ++--+
> | 1 |1 |
> | 2 |2 |
> ++--+
> {code}
>
> PS: the sql statement generating the above outputs:
> {code:sql}
> select d.deptno, min(e.empid) from hr.emps as e join hr.depts as d on
> e.deptno = d.deptno group by d.deptno having count(*) > 1
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (CALCITE-3261) Make JDBC output column labels more human readable
[
https://issues.apache.org/jira/browse/CALCITE-3261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaoguang Zhang updated CALCITE-3261:
-
Summary: Make JDBC output column labels more human readable (was:
Incorrect output field names from jdbc)
> Make JDBC output column labels more human readable
> --
>
> Key: CALCITE-3261
> URL: https://issues.apache.org/jira/browse/CALCITE-3261
> Project: Calcite
> Issue Type: Improvement
> Components: core, jdbc-adapter
>Reporter: Xiaoguang zhang
>Priority: Major
>
> After implementing the missing `print(ResultSet)` method from the examples in
> Calcite [tutorial|[https://calcite.apache.org/docs/index.html]], I found that
> the output field names are incorrect:
> {code:java}
> [deptno, EXPR$1]
> [1, 1]
> [2, 2]
> {code}
> The field name `EXPR$1` can be improved to be more human readable like MySQL
> client:
> {code:java}
> ++--+
> | deptno | min(e.empid) |
> ++--+
> | 1 |1 |
> | 2 |2 |
> ++--+
> {code}
>
> PS: the sql statement generating the above outputs:
> {code:sql}
> select d.deptno, min(e.empid) from hr.emps as e join hr.depts as d on
> e.deptno = d.deptno group by d.deptno having count(*) > 1
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (CALCITE-3261) Incorrect output field names from jdbc
[
https://issues.apache.org/jira/browse/CALCITE-3261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911323#comment-16911323
]
Xiaoguang Zhang commented on CALCITE-3261:
--
Thanks [~danny0405] [~julianhyde] for replaying.
I agree with [~julianhyde] that this is indeed not a incorrect behavior, it's
more of a potential improvement. The issue type is marked as "Improvement" but
the title is misleading, I'll change it later.
I've did more tests and it shows that the column labels are `min(e.empid)` from
both MySQL CLI and JDBC driver. Besides, from a user's perspective, I think
`min(e.empid)` is more friendly. What do you think?
Seems like this is a nice-to-have issue, I would like to give it a try if you
don't mind.
> Incorrect output field names from jdbc
> --
>
> Key: CALCITE-3261
> URL: https://issues.apache.org/jira/browse/CALCITE-3261
> Project: Calcite
> Issue Type: Improvement
> Components: core, jdbc-adapter
>Reporter: Xiaoguang zhang
>Priority: Major
>
> After implementing the missing `print(ResultSet)` method from the examples in
> Calcite [tutorial|[https://calcite.apache.org/docs/index.html]], I found that
> the output field names are incorrect:
> {code:java}
> [deptno, EXPR$1]
> [1, 1]
> [2, 2]
> {code}
> The field name `EXPR$1` can be improved to be more human readable like MySQL
> client:
> {code:java}
> ++--+
> | deptno | min(e.empid) |
> ++--+
> | 1 |1 |
> | 2 |2 |
> ++--+
> {code}
>
> PS: the sql statement generating the above outputs:
> {code:sql}
> select d.deptno, min(e.empid) from hr.emps as e join hr.depts as d on
> e.deptno = d.deptno group by d.deptno having count(*) > 1
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Created] (CALCITE-3270) Close resources
Fokko Driesprong created CALCITE-3270: - Summary: Close resources Key: CALCITE-3270 URL: https://issues.apache.org/jira/browse/CALCITE-3270 Project: Calcite Issue Type: Task Reporter: Fokko Driesprong -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (CALCITE-3115) Cannot add JdbcRules which have different JdbcConvention to same VolcanoPlanner's RuleSet.
[
https://issues.apache.org/jira/browse/CALCITE-3115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911238#comment-16911238
]
Igor Guzenko commented on CALCITE-3115:
---
Hello [~zabetak] and [~vvysotskyi] , I've assigned the issue to me. I'll
preserve efforts made by [~winipanda] , next goal is just to address review
comments.
> Cannot add JdbcRules which have different JdbcConvention to same
> VolcanoPlanner's RuleSet.
> --
>
> Key: CALCITE-3115
> URL: https://issues.apache.org/jira/browse/CALCITE-3115
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.19.0
>Reporter: TANG Wen-hui
>Assignee: Igor Guzenko
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> When we use Calcite via JDBC to run a sql which involves two difference jdbc
> schema:
> {code:java}
> select * from (select "a",max("b") as max_b, sum("c") as sum_c from
> "test"."temp" where "d" > 10 or "b" <> 'hello' group by "a", "e", "f" having
> "a" > 100 and max("b") < 20 limit 10) t union select "a", "b","c" from
> "test2"."temp2" group by "a","b","c"
> {code}
> the sql get a plan like that:
> {code:java}
> EnumerableUnion(all=[false])
> JdbcToEnumerableConverter
> JdbcProject(a=[$0], MAX_B=[$3], SUM_C=[$4])
> JdbcSort(fetch=[10])
> JdbcFilter(condition=[<(CAST($3):BIGINT, 20)])
> JdbcAggregate(group=[{0, 4, 5}], MAX_B=[MAX($1)], SUM_C=[SUM($2)])
> JdbcFilter(condition=[AND(OR(>($3, 10), <>($1, 'hello')), >($0,
> 100))])
> JdbcTableScan(table=[[test, temp]])
> EnumerableAggregate(group=[{0, 1, 2}])
> JdbcToEnumerableConverter
> JdbcTableScan(table=[[test2, temp2]])
> {code}
> And the EnumerableAggregate for table test2.temp2 cannot be converted to
> JdbcAggregate.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Assigned] (CALCITE-3115) Cannot add JdbcRules which have different JdbcConvention to same VolcanoPlanner's RuleSet.
[
https://issues.apache.org/jira/browse/CALCITE-3115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Guzenko reassigned CALCITE-3115:
-
Assignee: Igor Guzenko (was: TANG Wen-hui)
> Cannot add JdbcRules which have different JdbcConvention to same
> VolcanoPlanner's RuleSet.
> --
>
> Key: CALCITE-3115
> URL: https://issues.apache.org/jira/browse/CALCITE-3115
> Project: Calcite
> Issue Type: Bug
> Components: core
>Affects Versions: 1.19.0
>Reporter: TANG Wen-hui
>Assignee: Igor Guzenko
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.21.0
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> When we use Calcite via JDBC to run a sql which involves two difference jdbc
> schema:
> {code:java}
> select * from (select "a",max("b") as max_b, sum("c") as sum_c from
> "test"."temp" where "d" > 10 or "b" <> 'hello' group by "a", "e", "f" having
> "a" > 100 and max("b") < 20 limit 10) t union select "a", "b","c" from
> "test2"."temp2" group by "a","b","c"
> {code}
> the sql get a plan like that:
> {code:java}
> EnumerableUnion(all=[false])
> JdbcToEnumerableConverter
> JdbcProject(a=[$0], MAX_B=[$3], SUM_C=[$4])
> JdbcSort(fetch=[10])
> JdbcFilter(condition=[<(CAST($3):BIGINT, 20)])
> JdbcAggregate(group=[{0, 4, 5}], MAX_B=[MAX($1)], SUM_C=[SUM($2)])
> JdbcFilter(condition=[AND(OR(>($3, 10), <>($1, 'hello')), >($0,
> 100))])
> JdbcTableScan(table=[[test, temp]])
> EnumerableAggregate(group=[{0, 1, 2}])
> JdbcToEnumerableConverter
> JdbcTableScan(table=[[test2, temp2]])
> {code}
> And the EnumerableAggregate for table test2.temp2 cannot be converted to
> JdbcAggregate.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Closed] (CALCITE-3266) Type mismatch when test sql “select distinct deptno from dept”
[
https://issues.apache.org/jira/browse/CALCITE-3266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Water Cut Off closed CALCITE-3266.
--
Resolution: Not A Bug
> Type mismatch when test sql “select distinct deptno from dept”
> --
>
> Key: CALCITE-3266
> URL: https://issues.apache.org/jira/browse/CALCITE-3266
> Project: Calcite
> Issue Type: Bug
>Reporter: Water Cut Off
>Priority: Major
> Attachments: image-2019-08-19-20-53-57-276.png,
> image-2019-08-19-20-54-21-347.png
>
>
> h2. *Hi， I don't understand why this happens like this, can someone tell me
> how to solve this and why,thanks a lot.*
>
> Exception in thread "main" java.lang.AssertionError: Type mismatch:Exception
> in thread "main" {color:#FF}java.lang.AssertionError: Type
> mismatch:rowtype of new rel:RecordType(INTEGER NOT NULL DEPTNO, BOOLEAN NOT
> NULL i$DEPTNO) NOT NULLrowtype of set:RecordType(INTEGER NOT NULL DEPTNO) NOT
> NULL{color} at org.apache.calcite.util.Litmus$1.fail(Litmus.java:31) at
> org.apache.calcite.plan.RelOptUtil.equal(RelOptUtil.java:1858) at
> org.apache.calcite.plan.volcano.RelSubset.add(RelSubset.java:284) at
> org.apache.calcite.plan.volcano.RelSet.add(RelSet.java:148) at
> org.apache.calcite.plan.volcano.VolcanoPlanner.addRelToSet(VolcanoPlanner.java:1837)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl(VolcanoPlanner.java:1783)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.register(VolcanoPlanner.java:850)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:872)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:1958)
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.transformTo(VolcanoRuleCall.java:126)
> at
> org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:234)
> at
> org.apache.calcite.rel.convert.ConverterRule.onMatch(ConverterRule.java:141)
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:205)
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:637)
> at calcite.CalciteTest.sqlToRelNode(CalciteTest.java:103) at
> calcite.CalciteTest.main(CalciteTest.java:53)
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Resolved] (CALCITE-3267) Remove method SqlDataTypeSpec#deriveType(RelDataTypefactory)
[ https://issues.apache.org/jira/browse/CALCITE-3267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chan resolved CALCITE-3267. - Resolution: Fixed Fixed in [6b600e4|https://github.com/apache/calcite/commit/6b600e4bd13d130a4ea48ab9b8451d81ed095da0] ! > Remove method SqlDataTypeSpec#deriveType(RelDataTypefactory) > > > Key: CALCITE-3267 > URL: https://issues.apache.org/jira/browse/CALCITE-3267 > Project: Calcite > Issue Type: Sub-task > Components: core >Affects Versions: 1.20.0 >Reporter: Danny Chan >Assignee: Danny Chan >Priority: Major > Labels: pull-request-available > Fix For: 1.21.0 > > Time Spent: 20m > Remaining Estimate: 0h > > As discussed in CALCITE-3250, we should always derive data type from > SqlValidator instead of RelDataTypeFactory. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (CALCITE-3013) support CeilFuncBigInt TO
[ https://issues.apache.org/jira/browse/CALCITE-3013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911086#comment-16911086 ] xzh_dz commented on CALCITE-3013: - Maybe the ceil function should support Long Type. > support CeilFuncBigInt TO > -- > > Key: CALCITE-3013 > URL: https://issues.apache.org/jira/browse/CALCITE-3013 > Project: Calcite > Issue Type: Test >Reporter: xzh_dz >Priority: Trivial > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (CALCITE-1917) Support column reference in "FOR SYSTEM_TIME AS OF"
[
https://issues.apache.org/jira/browse/CALCITE-1917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911072#comment-16911072
]
ShuMing Li commented on CALCITE-1917:
-
Has this issue already resolved? I find Calcite already support such tests in
`SqlToRelConverterTest.java`:
{code:java}
@Test public void testJoinTemporalTableOnColumnReference() {
final String sql = "select stream *\n"
+ "from orders\n"
+ "join products_temporal for system_time as of orders.rowtime\n"
+ "on orders.productid = products_temporal.productid";
sql(sql).ok();
}
{code}
> Support column reference in "FOR SYSTEM_TIME AS OF"
> ---
>
> Key: CALCITE-1917
> URL: https://issues.apache.org/jira/browse/CALCITE-1917
> Project: Calcite
> Issue Type: New Feature
>Reporter: Jark Wu
>Priority: Major
>
> As discussed in mailing list[1], the standard says QSTPS can’t contain a
> column reference. So when joining the Orders to the Products table for the
> price as of the time the order was placed is impossible using "FOR
> SYSTEM_TIME AS OF". But can be expressed using a subquery, such as:
> {code}
> SELECT *
> FROM Orders AS o
> JOIN LATERAL (SELECT * FROM ProductPrices WHERE sysStart <= O.orderTime
> AND sysEnd > O.orderTime) AS P
> ON o.productId = p.productId
> {code}
> But subquery is too complex for users. We know that the standard says it
> can’t contain a column reference. We initialize this discuss as we would like
> to "extend" the standard to simplify such query:
> {code}
> SELECT *
> FROM Orders AS o
> JOIN LATERAL ProductPrices FOR SYSTEM_TIME AS OF O.orderTime AS P
> ON o.productId = p.productId
> {code}
> [1]
> https://lists.apache.org/thread.html/f877f356a8365bf74ea7d8e4a171224104d653cf73861afb2901a58f@%3Cdev.calcite.apache.org%3E
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (CALCITE-2772) Support varargs for user-defined functions (UDFs)
[
https://issues.apache.org/jira/browse/CALCITE-2772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911031#comment-16911031
]
Stamatis Zampetakis commented on CALCITE-2772:
--
Hi [~gr4ve], the issue is not solved; thanks for submitting the patch!
Usually, we work with pull requests on GitHub so if you can transform your
patch into a PR it will be much more convenient.
> Support varargs for user-defined functions (UDFs)
> -
>
> Key: CALCITE-2772
> URL: https://issues.apache.org/jira/browse/CALCITE-2772
> Project: Calcite
> Issue Type: Improvement
> Components: core
>Reporter: pengzhiwei
>Assignee: pengzhiwei
>Priority: Major
> Attachments: support_varargs_udf.patch
>
>
> Support varargs for user-defined functions as the case followed:
> {code:java}
> public class ConcatWs {
> public String eval(String sep, String... strs) {...}
> }{code}
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
