Re: [PR] [CALCITE-7254] Add rule for sharing trivially equivalent RelNodes within Combine [calcite]
tjbanghart merged PR #4658: URL: https://github.com/apache/calcite/pull/4658 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [CALCITE-7254] Add rule for sharing trivially equivalent RelNodes within Combine [calcite]
sonarqubecloud[bot] commented on PR #4658: URL: https://github.com/apache/calcite/pull/4658#issuecomment-3629098014 ## [](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4658) **Quality Gate passed** Issues  [3 New issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4658&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4658&issueStatuses=ACCEPTED) Measures  [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_calcite&pullRequest=4658&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  [92.6% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4658&metric=new_coverage&view=list)  [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4658&metric=new_duplicated_lines_density&view=list) [See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4658) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [CALCITE-7254] Add rule for sharing trivially equivalent RelNodes within Combine [calcite]
tjbanghart commented on code in PR #4658:
URL: https://github.com/apache/calcite/pull/4658#discussion_r2599918942
##
core/src/main/java/org/apache/calcite/runtime/SqlFunctions.java:
##
@@ -6884,6 +6884,43 @@ public static Map map(Object... args) {
return map;
}
+ /** Combines multiple query result lists into rows for the Combine operator.
+ *
+ * Each input list contains maps representing rows from a query.
+ * The output is a list of Object arrays, where each array is a row
+ * with one element per query. The number of output rows equals the
+ * maximum size across all input lists. Shorter lists are padded with nulls.
+ *
+ * @param queryLists array of lists, one per query
+ * @return list of Object arrays representing combined rows
+ */
+ public static List combineQueryResults(List[] queryLists) {
+// Find the maximum row count across all queries
+int maxRows = 0;
+for (List list : queryLists) {
+ if (list.size() > maxRows) {
+maxRows = list.size();
+ }
+}
+
+// Build the result rows
+List result = new ArrayList<>(maxRows);
+for (int rowIdx = 0; rowIdx < maxRows; rowIdx++) {
+ Object[] row = new Object[queryLists.length];
+ for (int queryIdx = 0; queryIdx < queryLists.length; queryIdx++) {
+List queryList = queryLists[queryIdx];
Review Comment:
Awesome, thank you - that was it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] [CALCITE-7254] Add rule for sharing trivially equivalent RelNodes within Combine [calcite]
tjbanghart commented on PR #4658: URL: https://github.com/apache/calcite/pull/4658#issuecomment-3628720023 Thank you so much for the reviews @xiedeyantu, much appreciated! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [CALCITE-7254] Add rule for sharing trivially equivalent RelNodes within Combine [calcite]
sonarqubecloud[bot] commented on PR #4658: URL: https://github.com/apache/calcite/pull/4658#issuecomment-3625192124 ## [](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4658) **Quality Gate passed** Issues  [3 New issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4658&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4658&issueStatuses=ACCEPTED) Measures  [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_calcite&pullRequest=4658&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  [92.6% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4658&metric=new_coverage&view=list)  [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4658&metric=new_duplicated_lines_density&view=list) [See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4658) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [CALCITE-7254] Add rule for sharing trivially equivalent RelNodes within Combine [calcite]
xuzifu666 commented on code in PR #4658:
URL: https://github.com/apache/calcite/pull/4658#discussion_r2596862394
##
core/src/main/java/org/apache/calcite/runtime/SqlFunctions.java:
##
@@ -6884,6 +6884,43 @@ public static Map map(Object... args) {
return map;
}
+ /** Combines multiple query result lists into rows for the Combine operator.
+ *
+ * Each input list contains maps representing rows from a query.
+ * The output is a list of Object arrays, where each array is a row
+ * with one element per query. The number of output rows equals the
+ * maximum size across all input lists. Shorter lists are padded with nulls.
+ *
+ * @param queryLists array of lists, one per query
+ * @return list of Object arrays representing combined rows
+ */
+ public static List combineQueryResults(List[] queryLists) {
+// Find the maximum row count across all queries
+int maxRows = 0;
+for (List list : queryLists) {
+ if (list.size() > maxRows) {
+maxRows = list.size();
+ }
+}
+
+// Build the result rows
+List result = new ArrayList<>(maxRows);
+for (int rowIdx = 0; rowIdx < maxRows; rowIdx++) {
+ Object[] row = new Object[queryLists.length];
+ for (int queryIdx = 0; queryIdx < queryLists.length; queryIdx++) {
+List queryList = queryLists[queryIdx];
Review Comment:
The ```@Nullable``` annotation is required here; otherwise CI will fail.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] [CALCITE-7254] Add rule for sharing trivially equivalent RelNodes within Combine [calcite]
xiedeyantu commented on PR #4658: URL: https://github.com/apache/calcite/pull/4658#issuecomment-3610433971 There are still some minor issues in the CI that need to be fixed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [CALCITE-7254] Add rule for sharing trivially equivalent RelNodes within Combine [calcite]
xiedeyantu commented on code in PR #4658:
URL: https://github.com/apache/calcite/pull/4658#discussion_r2587675293
##
core/src/test/java/org/apache/calcite/test/enumerable/EnumerableCombineTest.java:
##
@@ -149,9 +154,62 @@ class EnumerableCombineTest {
return builder.combine(2).build();
})
.returnsUnordered(
-"QUERY=[{name=Sales}, {name=Marketing}, {name=HR}]",
-"QUERY=[{empid=100, name=Bill, deptno=10}, {empid=150,
name=Sebastian, deptno=10}, "
-+ "{empid=110, name=Theodore, deptno=10}]");
+"QUERY_0={name=Sales}; QUERY_1={empid=100, name=Bill, deptno=10}",
+"QUERY_0={name=Marketing}; QUERY_1={empid=150, name=Sebastian,
deptno=10}",
+"QUERY_0={name=HR}; QUERY_1={empid=110, name=Theodore,
deptno=10}");
+ }
+
+ /**
+ * Test that two queries selecting from the same table with different sort
+ * orders do not conflict with each other. Each query maintains its own
+ * independent results without one sort order overriding the other.
+ *
+ * Query 1 sorts employees by empid ascending, Query 2 sorts by name
descending.
+ * The key verification is that both queries execute independently and
produce
+ * their own ordered results.
+ */
+ @Test void testCombineSameTableDifferentSortOrders() {
+tester(new HrSchema())
+.withRel(
+builder -> {
+ // Query 1: SELECT empid, name FROM emps ORDER BY empid ASC
+ builder.scan("s", "emps")
+ .project(
+ builder.field("empid"),
+ builder.field("name"))
+ .sort(builder.field("empid"));
+
+ // Query 2: SELECT empid, name FROM emps ORDER BY name DESC
+ builder.scan("s", "emps")
+ .project(
+ builder.field("empid"),
+ builder.field("name"))
+ .sort(builder.desc(builder.field("name")));
+
+ // Combine both queries
+ return builder.combine(2).build();
+})
+.returns(resultSet -> {
+ try {
+int rowCount = 0;
+while (resultSet.next()) {
+ rowCount++;
+ // Verify both columns have values (no cross-contamination)
+ Object query0 = resultSet.getObject("QUERY_0");
+ Object query1 = resultSet.getObject("QUERY_1");
+ if (query0 == null || query1 == null) {
Review Comment:
Would it be better to use AssertThat for positive validation? I don't
understand why the throw new AssertionError approach is used here.
##
core/src/main/java/org/apache/calcite/runtime/SqlFunctions.java:
##
@@ -6884,6 +6884,43 @@ public static Map map(Object... args) {
return map;
}
+ /** Combines multiple query result lists into rows for the Combine operator.
+ *
+ * Each input list contains maps representing rows from a query.
+ * The output is a list of Object arrays, where each array is a row
+ * with one element per query. The number of output rows equals the
+ * maximum size across all input lists. Shorter lists are padded with nulls.
+ *
+ * @param queryLists array of lists, one per query
+ * @return list of Object arrays representing combined rows
+ */
+ public static List combineQueryResults(List[] queryLists) {
Review Comment:
Can you add a small test for this method?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] [CALCITE-7254] Add rule for sharing trivially equivalent RelNodes within Combine [calcite]
sonarqubecloud[bot] commented on PR #4658: URL: https://github.com/apache/calcite/pull/4658#issuecomment-3608569744 ## [](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4658) **Quality Gate passed** Issues  [3 New issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4658&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4658&issueStatuses=ACCEPTED) Measures  [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_calcite&pullRequest=4658&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  [92.6% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4658&metric=new_coverage&view=list)  [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4658&metric=new_duplicated_lines_density&view=list) [See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4658) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [CALCITE-7254] Add rule for sharing trivially equivalent RelNodes within Combine [calcite]
tjbanghart commented on code in PR #4658:
URL: https://github.com/apache/calcite/pull/4658#discussion_r2586351999
##
core/src/test/java/org/apache/calcite/test/enumerable/EnumerableCombineTest.java:
##
@@ -0,0 +1,162 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.test.enumerable;
+
+import org.apache.calcite.adapter.java.ReflectiveSchema;
+import org.apache.calcite.config.CalciteConnectionProperty;
+import org.apache.calcite.config.Lex;
+import org.apache.calcite.test.CalciteAssert;
+import org.apache.calcite.test.schemata.hr.HrSchema;
+
+import org.junit.jupiter.api.Test;
+
+/**
+ * Unit tests for {@link
org.apache.calcite.adapter.enumerable.EnumerableCombine}.
+ */
+class EnumerableCombineTest {
+
+ /**
+ * Test that executes two simple queries combined.
+ * Query 1: Select employee names from department 10
+ * Query 2: Select department names
+ *
+ * The Combine operator returns results in a structured format where each
+ * query's results are grouped: {@code QUERY_0={...}; QUERY_1={...}}
+ */
+ @Test void testCombineTwoQueries() {
+tester(new HrSchema())
+.withRel(
+builder -> {
+ // Query 1: SELECT name FROM emps WHERE deptno = 10
+ builder.scan("s", "emps")
+ .filter(
+ builder.equals(
+ builder.field("deptno"),
+ builder.literal(10)))
+ .project(builder.field("name"));
+
+ // Query 2: SELECT name FROM depts
+ builder.scan("s", "depts")
+ .project(builder.field("name"));
+
+ // Combine both queries
+ return builder.combine(2).build();
+})
+.returnsOrdered(
+"QUERY=[{name=Bill}, {name=Sebastian}, {name=Theodore}]",
+"QUERY=[{name=Sales}, {name=Marketing}, {name=HR}]");
+ }
+
+ /**
+ * Test that executes two queries with aggregates.
+ * Query 1: Count of employees
+ * Query 2: Average salary of employees
+ */
+ @Test void testCombineWithAggregates() {
+tester(new HrSchema())
+.withRel(
+builder -> {
+ // Query 1: SELECT deptno, COUNT(*) AS emp_count FROM emps GROUP
BY deptno
+ builder.scan("s", "emps")
+ .aggregate(builder.groupKey("deptno"),
+ builder.count().as("emp_count"));
+
+ // Query 2: SELECT AVG(salary) AS avg_salary FROM emps
+ builder.scan("s", "emps")
+ .aggregate(builder.groupKey(),
+ builder.avg(builder.field("salary")).as("avg_salary"));
+
+ // Combine both queries
+ return builder.combine(2).build();
+})
+.returnsUnordered(
+"QUERY=[{deptno=20, emp_count=1}, {deptno=10, emp_count=3}]",
+"QUERY=[{avg_salary=9125.0}]");
+ }
+
+ /**
+ * Test that executes two queries with multiple columns each.
+ * Query 1: Select empid and name from employees in department 10
+ * Query 2: Select deptno and name from departments
+ */
+ @Test void testCombineMultipleColumns() {
+tester(new HrSchema())
+.withRel(
+builder -> {
+ // Query 1: SELECT empid, name FROM emps WHERE deptno = 10
+ builder.scan("s", "emps")
+ .filter(
+ builder.equals(
+ builder.field("deptno"),
+ builder.literal(10)))
+ .project(
+ builder.field("empid"),
+ builder.field("name"));
+
+ // Query 2: SELECT deptno, name FROM depts
+ builder.scan("s", "depts")
+ .project(
+ builder.field("deptno"),
+ builder.field("name"));
+
+ // Combine both queries
+ return builder.combine(2).build();
+})
+.returnsUnordered(
+"QUERY=[{empid=100, name=Bill}, {empid=150, name=Sebastian},
{empid=110, name=Theodore}]",
+"QUERY=[{deptno=10, name=Sales}, {deptno=30, name=Marketing},
{deptn
Re: [PR] [CALCITE-7254] Add rule for sharing trivially equivalent RelNodes within Combine [calcite]
tjbanghart commented on code in PR #4658:
URL: https://github.com/apache/calcite/pull/4658#discussion_r2586351008
##
core/src/test/java/org/apache/calcite/test/enumerable/EnumerableCombineTest.java:
##
@@ -0,0 +1,126 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.test.enumerable;
+
+import org.apache.calcite.adapter.java.ReflectiveSchema;
+import org.apache.calcite.config.CalciteConnectionProperty;
+import org.apache.calcite.config.Lex;
+import org.apache.calcite.test.CalciteAssert;
+import org.apache.calcite.test.schemata.hr.HrSchema;
+
+import org.junit.jupiter.api.Test;
+
+/**
+ * Unit tests for {@link
org.apache.calcite.adapter.enumerable.EnumerableCombine}.
+ */
+class EnumerableCombineTest {
+
+ /**
+ * Test that executes two simple queries combined.
+ * Query 1: Select employee names from department 10
+ * Query 2: Select department names
+ *
+ * The Combine operator returns results in a structured format where each
+ * query's results are grouped: {@code QUERY_0={...}; QUERY_1={...}}
+ */
+ @Test void testCombineTwoQueries() {
+tester(new HrSchema())
+.withRel(
+builder -> {
+ // Query 1: SELECT name FROM emps WHERE deptno = 10
+ builder.scan("s", "emps")
+ .filter(
+ builder.equals(
+ builder.field("deptno"),
+ builder.literal(10)))
+ .project(builder.field("name"));
+
+ // Query 2: SELECT name FROM depts
+ builder.scan("s", "depts")
+ .project(builder.field("name"));
+
+ // Combine both queries
+ return builder.combine(2).build();
+})
+.returnsUnordered(
Review Comment:
To later accommodate named queries I've changed the format once more. From
the comment in `EnumerableCombine`:
The output format is a wide table where each column corresponds to a query
(named QUERY_0, QUERY_1, etc.) and each row contains a struct with that query's
column values for that row index. The number of output rows equals the maximum
row count across all input queries. Queries with fewer rows have null values
for the additional rows.
Example output for two queries:
```
QUERY_0 | QUERY_1
|
{empno=100, name=Bill} | {deptno=10, name=Sales}
{empno=110, name=Eric} | {deptno=20, name=HR}
{empno=120, name=Ted}| null
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] [CALCITE-7254] Add rule for sharing trivially equivalent RelNodes within Combine [calcite]
tjbanghart commented on code in PR #4658:
URL: https://github.com/apache/calcite/pull/4658#discussion_r2586346178
##
core/src/main/java/org/apache/calcite/rel/rules/CombineSimpleEquivalenceRule.java:
##
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelDigest;
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.plan.SpoolRelOptTable;
+import org.apache.calcite.rel.RelCommonExpressionBasicSuggester;
+import org.apache.calcite.rel.RelHomogeneousShuttle;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Combine;
+import org.apache.calcite.rel.core.RelFactories;
+import org.apache.calcite.rel.core.Spool;
+import org.apache.calcite.rel.logical.LogicalTableScan;
+import org.apache.calcite.rel.logical.LogicalTableSpool;
+import org.apache.calcite.rel.metadata.RelMetadataQuery;
+
+import com.google.common.collect.ImmutableList;
+
+import org.immutables.value.Value;
+
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Rule that optimizes a {@link Combine} operator by detecting shared
sub-expressions
+ * across its inputs and introducing {@link Spool}s to avoid redundant
computation.
+ *
+ * This rule identifies structurally equivalent sub-plans within a
Combine's inputs
+ * and replaces them with a spool pattern: the first occurrence becomes a
producer
+ * (TableSpool that materializes the result), and subsequent occurrences become
+ * consumers (TableScan reading from the spooled data).
+ *
+ * Example
+ *
+ * Consider two queries combined that share a common filtered table scan:
+ *
+ * {@code
+ * -- Query 1: Count high earners
+ * SELECT COUNT(*) FROM EMP WHERE SAL > 2000
+ * -- Query 2: Average salary of high earners
+ * SELECT AVG(SAL) FROM EMP WHERE SAL > 2000
+ * }
+ *
+ * Before this rule applies, the plan looks like:
+ *
+ * {@code
+ * Combine
+ * LogicalAggregate(group=[{}], CNT=[COUNT()])
+ * LogicalFilter(condition=[>(SAL, 2000)])
+ * LogicalTableScan(table=[EMP])
+ * LogicalAggregate(group=[{}], AVG_SAL=[AVG(SAL)])
+ * LogicalFilter(condition=[>(SAL, 2000)])
+ * LogicalTableScan(table=[EMP])
+ * }
+ *
+ * After this rule identifies the shared {@code Filter(SAL > 2000) ->
TableScan(EMP)}
+ * sub-expression, the plan becomes:
+ *
+ * {@code
+ * Combine
+ * LogicalAggregate(group=[{}], CNT=[COUNT()])
+ * LogicalTableSpool(table=[spool_0])-- Producer: materializes
filtered rows
+ * LogicalFilter(condition=[>(SAL, 2000)])
+ * LogicalTableScan(table=[EMP])
+ * LogicalAggregate(group=[{}], AVG_SAL=[AVG(SAL)])
+ * LogicalTableScan(table=[spool_0]) -- Consumer: reads from spool
+ * }
+ *
+ * @see Combine
+ * @see Spool
+ * @see RelCommonExpressionBasicSuggester
+ */
[email protected]
+public class CombineSimpleEquivalenceRule extends
RelRule {
+
+ /** Creates a CombineSharedComponentsRule. */
+ protected CombineSimpleEquivalenceRule(Config config) {
+super(config);
+ }
+
+ @Override public void onMatch(RelOptRuleCall call) {
+RelNode combine = RelOptUtil.stripAll(call.rel(0));
+
+// Use the suggester to find shared components
+RelCommonExpressionBasicSuggester suggester = new
RelCommonExpressionBasicSuggester();
+Collection sharedComponents = suggester.suggest(combine, null);
+
+// Filter out any components that are already spools or scans from spool
tables
+// to avoid creating spools of spools
+sharedComponents = sharedComponents.stream()
+.filter(node -> {
+ if (node instanceof Spool) {
+return false;
+ }
+ // Skip if it's a TableScan reading from a spool table
+ if (node instanceof LogicalTableScan) {
+LogicalTableScan scan = (LogicalTableScan) node;
+// Check if the underlying table is a SpoolRelOptTable
+return !(scan.getTable() instanceof SpoolRelOptTable);
+ }
+ return true;
+})
+.collect(j
Re: [PR] [CALCITE-7254] Add rule for sharing trivially equivalent RelNodes within Combine [calcite]
tjbanghart commented on code in PR #4658:
URL: https://github.com/apache/calcite/pull/4658#discussion_r2586040693
##
core/src/test/java/org/apache/calcite/test/enumerable/EnumerableCombineTest.java:
##
@@ -0,0 +1,162 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.test.enumerable;
+
+import org.apache.calcite.adapter.java.ReflectiveSchema;
+import org.apache.calcite.config.CalciteConnectionProperty;
+import org.apache.calcite.config.Lex;
+import org.apache.calcite.test.CalciteAssert;
+import org.apache.calcite.test.schemata.hr.HrSchema;
+
+import org.junit.jupiter.api.Test;
+
+/**
+ * Unit tests for {@link
org.apache.calcite.adapter.enumerable.EnumerableCombine}.
+ */
+class EnumerableCombineTest {
+
+ /**
+ * Test that executes two simple queries combined.
+ * Query 1: Select employee names from department 10
+ * Query 2: Select department names
+ *
+ * The Combine operator returns results in a structured format where each
+ * query's results are grouped: {@code QUERY_0={...}; QUERY_1={...}}
+ */
+ @Test void testCombineTwoQueries() {
+tester(new HrSchema())
+.withRel(
+builder -> {
+ // Query 1: SELECT name FROM emps WHERE deptno = 10
+ builder.scan("s", "emps")
+ .filter(
+ builder.equals(
+ builder.field("deptno"),
+ builder.literal(10)))
+ .project(builder.field("name"));
+
+ // Query 2: SELECT name FROM depts
+ builder.scan("s", "depts")
+ .project(builder.field("name"));
+
+ // Combine both queries
+ return builder.combine(2).build();
+})
+.returnsOrdered(
+"QUERY=[{name=Bill}, {name=Sebastian}, {name=Theodore}]",
+"QUERY=[{name=Sales}, {name=Marketing}, {name=HR}]");
+ }
+
+ /**
+ * Test that executes two queries with aggregates.
+ * Query 1: Count of employees
+ * Query 2: Average salary of employees
+ */
+ @Test void testCombineWithAggregates() {
+tester(new HrSchema())
+.withRel(
+builder -> {
+ // Query 1: SELECT deptno, COUNT(*) AS emp_count FROM emps GROUP
BY deptno
+ builder.scan("s", "emps")
+ .aggregate(builder.groupKey("deptno"),
+ builder.count().as("emp_count"));
+
+ // Query 2: SELECT AVG(salary) AS avg_salary FROM emps
+ builder.scan("s", "emps")
+ .aggregate(builder.groupKey(),
+ builder.avg(builder.field("salary")).as("avg_salary"));
+
+ // Combine both queries
+ return builder.combine(2).build();
+})
+.returnsUnordered(
+"QUERY=[{deptno=20, emp_count=1}, {deptno=10, emp_count=3}]",
+"QUERY=[{avg_salary=9125.0}]");
+ }
+
+ /**
+ * Test that executes two queries with multiple columns each.
+ * Query 1: Select empid and name from employees in department 10
+ * Query 2: Select deptno and name from departments
+ */
+ @Test void testCombineMultipleColumns() {
+tester(new HrSchema())
+.withRel(
+builder -> {
+ // Query 1: SELECT empid, name FROM emps WHERE deptno = 10
+ builder.scan("s", "emps")
+ .filter(
+ builder.equals(
+ builder.field("deptno"),
+ builder.literal(10)))
+ .project(
+ builder.field("empid"),
+ builder.field("name"));
+
+ // Query 2: SELECT deptno, name FROM depts
+ builder.scan("s", "depts")
+ .project(
+ builder.field("deptno"),
+ builder.field("name"));
+
+ // Combine both queries
+ return builder.combine(2).build();
+})
+.returnsUnordered(
+"QUERY=[{empid=100, name=Bill}, {empid=150, name=Sebastian},
{empid=110, name=Theodore}]",
+"QUERY=[{deptno=10, name=Sales}, {deptno=30, name=Marketing},
{deptn
Re: [PR] [CALCITE-7254] Add rule for sharing trivially equivalent RelNodes within Combine [calcite]
xiedeyantu commented on code in PR #4658:
URL: https://github.com/apache/calcite/pull/4658#discussion_r2583861926
##
core/src/test/java/org/apache/calcite/test/enumerable/EnumerableCombineTest.java:
##
@@ -0,0 +1,126 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.test.enumerable;
+
+import org.apache.calcite.adapter.java.ReflectiveSchema;
+import org.apache.calcite.config.CalciteConnectionProperty;
+import org.apache.calcite.config.Lex;
+import org.apache.calcite.test.CalciteAssert;
+import org.apache.calcite.test.schemata.hr.HrSchema;
+
+import org.junit.jupiter.api.Test;
+
+/**
+ * Unit tests for {@link
org.apache.calcite.adapter.enumerable.EnumerableCombine}.
+ */
+class EnumerableCombineTest {
+
+ /**
+ * Test that executes two simple queries combined.
+ * Query 1: Select employee names from department 10
+ * Query 2: Select department names
+ *
+ * The Combine operator returns results in a structured format where each
+ * query's results are grouped: {@code QUERY_0={...}; QUERY_1={...}}
+ */
+ @Test void testCombineTwoQueries() {
+tester(new HrSchema())
+.withRel(
+builder -> {
+ // Query 1: SELECT name FROM emps WHERE deptno = 10
+ builder.scan("s", "emps")
+ .filter(
+ builder.equals(
+ builder.field("deptno"),
+ builder.literal(10)))
+ .project(builder.field("name"));
+
+ // Query 2: SELECT name FROM depts
+ builder.scan("s", "depts")
+ .project(builder.field("name"));
+
+ // Combine both queries
+ return builder.combine(2).build();
+})
+.returnsUnordered(
Review Comment:
Thank you for the very detailed explanation. I think the current format of
the result presentation looks quite good.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] [CALCITE-7254] Add rule for sharing trivially equivalent RelNodes within Combine [calcite]
xiedeyantu commented on code in PR #4658:
URL: https://github.com/apache/calcite/pull/4658#discussion_r2583849312
##
core/src/main/java/org/apache/calcite/rel/rules/CombineSimpleEquivalenceRule.java:
##
@@ -0,0 +1,226 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelDigest;
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.plan.SpoolRelOptTable;
+import org.apache.calcite.rel.RelCommonExpressionBasicSuggester;
+import org.apache.calcite.rel.RelHomogeneousShuttle;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Combine;
+import org.apache.calcite.rel.core.RelFactories;
+import org.apache.calcite.rel.core.Spool;
+import org.apache.calcite.rel.logical.LogicalTableScan;
+import org.apache.calcite.rel.logical.LogicalTableSpool;
+import org.apache.calcite.rel.metadata.RelMetadataQuery;
+
+import com.google.common.collect.ImmutableList;
+
+import org.immutables.value.Value;
+
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Rule that optimizes a {@link Combine} operator by detecting shared
sub-expressions
+ * across its inputs and introducing {@link Spool}s to avoid redundant
computation.
+ *
+ * This rule identifies structurally equivalent sub-plans within a
Combine's inputs
+ * and replaces them with a spool pattern: the first occurrence becomes a
producer
+ * (TableSpool that materializes the result), and subsequent occurrences become
+ * consumers (TableScan reading from the spooled data).
+ *
+ * Example
+ *
+ * Consider two queries combined that share a common filtered table scan:
+ *
+ * {@code
+ * -- Query 1: Count high earners
+ * SELECT COUNT(*) FROM EMP WHERE SAL > 2000
+ * -- Query 2: Average salary of high earners
+ * SELECT AVG(SAL) FROM EMP WHERE SAL > 2000
+ * }
+ *
+ * Before this rule applies, the plan looks like:
+ *
+ * {@code
+ * Combine
+ * LogicalAggregate(group=[{}], CNT=[COUNT()])
+ * LogicalFilter(condition=[>(SAL, 2000)])
+ * LogicalTableScan(table=[EMP])
+ * LogicalAggregate(group=[{}], AVG_SAL=[AVG(SAL)])
+ * LogicalFilter(condition=[>(SAL, 2000)])
+ * LogicalTableScan(table=[EMP])
+ * }
+ *
+ * After this rule identifies the shared {@code Filter(SAL > 2000) ->
TableScan(EMP)}
+ * sub-expression, the plan becomes:
+ *
+ * {@code
+ * Combine
+ * LogicalAggregate(group=[{}], CNT=[COUNT()])
+ * LogicalTableSpool(table=[spool_0])-- Producer: materializes
filtered rows
+ * LogicalFilter(condition=[>(SAL, 2000)])
+ * LogicalTableScan(table=[EMP])
+ * LogicalAggregate(group=[{}], AVG_SAL=[AVG(SAL)])
+ * LogicalTableScan(table=[spool_0]) -- Consumer: reads from spool
+ * }
+ *
+ * @see Combine
+ * @see Spool
+ * @see RelCommonExpressionBasicSuggester
+ */
[email protected]
+public class CombineSimpleEquivalenceRule extends
RelRule {
+
+ /** Creates a CombineSharedComponentsRule. */
+ protected CombineSimpleEquivalenceRule(Config config) {
+super(config);
+ }
+
+ @Override public void onMatch(RelOptRuleCall call) {
+RelNode combine = RelOptUtil.stripAll(call.rel(0));
+
+// Use the suggester to find shared components
+RelCommonExpressionBasicSuggester suggester = new
RelCommonExpressionBasicSuggester();
+Collection sharedComponents = suggester.suggest(combine, null);
+
+// Filter out any components that are already spools or scans from spool
tables
+// to avoid creating spools of spools
+sharedComponents = sharedComponents.stream()
+.filter(node -> {
+ if (node instanceof Spool) {
+return false;
+ }
+ // Skip if it's a TableScan reading from a spool table
+ if (node instanceof LogicalTableScan) {
+LogicalTableScan scan = (LogicalTableScan) node;
+// Check if the underlying table is a SpoolRelOptTable
+return !(scan.getTable() instanceof SpoolRelOptTable);
+ }
+ return true;
+})
+.collect(j
Re: [PR] [CALCITE-7254] Add rule for sharing trivially equivalent RelNodes within Combine [calcite]
sonarqubecloud[bot] commented on PR #4658: URL: https://github.com/apache/calcite/pull/4658#issuecomment-3605313257 ## [](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4658) **Quality Gate passed** Issues  [3 New issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4658&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4658&issueStatuses=ACCEPTED) Measures  [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_calcite&pullRequest=4658&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  [91.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4658&metric=new_coverage&view=list)  [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4658&metric=new_duplicated_lines_density&view=list) [See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4658) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [CALCITE-7254] Add rule for sharing trivially equivalent RelNodes within Combine [calcite]
sonarqubecloud[bot] commented on PR #4658: URL: https://github.com/apache/calcite/pull/4658#issuecomment-3605310147 ## [](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4658) **Quality Gate passed** Issues  [3 New issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4658&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4658&issueStatuses=ACCEPTED) Measures  [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_calcite&pullRequest=4658&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  [91.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4658&metric=new_coverage&view=list)  [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4658&metric=new_duplicated_lines_density&view=list) [See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4658) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [CALCITE-7254] Add rule for sharing trivially equivalent RelNodes within Combine [calcite]
tjbanghart commented on code in PR #4658:
URL: https://github.com/apache/calcite/pull/4658#discussion_r2583790673
##
core/src/test/java/org/apache/calcite/test/enumerable/EnumerableCombineTest.java:
##
@@ -0,0 +1,126 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.test.enumerable;
+
+import org.apache.calcite.adapter.java.ReflectiveSchema;
+import org.apache.calcite.config.CalciteConnectionProperty;
+import org.apache.calcite.config.Lex;
+import org.apache.calcite.test.CalciteAssert;
+import org.apache.calcite.test.schemata.hr.HrSchema;
+
+import org.junit.jupiter.api.Test;
+
+/**
+ * Unit tests for {@link
org.apache.calcite.adapter.enumerable.EnumerableCombine}.
+ */
+class EnumerableCombineTest {
+
+ /**
+ * Test that executes two simple queries combined.
+ * Query 1: Select employee names from department 10
+ * Query 2: Select department names
+ *
+ * The Combine operator returns results in a structured format where each
+ * query's results are grouped: {@code QUERY_0={...}; QUERY_1={...}}
+ */
+ @Test void testCombineTwoQueries() {
+tester(new HrSchema())
+.withRel(
+builder -> {
+ // Query 1: SELECT name FROM emps WHERE deptno = 10
+ builder.scan("s", "emps")
+ .filter(
+ builder.equals(
+ builder.field("deptno"),
+ builder.literal(10)))
+ .project(builder.field("name"));
+
+ // Query 2: SELECT name FROM depts
+ builder.scan("s", "depts")
+ .project(builder.field("name"));
+
+ // Combine both queries
+ return builder.combine(2).build();
+})
+.returnsUnordered(
Review Comment:
Good questions, I've updated the PR so `Combine` now returns one row per
input query. Each row holds an array of maps, where each map represents a row
from that query with column names as keys.
We are constrained by JDBC's limitations of returning a single result set
per query but this would allow iterating over individual query results with
`resultSet.next()` and then iterating over each query's results via the array.
For example, combining:
`SELECT name FROM depts;`
and
`SELECT empid, name, deptno FROM emps WHERE deptno = 10;`
Yields:
```
-- Row 1
QUERY=[
{name=Sales},
{name=Marketing},
{name=HR}
],
-- Row 2
QUERY=[
{empid=100, name=Bill, deptno=10},
{empid=150, name=Sebastian, deptno=10} ,
{empid=110, name=Theodore, deptno=10}
]
```
Later, we should probably allow for aliases for queries so they don't have
the generic `QUERY` column label but that may be more challenging (see the
syntax for `MULTI` described in
[CALCITE-6188](https://issues.apache.org/jira/browse/CALCITE-6188)).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] [CALCITE-7254] Add rule for sharing trivially equivalent RelNodes within Combine [calcite]
xiedeyantu commented on code in PR #4658:
URL: https://github.com/apache/calcite/pull/4658#discussion_r2583053493
##
core/src/main/java/org/apache/calcite/rel/rules/CombineSimpleEquivalenceRule.java:
##
@@ -0,0 +1,227 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelDigest;
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.plan.SpoolRelOptTable;
+import org.apache.calcite.rel.RelCommonExpressionBasicSuggester;
+import org.apache.calcite.rel.RelHomogeneousShuttle;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Combine;
+import org.apache.calcite.rel.core.RelFactories;
+import org.apache.calcite.rel.core.Spool;
+import org.apache.calcite.rel.logical.LogicalTableScan;
+import org.apache.calcite.rel.logical.LogicalTableSpool;
+import org.apache.calcite.rel.metadata.RelMetadataQuery;
+
+import com.google.common.collect.ImmutableList;
+
+import org.immutables.value.Value;
+
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Rule that optimizes a {@link Combine} operator by detecting shared
sub-expressions
+ * across its inputs and introducing {@link Spool}s to avoid redundant
computation.
+ *
+ * This rule identifies structurally equivalent sub-plans within a
Combine's inputs
+ * and replaces them with a spool pattern: the first occurrence becomes a
producer
+ * (TableSpool that materializes the result), and subsequent occurrences become
+ * consumers (TableScan reading from the spooled data).
+ *
+ * Example
+ *
+ * Consider two queries combined that share a common filtered table scan:
+ *
+ * {@code
Review Comment:
Excellent documentation, thank you.
##
core/src/test/java/org/apache/calcite/test/enumerable/EnumerableCombineTest.java:
##
@@ -0,0 +1,126 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.test.enumerable;
+
+import org.apache.calcite.adapter.java.ReflectiveSchema;
+import org.apache.calcite.config.CalciteConnectionProperty;
+import org.apache.calcite.config.Lex;
+import org.apache.calcite.test.CalciteAssert;
+import org.apache.calcite.test.schemata.hr.HrSchema;
+
+import org.junit.jupiter.api.Test;
+
+/**
+ * Unit tests for {@link
org.apache.calcite.adapter.enumerable.EnumerableCombine}.
+ */
+class EnumerableCombineTest {
+
+ /**
+ * Test that executes two simple queries combined.
+ * Query 1: Select employee names from department 10
+ * Query 2: Select department names
+ *
+ * The Combine operator returns results in a structured format where each
+ * query's results are grouped: {@code QUERY_0={...}; QUERY_1={...}}
+ */
+ @Test void testCombineTwoQueries() {
+tester(new HrSchema())
+.withRel(
+builder -> {
+ // Query 1: SELECT name FROM emps WHERE deptno = 10
+ builder.scan("s", "emps")
+ .filter(
+ builder.equals(
+ builder.field("deptno"),
+ builder.literal(10)))
+ .project(builder.field("name"));
+
+ // Query 2: SELECT name FROM depts
+ builder.scan("s", "depts")
+ .project(builder.field("name"));
+
+
Re: [PR] [CALCITE-7254] Add rule for sharing trivially equivalent RelNodes within Combine [calcite]
sonarqubecloud[bot] commented on PR #4658: URL: https://github.com/apache/calcite/pull/4658#issuecomment-3603832195 ## [](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4658) **Quality Gate passed** Issues  [1 New issue](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4658&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4658&issueStatuses=ACCEPTED) Measures  [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_calcite&pullRequest=4658&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  [90.3% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4658&metric=new_coverage&view=list)  [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4658&metric=new_duplicated_lines_density&view=list) [See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4658) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [CALCITE-7254] Add rule for sharing trivially equivalent RelNodes within Combine [calcite]
tjbanghart commented on code in PR #4658:
URL: https://github.com/apache/calcite/pull/4658#discussion_r2582093403
##
core/src/main/java/org/apache/calcite/adapter/enumerable/EnumerableCombine.java:
##
@@ -0,0 +1,98 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.adapter.enumerable;
+
+import org.apache.calcite.linq4j.Ord;
+import org.apache.calcite.linq4j.tree.BlockBuilder;
+import org.apache.calcite.linq4j.tree.Expression;
+import org.apache.calcite.linq4j.tree.Expressions;
+import org.apache.calcite.linq4j.tree.Types;
+import org.apache.calcite.plan.RelOptCluster;
+import org.apache.calcite.plan.RelTraitSet;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Combine;
+import org.apache.calcite.rel.type.RelDataType;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/** Implementation of {@link org.apache.calcite.rel.core.Combine} in
+ * {@link org.apache.calcite.adapter.enumerable.EnumerableConvention
enumerable calling convention}. */
+public class EnumerableCombine extends Combine implements EnumerableRel {
+ public EnumerableCombine(RelOptCluster cluster, RelTraitSet traitSet,
+ List inputs) {
+super(cluster, traitSet, inputs);
+ }
+
+ @Override public EnumerableCombine copy(RelTraitSet traitSet, List
inputs) {
+return new EnumerableCombine(getCluster(), traitSet, inputs);
+ }
+
+ @Override public Result implement(EnumerableRelImplementor implementor,
Prefer pref) {
+final BlockBuilder builder = new BlockBuilder();
+final RelDataType rowType = getRowType();
+final List fieldExpressions = new ArrayList<>();
+
+// Implement each input and collect their results
+// Convert each Enumerable to a List since the row type is STRUCT, ...>
Review Comment:
Good idea, I will add some tests. In a branch I have quidem tests that check
execution but the formatting is quite difficult to handle and it relies on a
`MULTI` keyword which was hacked together. Probably best to use non-quidem
execution tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] [CALCITE-7254] Add rule for sharing trivially equivalent RelNodes within Combine [calcite]
xiedeyantu commented on code in PR #4658:
URL: https://github.com/apache/calcite/pull/4658#discussion_r2581129210
##
core/src/main/java/org/apache/calcite/rel/rules/CombineSimpleEquivalenceRule.java:
##
@@ -0,0 +1,185 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.rel.rules;
+
+import org.apache.calcite.plan.RelDigest;
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptUtil;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.plan.SpoolRelOptTable;
+import org.apache.calcite.rel.RelCommonExpressionBasicSuggester;
+import org.apache.calcite.rel.RelHomogeneousShuttle;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Combine;
+import org.apache.calcite.rel.core.RelFactories;
+import org.apache.calcite.rel.core.Spool;
+import org.apache.calcite.rel.logical.LogicalTableScan;
+import org.apache.calcite.rel.logical.LogicalTableSpool;
+import org.apache.calcite.rel.metadata.RelMetadataQuery;
+
+import com.google.common.collect.ImmutableList;
+
+import org.immutables.value.Value;
+
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Rule that optimizes a Combine operator by detecting shared components
Review Comment:
Can we add a plan example here? Should we also declare this rule in
CoreRules?
##
core/src/main/java/org/apache/calcite/adapter/enumerable/EnumerableCombine.java:
##
@@ -0,0 +1,98 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.calcite.adapter.enumerable;
+
+import org.apache.calcite.linq4j.Ord;
+import org.apache.calcite.linq4j.tree.BlockBuilder;
+import org.apache.calcite.linq4j.tree.Expression;
+import org.apache.calcite.linq4j.tree.Expressions;
+import org.apache.calcite.linq4j.tree.Types;
+import org.apache.calcite.plan.RelOptCluster;
+import org.apache.calcite.plan.RelTraitSet;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Combine;
+import org.apache.calcite.rel.type.RelDataType;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/** Implementation of {@link org.apache.calcite.rel.core.Combine} in
+ * {@link org.apache.calcite.adapter.enumerable.EnumerableConvention
enumerable calling convention}. */
+public class EnumerableCombine extends Combine implements EnumerableRel {
+ public EnumerableCombine(RelOptCluster cluster, RelTraitSet traitSet,
+ List inputs) {
+super(cluster, traitSet, inputs);
+ }
+
+ @Override public EnumerableCombine copy(RelTraitSet traitSet, List
inputs) {
+return new EnumerableCombine(getCluster(), traitSet, inputs);
+ }
+
+ @Override public Result implement(EnumerableRelImplementor implementor,
Prefer pref) {
+final BlockBuilder builder = new BlockBuilder();
+final RelDataType rowType = getRowType();
+final List fieldExpressions = new ArrayList<>();
+
+// Implement each input and collect their results
+// Convert each Enumerable to a List since the row type is STRUCT, ...>
Review Comment:
Is there a way to add tests to verify the actual execution logic?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact
Re: [PR] [CALCITE-7254] Add rule for sharing trivially equivalent RelNodes within Combine [calcite]
sonarqubecloud[bot] commented on PR #4658: URL: https://github.com/apache/calcite/pull/4658#issuecomment-3600200335 ## [](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4658) **Quality Gate passed** Issues  [1 New issue](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4658&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache_calcite&pullRequest=4658&issueStatuses=ACCEPTED) Measures  [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_calcite&pullRequest=4658&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  [72.7% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4658&metric=new_coverage&view=list)  [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache_calcite&pullRequest=4658&metric=new_duplicated_lines_density&view=list) [See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache_calcite&pullRequest=4658) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
