>From Ali Alsuliman <[email protected]>: Ali Alsuliman has submitted this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/19410 )
Change subject: [ASTERIXDB-3562][COMP] Align subplan variable propagation with runtime output ...................................................................... [ASTERIXDB-3562][COMP] Align subplan variable propagation with runtime output - user model changes: no - storage format changes: no - interface changes: no Details: Subplan output tuple is formed by concatenating the input tuple to the subplan and the subplan's root operator. The variable propagation policy should do the same thing. Ext-ref: MB-65248 Change-Id: I1c30a8fd7e0ab0a9c1629f749b5955b45b40b4d8 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/19410 Tested-by: Ian Maxon <[email protected]> Tested-by: Ali Alsuliman <[email protected]> Reviewed-by: Ali Alsuliman <[email protected]> Reviewed-by: Michael Blow <[email protected]> Integration-Tests: Jenkins <[email protected]> --- A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/query-ASTERIXDB-3562/query-ASTERIXDB-3562.4.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results_column/misc/query-ASTERIXDB-3562/query-ASTERIXDB-3562.4.plan M asterixdb/asterix-app/src/test/resources/runtimets/sqlpp_queries.xml A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/query-ASTERIXDB-3562/query-ASTERIXDB-3562.3.adm A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/query-ASTERIXDB-3562/query-ASTERIXDB-3562.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/query-ASTERIXDB-3562/query-ASTERIXDB-3562.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/query-ASTERIXDB-3562/query-ASTERIXDB-3562.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results_cbo/misc/query-ASTERIXDB-3562/query-ASTERIXDB-3562.4.plan M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/SubplanOperator.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/query-ASTERIXDB-3562/query-ASTERIXDB-3562.5.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/query-ASTERIXDB-3562/query-ASTERIXDB-3562.4.plan 11 files changed, 251 insertions(+), 1 deletion(-) Approvals: Michael Blow: Looks good to me, approved Ali Alsuliman: Looks good to me, but someone else must approve; Verified Ian Maxon: Verified Jenkins: Verified diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/query-ASTERIXDB-3562/query-ASTERIXDB-3562.1.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/query-ASTERIXDB-3562/query-ASTERIXDB-3562.1.ddl.sqlpp new file mode 100644 index 0000000..37472fc --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/query-ASTERIXDB-3562/query-ASTERIXDB-3562.1.ddl.sqlpp @@ -0,0 +1,29 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +/* + * Description: This test case is to verify the fix for ASTERIXDB-3562 + */ + +drop dataverse test if exists; +create dataverse test; + +use test; +create type dt1 as {id: int}; +create dataset collection1(dt1) PRIMARY KEY id; \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/query-ASTERIXDB-3562/query-ASTERIXDB-3562.2.update.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/query-ASTERIXDB-3562/query-ASTERIXDB-3562.2.update.sqlpp new file mode 100644 index 0000000..1478405 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/query-ASTERIXDB-3562/query-ASTERIXDB-3562.2.update.sqlpp @@ -0,0 +1,22 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +use test; + +upsert into collection1 { "id":1, "array1": [1,2,3]}; \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/query-ASTERIXDB-3562/query-ASTERIXDB-3562.3.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/query-ASTERIXDB-3562/query-ASTERIXDB-3562.3.query.sqlpp new file mode 100644 index 0000000..e64ea19 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/query-ASTERIXDB-3562/query-ASTERIXDB-3562.3.query.sqlpp @@ -0,0 +1,22 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +use test; + +SELECT array1[0:1] AS f_1, array1[0:2] AS f_2 FROM collection1; \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/query-ASTERIXDB-3562/query-ASTERIXDB-3562.4.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/query-ASTERIXDB-3562/query-ASTERIXDB-3562.4.query.sqlpp new file mode 100644 index 0000000..32f29d7 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/query-ASTERIXDB-3562/query-ASTERIXDB-3562.4.query.sqlpp @@ -0,0 +1,23 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +// NOTE: this plan is needed to contain 2 consecutive subplan operators since that triggers the bug. +use test; + +EXPLAIN SELECT array1[0:1] AS f_1, array1[0:2] AS f_2 FROM collection1; \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/query-ASTERIXDB-3562/query-ASTERIXDB-3562.5.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/query-ASTERIXDB-3562/query-ASTERIXDB-3562.5.ddl.sqlpp new file mode 100644 index 0000000..548e632 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/query-ASTERIXDB-3562/query-ASTERIXDB-3562.5.ddl.sqlpp @@ -0,0 +1,20 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +drop dataverse test if exists; \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/misc/query-ASTERIXDB-3562/query-ASTERIXDB-3562.3.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/misc/query-ASTERIXDB-3562/query-ASTERIXDB-3562.3.adm new file mode 100644 index 0000000..2221625 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/misc/query-ASTERIXDB-3562/query-ASTERIXDB-3562.3.adm @@ -0,0 +1 @@ +{ "f_1": [ 1 ], "f_2": [ 1, 2 ] } \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/misc/query-ASTERIXDB-3562/query-ASTERIXDB-3562.4.plan b/asterixdb/asterix-app/src/test/resources/runtimets/results/misc/query-ASTERIXDB-3562/query-ASTERIXDB-3562.4.plan new file mode 100644 index 0000000..cb0845c --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/misc/query-ASTERIXDB-3562/query-ASTERIXDB-3562.4.plan @@ -0,0 +1,34 @@ +distribute result [$$18] [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] +-- DISTRIBUTE_RESULT |PARTITIONED| + exchange [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] + -- ONE_TO_ONE_EXCHANGE |PARTITIONED| + assign [$$18] <- [{"f_1": $$15, "f_2": $$17}] project: [$$18] [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] + -- ASSIGN |PARTITIONED| + project ([$$15, $$17]) [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] + -- STREAM_PROJECT |PARTITIONED| + subplan { + assign [$$17] <- [array-slice($$20, 0, 2)] [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] + -- ASSIGN |LOCAL| + nested tuple source [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] + -- NESTED_TUPLE_SOURCE |LOCAL| + } [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] + -- SUBPLAN |PARTITIONED| + subplan { + assign [$$15] <- [array-slice($$20, 0, 1)] [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] + -- ASSIGN |LOCAL| + nested tuple source [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] + -- NESTED_TUPLE_SOURCE |LOCAL| + } [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] + -- SUBPLAN |PARTITIONED| + assign [$$20] <- [$$collection1.getField("array1")] project: [$$20] [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] + -- ASSIGN |PARTITIONED| + project ([$$collection1]) [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] + -- STREAM_PROJECT |PARTITIONED| + exchange [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] + -- ONE_TO_ONE_EXCHANGE |PARTITIONED| + data-scan []<-[$$19, $$collection1] <- test.collection1 [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] + -- DATASOURCE_SCAN |PARTITIONED| + exchange [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] + -- ONE_TO_ONE_EXCHANGE |PARTITIONED| + empty-tuple-source [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] + -- EMPTY_TUPLE_SOURCE |PARTITIONED| \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results_cbo/misc/query-ASTERIXDB-3562/query-ASTERIXDB-3562.4.plan b/asterixdb/asterix-app/src/test/resources/runtimets/results_cbo/misc/query-ASTERIXDB-3562/query-ASTERIXDB-3562.4.plan new file mode 100644 index 0000000..75967bc --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results_cbo/misc/query-ASTERIXDB-3562/query-ASTERIXDB-3562.4.plan @@ -0,0 +1,34 @@ +distribute result [$$18] [cardinality: 1.0, op-cost: 0.0, total-cost: 1.0] +-- DISTRIBUTE_RESULT |PARTITIONED| + exchange [cardinality: 1.0, op-cost: 0.0, total-cost: 1.0] + -- ONE_TO_ONE_EXCHANGE |PARTITIONED| + assign [$$18] <- [{"f_1": $$15, "f_2": $$17}] project: [$$18] [cardinality: 1.0, op-cost: 0.0, total-cost: 1.0] + -- ASSIGN |PARTITIONED| + project ([$$15, $$17]) [cardinality: 1.0, op-cost: 0.0, total-cost: 1.0] + -- STREAM_PROJECT |PARTITIONED| + subplan { + assign [$$17] <- [array-slice($$20, 0, 2)] [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] + -- ASSIGN |LOCAL| + nested tuple source [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] + -- NESTED_TUPLE_SOURCE |LOCAL| + } [cardinality: 1.0, op-cost: 0.0, total-cost: 1.0] + -- SUBPLAN |PARTITIONED| + subplan { + assign [$$15] <- [array-slice($$20, 0, 1)] [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] + -- ASSIGN |LOCAL| + nested tuple source [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] + -- NESTED_TUPLE_SOURCE |LOCAL| + } [cardinality: 1.0, op-cost: 0.0, total-cost: 1.0] + -- SUBPLAN |PARTITIONED| + assign [$$20] <- [$$collection1.getField("array1")] project: [$$20] [cardinality: 1.0, op-cost: 0.0, total-cost: 1.0] + -- ASSIGN |PARTITIONED| + project ([$$collection1]) [cardinality: 1.0, op-cost: 0.0, total-cost: 1.0] + -- STREAM_PROJECT |PARTITIONED| + exchange [cardinality: 1.0, op-cost: 0.0, total-cost: 1.0] + -- ONE_TO_ONE_EXCHANGE |PARTITIONED| + data-scan []<-[$$19, $$collection1] <- test.collection1 [cardinality: 1.0, op-cost: 1.0, total-cost: 1.0] + -- DATASOURCE_SCAN |PARTITIONED| + exchange [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] + -- ONE_TO_ONE_EXCHANGE |PARTITIONED| + empty-tuple-source [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] + -- EMPTY_TUPLE_SOURCE |PARTITIONED| \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results_column/misc/query-ASTERIXDB-3562/query-ASTERIXDB-3562.4.plan b/asterixdb/asterix-app/src/test/resources/runtimets/results_column/misc/query-ASTERIXDB-3562/query-ASTERIXDB-3562.4.plan new file mode 100644 index 0000000..b2ca6f6 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results_column/misc/query-ASTERIXDB-3562/query-ASTERIXDB-3562.4.plan @@ -0,0 +1,34 @@ +distribute result [$$18] [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] +-- DISTRIBUTE_RESULT |PARTITIONED| + exchange [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] + -- ONE_TO_ONE_EXCHANGE |PARTITIONED| + assign [$$18] <- [{"f_1": $$15, "f_2": $$17}] project: [$$18] [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] + -- ASSIGN |PARTITIONED| + project ([$$15, $$17]) [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] + -- STREAM_PROJECT |PARTITIONED| + subplan { + assign [$$17] <- [array-slice($$20, 0, 2)] [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] + -- ASSIGN |LOCAL| + nested tuple source [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] + -- NESTED_TUPLE_SOURCE |LOCAL| + } [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] + -- SUBPLAN |PARTITIONED| + subplan { + assign [$$15] <- [array-slice($$20, 0, 1)] [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] + -- ASSIGN |LOCAL| + nested tuple source [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] + -- NESTED_TUPLE_SOURCE |LOCAL| + } [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] + -- SUBPLAN |PARTITIONED| + assign [$$20] <- [$$collection1.getField("array1")] project: [$$20] [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] + -- ASSIGN |PARTITIONED| + project ([$$collection1]) [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] + -- STREAM_PROJECT |PARTITIONED| + exchange [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] + -- ONE_TO_ONE_EXCHANGE |PARTITIONED| + data-scan []<-[$$19, $$collection1] <- test.collection1 project ({array1:any}) [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] + -- DATASOURCE_SCAN |PARTITIONED| + exchange [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] + -- ONE_TO_ONE_EXCHANGE |PARTITIONED| + empty-tuple-source [cardinality: 0.0, op-cost: 0.0, total-cost: 0.0] + -- EMPTY_TUPLE_SOURCE |PARTITIONED| \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/sqlpp_queries.xml b/asterixdb/asterix-app/src/test/resources/runtimets/sqlpp_queries.xml index 6651dee..9480064 100644 --- a/asterixdb/asterix-app/src/test/resources/runtimets/sqlpp_queries.xml +++ b/asterixdb/asterix-app/src/test/resources/runtimets/sqlpp_queries.xml @@ -7568,6 +7568,11 @@ <output-dir compare="Text">query-ASTERIXDB-3410</output-dir> </compilation-unit> </test-case> + <test-case FilePath="misc"> + <compilation-unit name="query-ASTERIXDB-3562"> + <output-dir compare="Text">query-ASTERIXDB-3562</output-dir> + </compilation-unit> + </test-case> </test-group> <test-group name="multipart-dataverse"> <test-case FilePath="multipart-dataverse"> diff --git a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/SubplanOperator.java b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/SubplanOperator.java index 72150a9..0edd8f7 100644 --- a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/SubplanOperator.java +++ b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/SubplanOperator.java @@ -70,7 +70,7 @@ @Override public VariablePropagationPolicy getVariablePropagationPolicy() { - return VariablePropagationPolicy.ADDNEWVARIABLES; + return VariablePropagationPolicy.ALL; } @Override -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/19410 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: ionic Gerrit-Change-Id: I1c30a8fd7e0ab0a9c1629f749b5955b45b40b4d8 Gerrit-Change-Number: 19410 Gerrit-PatchSet: 5 Gerrit-Owner: Ali Alsuliman <[email protected]> Gerrit-Reviewer: Ali Alsuliman <[email protected]> Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Ian Maxon <[email protected]> Gerrit-Reviewer: Jenkins <[email protected]> Gerrit-Reviewer: Michael Blow <[email protected]> Gerrit-Reviewer: Murtadha Hubail <[email protected]> Gerrit-Reviewer: Peeyush Gupta <[email protected]> Gerrit-MessageType: merged
