>From Vijay Sarathy <[email protected]>: Vijay Sarathy has submitted this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17411 )
Change subject: [ASTERIXDB-3128][COMP] Hashjoin hint with build generates warning ...................................................................... [ASTERIXDB-3128][COMP] Hashjoin hint with build generates warning [ASTERIXDB-3126][COMP] CBO: hint productivity NPE if missing cardinality estimate on collection Change-Id: Ia26a912063aaf063a530acc7ff1a7a006f0ad0c9 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17411 Tested-by: Jenkins <[email protected]> Reviewed-by: Vijay Sarathy <[email protected]> Reviewed-by: Ali Alsuliman <[email protected]> Integration-Tests: Jenkins <[email protected]> --- A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/cardinality-hint-warning/cardinality-hint-warning.04.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/cardinality-hint-warning/cardinality-hint-warning.01.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/cardinality-hint-warning/cardinality-hint-warning.03.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/cardinality-hint-warning/cardinality-hint-warning.05.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/cardinality-hint-warning/cardinality-hint-warning.02.query.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/hashjoin-hint-warning/hashjoin-hint-warning.01.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/cardinality-hint-warning/cardinality-hint-warning.07.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/cardinality-hint-warning/cardinality-hint-warning.05.adm A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/cardinality-hint-warning/cardinality-hint-warning.06.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/cardinality-hint-warning/cardinality-hint-warning.09.adm M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/util/JoinUtils.java A asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/cardinality-hint-warning/cardinality-hint-warning.02.adm M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/VariableUtilities.java A asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/hashjoin-hint-warning/hashjoin-hint-warning.12.adm A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/hashjoin-hint-warning/hashjoin-hint-warning.12.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/cardinality-hint-warning/cardinality-hint-warning.04.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/cardinality-hint-warning/cardinality-hint-warning.08.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/cardinality-hint-warning/cardinality-hint-warning.06.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/cardinality-hint-warning/cardinality-hint-warning.03.adm M asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/cardinality-hint-warning/cardinality-hint-warning.07.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/cardinality-hint-warning/cardinality-hint-warning.09.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/cardinality-hint-warning/cardinality-hint-warning.08.query.sqlpp 24 files changed, 433 insertions(+), 37 deletions(-) Approvals: Ali Alsuliman: Looks good to me, approved Vijay Sarathy: Looks good to me, but someone else must approve Jenkins: Verified; Verified Anon. E. Moose #1000171: diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/cardinality-hint-warning/cardinality-hint-warning.01.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/cardinality-hint-warning/cardinality-hint-warning.01.ddl.sqlpp new file mode 100644 index 0000000..e4727dc --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/cardinality-hint-warning/cardinality-hint-warning.01.ddl.sqlpp @@ -0,0 +1,48 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +/* +* Description : Test warnings for cardinality hints +* Expected Res : Warning, ignore hint +* Date : 03/09/2023 +*/ + +DROP dataverse tpch IF EXISTS; +CREATE dataverse tpch; + +USE tpch; + + +CREATE TYPE OrderType AS { + o_orderkey : integer +}; + +CREATE TYPE CustomerType AS { + c_custkey : integer +}; + +CREATE TYPE NationType AS { + n_nationkey : integer, + n_name : string +}; + +CREATE DATASET orders(OrderType) PRIMARY KEY o_orderkey; + +CREATE DATASET customer(CustomerType) PRIMARY KEY c_custkey; + +CREATE DATASET nation(NationType) PRIMARY KEY n_nationkey; diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/cardinality-hint-warning/cardinality-hint-warning.02.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/cardinality-hint-warning/cardinality-hint-warning.02.query.sqlpp new file mode 100644 index 0000000..0ed9c00 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/cardinality-hint-warning/cardinality-hint-warning.02.query.sqlpp @@ -0,0 +1,32 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +/* +* Description : Test warnings for cardinality hints +* Expected Res : Warning, ignore hint +* Date : 03/09/2023 +*/ +// requesttype=application/json +// param max-warnings:json=10 + +use tpch; + +SELECT count(*) +FROM customer c, nation n +WHERE c.c_nationkey = n.n_nationkey and n.n_name /*+ selectivity */ = 'UNITED STATES'; + diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/cardinality-hint-warning/cardinality-hint-warning.03.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/cardinality-hint-warning/cardinality-hint-warning.03.query.sqlpp new file mode 100644 index 0000000..f3838eb9 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/cardinality-hint-warning/cardinality-hint-warning.03.query.sqlpp @@ -0,0 +1,31 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +/* +* Description : Test warnings for cardinality hints +* Expected Res : Warning, ignore hint +* Date : 03/09/2023 +*/ +// requesttype=application/json +// param max-warnings:json=10 + +use tpch; + +SELECT count(*) +FROM customer c, nation n +WHERE c.c_nationkey = n.n_nationkey and n.n_name /*+ selectivity 1 */ = 'UNITED STATES'; diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/cardinality-hint-warning/cardinality-hint-warning.04.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/cardinality-hint-warning/cardinality-hint-warning.04.query.sqlpp new file mode 100644 index 0000000..b40ff68 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/cardinality-hint-warning/cardinality-hint-warning.04.query.sqlpp @@ -0,0 +1,31 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +/* +* Description : Test warnings for cardinality hints +* Expected Res : No warning, honor hint +* Date : 03/09/2023 +*/ +// requesttype=application/json +// param max-warnings:json=10 + +use tpch; + +SELECT count(*) +FROM customer c, nation n +WHERE c.c_nationkey = n.n_nationkey and n.n_name /*+ selectivity 0.1 */ = 'UNITED STATES'; diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/cardinality-hint-warning/cardinality-hint-warning.05.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/cardinality-hint-warning/cardinality-hint-warning.05.query.sqlpp new file mode 100644 index 0000000..0d98c86 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/cardinality-hint-warning/cardinality-hint-warning.05.query.sqlpp @@ -0,0 +1,31 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +/* +* Description : Test warnings for cardinality hints +* Expected Res : Warning, ignore hint +* Date : 03/09/2023 +*/ +// requesttype=application/json +// param max-warnings:json=10 + +use tpch; + +SELECT count(*) +FROM customer c, nation n +WHERE c.c_nationkey /*+ productivity */ = n.n_nationkey; diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/cardinality-hint-warning/cardinality-hint-warning.06.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/cardinality-hint-warning/cardinality-hint-warning.06.query.sqlpp new file mode 100644 index 0000000..21d4735 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/cardinality-hint-warning/cardinality-hint-warning.06.query.sqlpp @@ -0,0 +1,31 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +/* +* Description : Test warnings for enhanced hash join hint +* Expected Res : Warning, ignore hint +* Date : 03/09/2023 +*/ +// requesttype=application/json +// param max-warnings:json=10 + +use tpch; + +SELECT count(*) +FROM customer c, nation n +WHERE c.c_nationkey /*+ productivity 100.0 */ = n.n_nationkey; diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/cardinality-hint-warning/cardinality-hint-warning.07.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/cardinality-hint-warning/cardinality-hint-warning.07.query.sqlpp new file mode 100644 index 0000000..626490a --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/cardinality-hint-warning/cardinality-hint-warning.07.query.sqlpp @@ -0,0 +1,31 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +/* +* Description : Test warnings for cardinality hints +* Expected Res : Warning, ignore hint +* Date : 03/09/2023 +*/ +// requesttype=application/json +// param max-warnings:json=10 + +use tpch; + +SELECT count(*) +FROM customer c, nation n +WHERE c.c_nationkey /*+ productivity n */ = n.n_nationkey; diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/cardinality-hint-warning/cardinality-hint-warning.08.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/cardinality-hint-warning/cardinality-hint-warning.08.query.sqlpp new file mode 100644 index 0000000..6a6a440 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/cardinality-hint-warning/cardinality-hint-warning.08.query.sqlpp @@ -0,0 +1,31 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +/* +* Description : Test warnings for cardinality hints +* Expected Res : Warning, ignore hint +* Date : 03/09/2023 +*/ +// requesttype=application/json +// param max-warnings:json=10 + +use tpch; + +SELECT count(*) +FROM customer c, nation n +WHERE c.c_nationkey /*+ productivity n 100 */ = n.n_nationkey; diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/cardinality-hint-warning/cardinality-hint-warning.09.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/cardinality-hint-warning/cardinality-hint-warning.09.query.sqlpp new file mode 100644 index 0000000..725b0ea --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/cardinality-hint-warning/cardinality-hint-warning.09.query.sqlpp @@ -0,0 +1,31 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +/* +* Description : Test warnings for cardinality hints +* Expected Res : No warning, honor hint +* Date : 03/09/2023 +*/ +// requesttype=application/json +// param max-warnings:json=10 + +use tpch; + +SELECT count(*) +FROM customer c, nation n +WHERE c.c_nationkey /*+ productivity n 100.0 */ = n.n_nationkey; diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/hashjoin-hint-warning/hashjoin-hint-warning.01.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/hashjoin-hint-warning/hashjoin-hint-warning.01.ddl.sqlpp index 14ff86d..b64fa28 100644 --- a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/hashjoin-hint-warning/hashjoin-hint-warning.01.ddl.sqlpp +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/hashjoin-hint-warning/hashjoin-hint-warning.01.ddl.sqlpp @@ -37,7 +37,8 @@ }; CREATE TYPE NationType AS { - n_nationkey : integer + n_nationkey : integer, + n_name : string }; CREATE DATASET orders(OrderType) PRIMARY KEY o_orderkey; diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/hashjoin-hint-warning/hashjoin-hint-warning.12.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/hashjoin-hint-warning/hashjoin-hint-warning.12.query.sqlpp new file mode 100644 index 0000000..f6aebcd --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/hashjoin-hint-warning/hashjoin-hint-warning.12.query.sqlpp @@ -0,0 +1,30 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +/* +* Description : Test warnings for enhanced hash join hint +* Expected Res : No warning, honor hint +* Date : 11/20/2022 +*/ +// requesttype=application/json +// param max-warnings:json=10 + +use tpch; +set `compiler.cbo` "true"; +SELECT COUNT(DISTINCT n.n_name) FROM customer c JOIN nation n +ON c.c_nationkey /*+ hashjoin build(n) */ = n.n_nationkey; diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/cardinality-hint-warning/cardinality-hint-warning.02.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/cardinality-hint-warning/cardinality-hint-warning.02.adm new file mode 100644 index 0000000..3ff59f6 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/cardinality-hint-warning/cardinality-hint-warning.02.adm @@ -0,0 +1 @@ +{ "$1": 0 } \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/cardinality-hint-warning/cardinality-hint-warning.03.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/cardinality-hint-warning/cardinality-hint-warning.03.adm new file mode 100644 index 0000000..3ff59f6 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/cardinality-hint-warning/cardinality-hint-warning.03.adm @@ -0,0 +1 @@ +{ "$1": 0 } \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/cardinality-hint-warning/cardinality-hint-warning.04.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/cardinality-hint-warning/cardinality-hint-warning.04.adm new file mode 100644 index 0000000..3ff59f6 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/cardinality-hint-warning/cardinality-hint-warning.04.adm @@ -0,0 +1 @@ +{ "$1": 0 } \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/cardinality-hint-warning/cardinality-hint-warning.05.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/cardinality-hint-warning/cardinality-hint-warning.05.adm new file mode 100644 index 0000000..3ff59f6 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/cardinality-hint-warning/cardinality-hint-warning.05.adm @@ -0,0 +1 @@ +{ "$1": 0 } \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/cardinality-hint-warning/cardinality-hint-warning.06.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/cardinality-hint-warning/cardinality-hint-warning.06.adm new file mode 100644 index 0000000..3ff59f6 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/cardinality-hint-warning/cardinality-hint-warning.06.adm @@ -0,0 +1 @@ +{ "$1": 0 } \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/cardinality-hint-warning/cardinality-hint-warning.07.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/cardinality-hint-warning/cardinality-hint-warning.07.adm new file mode 100644 index 0000000..3ff59f6 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/cardinality-hint-warning/cardinality-hint-warning.07.adm @@ -0,0 +1 @@ +{ "$1": 0 } \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/cardinality-hint-warning/cardinality-hint-warning.08.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/cardinality-hint-warning/cardinality-hint-warning.08.adm new file mode 100644 index 0000000..3ff59f6 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/cardinality-hint-warning/cardinality-hint-warning.08.adm @@ -0,0 +1 @@ +{ "$1": 0 } \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/cardinality-hint-warning/cardinality-hint-warning.09.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/cardinality-hint-warning/cardinality-hint-warning.09.adm new file mode 100644 index 0000000..3ff59f6 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/cardinality-hint-warning/cardinality-hint-warning.09.adm @@ -0,0 +1 @@ +{ "$1": 0 } \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/hashjoin-hint-warning/hashjoin-hint-warning.12.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/hashjoin-hint-warning/hashjoin-hint-warning.12.adm new file mode 100644 index 0000000..267992b --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/hashjoin-hint-warning/hashjoin-hint-warning.12.adm @@ -0,0 +1 @@ +{ "$1": 0 } diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml b/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml index 31183f3..f9e2e70 100644 --- a/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml +++ b/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml @@ -15461,6 +15461,17 @@ </compilation-unit> </test-case> <test-case FilePath="warnings" check-warnings="true"> + <compilation-unit name="cardinality-hint-warning"> + <output-dir compare="Text">cardinality-hint-warning</output-dir> + <expected-warn><![CDATA[ASX1132: Invalid specification for hint selectivity. Expected selectivity value (in line 31, at column 52)]]></expected-warn> + <expected-warn><![CDATA[ASX1132: Invalid specification for hint selectivity. Selectivity value has to be in decimal format (in line 31, at column 52)]]></expected-warn> + <expected-warn><![CDATA[ASX1132: Invalid specification for hint productivity. Expected productivity collection name and value (in line 31, at column 23)]]></expected-warn> + <expected-warn><![CDATA[ASX1132: Invalid specification for hint productivity. Invalid format for productivity values (in line 31, at column 23)]]></expected-warn> + <expected-warn><![CDATA[ASX1132: Invalid specification for hint productivity. Invalid format for productivity values (in line 31, at column 23)]]></expected-warn> + <expected-warn><![CDATA[ASX1132: Invalid specification for hint productivity. Invalid format for productivity values (in line 31, at column 23)]]></expected-warn> + </compilation-unit> + </test-case> + <test-case FilePath="warnings" check-warnings="true"> <compilation-unit name="inapplicable-hint-warning"> <output-dir compare="Text">inapplicable-hint-warning</output-dir> <expected-warn>HYR10006: Could not apply Group By hint: hash</expected-warn> diff --git a/asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj b/asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj index 98fc560..875de45 100644 --- a/asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj +++ b/asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj @@ -689,39 +689,53 @@ Pattern lessThanOnePat = Pattern.compile("0\\.\\d+"); try { switch (hintToken.hint) { - case SINGLE_DATASET_PREDICATE_SELECTIVITY_HINT: - if (hintToken.hintParams == null) { - throw new SqlppParseException(getSourceLocation(hintToken), "Expected selectivity value"); - } - else { - selectivity = 1.0; // uninitialized - Matcher mat = lessThanOnePat.matcher(hintToken.hintParams); - if (mat.find()) { - selectivity = Double.parseDouble (mat.group()); - } - return new PredicateCardinalityAnnotation(selectivity); - } - case JOIN_PREDICATE_PRODUCTIVITY_HINT: - if (hintToken.hintParams == null) { - throw new SqlppParseException(getSourceLocation(hintToken), "Expected productivity value"); - } - else { - productivity = 1.0; // uninitialized - String leftSideDataSet = null; - Matcher StringNum = stringNumber.matcher(hintToken.hintParams); - if (StringNum.find()) { - String matchedGroup = StringNum.group(); - Pattern var = Pattern.compile("[a-zA-Z]\\w*"); // any word character [a-zA-Z_0-9] - Matcher matVar = var.matcher(matchedGroup); - if (matVar.find()) - leftSideDataSet = matVar.group(); - Matcher numMat = number.matcher(matchedGroup); - if (numMat.find()) - productivity = Double.parseDouble (numMat.group()); - } - // attach hint to global scope - return new JoinProductivityAnnotation (productivity, leftSideDataSet); - } + case SINGLE_DATASET_PREDICATE_SELECTIVITY_HINT: + if (hintToken.hintParams == null) { + throw new SqlppParseException(getSourceLocation(hintToken), "Expected selectivity value"); + } + else { + selectivity = 1.0; // uninitialized + Matcher mat = lessThanOnePat.matcher(hintToken.hintParams); + if (mat.find()) { + selectivity = Double.parseDouble (mat.group()); + } + else { + throw new SqlppParseException(getSourceLocation(hintToken), "Selectivity value has to be in decimal format"); + } + return new PredicateCardinalityAnnotation(selectivity); + } + case JOIN_PREDICATE_PRODUCTIVITY_HINT: + if (hintToken.hintParams == null) { + throw new SqlppParseException(getSourceLocation(hintToken), "Expected productivity collection name and value"); + } + else { + productivity = 1.0; // uninitialized + String leftSideDataSet = null; + Matcher StringNum = stringNumber.matcher(hintToken.hintParams); + if (StringNum.find()) { + String matchedGroup = StringNum.group(); + Pattern var = Pattern.compile("[a-zA-Z]\\w*"); // any word character [a-zA-Z_0-9] + Matcher matVar = var.matcher(matchedGroup); + if (matVar.find()) { + leftSideDataSet = matVar.group(); + } + else { + throw new SqlppParseException(getSourceLocation(hintToken), "Expected productivity collection name"); + } + Matcher numMat = number.matcher(matchedGroup); + if (numMat.find()) { + productivity = Double.parseDouble (numMat.group()); + } + else { + throw new SqlppParseException(getSourceLocation(hintToken), "Productivity value has to be in decimal format"); + } + } + else { + throw new SqlppParseException(getSourceLocation(hintToken), "Invalid format for productivity values"); + } + // attach hint to global scope + return new JoinProductivityAnnotation (productivity, leftSideDataSet); + } case HASH_BROADCAST_JOIN_HINT: if (hintToken.hintParams == null) { return new BroadcastExpressionAnnotation(BroadcastExpressionAnnotation.BroadcastSide.RIGHT); diff --git a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/VariableUtilities.java b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/VariableUtilities.java index 69db58d..623e15a 100644 --- a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/VariableUtilities.java +++ b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/VariableUtilities.java @@ -33,6 +33,7 @@ import org.apache.hyracks.algebricks.core.algebra.base.ILogicalExpression; import org.apache.hyracks.algebricks.core.algebra.base.ILogicalOperator; import org.apache.hyracks.algebricks.core.algebra.base.LogicalExpressionTag; +import org.apache.hyracks.algebricks.core.algebra.base.LogicalOperatorTag; import org.apache.hyracks.algebricks.core.algebra.base.LogicalVariable; import org.apache.hyracks.algebricks.core.algebra.expressions.VariableReferenceExpression; import org.apache.hyracks.algebricks.core.algebra.typing.ITypingContext; @@ -102,6 +103,17 @@ } } + public static void getLiveVariablesInDescendantDataScans(ILogicalOperator op, Collection<LogicalVariable> vars) + throws AlgebricksException { + // DFS traversal + if (op.getOperatorTag() == LogicalOperatorTag.DATASOURCESCAN) { + VariableUtilities.getLiveVariables(op, vars); + } + for (Mutable<ILogicalOperator> c : op.getInputs()) { + getLiveVariablesInDescendantDataScans(c.getValue(), vars); + } + } + public static void getProducedVariablesInDescendantsAndSelf(ILogicalOperator op, Collection<LogicalVariable> vars) throws AlgebricksException { // DFS traversal diff --git a/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/util/JoinUtils.java b/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/util/JoinUtils.java index fa0549f..49ff483 100644 --- a/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/util/JoinUtils.java +++ b/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/util/JoinUtils.java @@ -40,6 +40,7 @@ import org.apache.hyracks.algebricks.core.algebra.functions.FunctionIdentifier; import org.apache.hyracks.algebricks.core.algebra.operators.logical.AbstractBinaryJoinOperator; import org.apache.hyracks.algebricks.core.algebra.operators.logical.visitors.LogicalPropertiesVisitor; +import org.apache.hyracks.algebricks.core.algebra.operators.logical.visitors.VariableUtilities; import org.apache.hyracks.algebricks.core.algebra.operators.physical.AbstractJoinPOperator.JoinPartitioningType; import org.apache.hyracks.algebricks.core.algebra.operators.physical.HybridHashJoinPOperator; import org.apache.hyracks.algebricks.core.algebra.operators.physical.InMemoryHashJoinPOperator; @@ -57,7 +58,7 @@ } public static void setJoinAlgorithmAndExchangeAlgo(AbstractBinaryJoinOperator op, boolean topLevelOp, - IOptimizationContext context) { + IOptimizationContext context) throws AlgebricksException { if (!topLevelOp) { throw new IllegalStateException("Micro operator not implemented for: " + op.getOperatorTag()); } @@ -67,9 +68,13 @@ List<LogicalVariable> varsRight = op.getInputs().get(1).getValue().getSchema(); ILogicalExpression conditionExpr = op.getCondition().getValue(); if (isHashJoinCondition(conditionExpr, varsLeft, varsRight, sideLeft, sideRight)) { - BroadcastSide broadcastSide = getBroadcastJoinSide(conditionExpr, varsLeft, varsRight, context); + List<LogicalVariable> scanVarsLeft = new LinkedList<>(); + List<LogicalVariable> scanVarsRight = new LinkedList<>(); + VariableUtilities.getLiveVariablesInDescendantDataScans(op.getInputs().get(0).getValue(), scanVarsLeft); + VariableUtilities.getLiveVariablesInDescendantDataScans(op.getInputs().get(1).getValue(), scanVarsRight); + BroadcastSide broadcastSide = getBroadcastJoinSide(conditionExpr, scanVarsLeft, scanVarsRight, context); if (broadcastSide == null) { - BuildSide buildSide = getHashJoinBuildSide(conditionExpr, varsLeft, varsRight, context); + BuildSide buildSide = getHashJoinBuildSide(conditionExpr, scanVarsLeft, scanVarsRight, context); if (buildSide == null) { setHashJoinOp(op, JoinPartitioningType.PAIRWISE, sideLeft, sideRight, context); } else { -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/17411 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: neo Gerrit-Change-Id: Ia26a912063aaf063a530acc7ff1a7a006f0ad0c9 Gerrit-Change-Number: 17411 Gerrit-PatchSet: 6 Gerrit-Owner: Vijay Sarathy <[email protected]> Gerrit-Reviewer: Ali Alsuliman <[email protected]> Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins <[email protected]> Gerrit-Reviewer: Vijay Sarathy <[email protected]> Gerrit-MessageType: merged
