>From Ali Alsuliman <[email protected]>: Ali Alsuliman has submitted this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18215 )
Change subject: [ASTERIXDB-3369][FUN] Add tests for SQL median() ...................................................................... [ASTERIXDB-3369][FUN] Add tests for SQL median() - user model changes: no - storage format changes: no - interface changes: no Change-Id: Ica618a51a0c577cc3e68c48d6cab1624e5a1ff4b Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18215 Reviewed-by: Ali Alsuliman <[email protected]> Reviewed-by: Murtadha Hubail <[email protected]> Tested-by: Jenkins <[email protected]> --- A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/median/median.05.adm A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.09.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/median/median.09.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/median/median.10.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/median/median.06.adm M hyracks-fullstack/hyracks/hyracks-comm/src/main/java/org/apache/hyracks/comm/channels/NetworkInputChannel.java M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.10.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.11.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/median/median.03.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/median/median.11.adm M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractMedianAggregateFunction.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.02.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/median/median.07.adm M hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/partitions/MaterializedPartition.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.07.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.08.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.99.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/median/median.04.adm A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.06.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.01.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.03.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.04.query.sqlpp M hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/partitions/PartitionFileReaderUtil.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.05.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/median/median.08.adm M hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/partitions/PartitionFileReader.java 27 files changed, 383 insertions(+), 13 deletions(-) Approvals: Murtadha Hubail: Looks good to me, approved Ali Alsuliman: Looks good to me, but someone else must approve Jenkins: Verified Objections: Anon. E. Moose #1000171: Violations found diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.01.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.01.ddl.sqlpp new file mode 100644 index 0000000..8739d27 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.01.ddl.sqlpp @@ -0,0 +1,29 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +DROP DATAVERSE test IF EXISTS; +CREATE DATAVERSE test; +USE test; + +CREATE TYPE openType AS {id: int}; +CREATE DATASET large_ds(openType) primary key id; +CREATE DATASET odd_ds(openType) primary key id; +CREATE DATASET even_ds(openType) primary key id; +CREATE DATASET empty_ds(openType) primary key id; +CREATE DATASET one_item_ds(openType) primary key id; \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.02.update.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.02.update.sqlpp new file mode 100644 index 0000000..5f3816b --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.02.update.sqlpp @@ -0,0 +1,44 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +USE test; + +UPSERT INTO one_item_ds {"id": 1, "m": 9}; + +UPSERT INTO large_ds +(FROM range(0, 5) as v + SELECT v AS id, v AS m, v % 15 AS g +); + +UPSERT INTO large_ds +(FROM range(6, 1000000) as v + SELECT v AS id, round_half_to_even(random(8) * 100, 1) AS m, v % 15 AS g +); + +UPSERT INTO large_ds( [{"id": 1000001, "m": null, "g": 1}, {"id": 1000002, "g": 7}] ); + +UPSERT INTO odd_ds +(FROM range(1, 15) as v + SELECT v AS id, 513 % v AS m +); + +UPSERT INTO even_ds +(FROM range(1, 14) as v + SELECT v AS id, 513 % v AS m +); \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.03.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.03.query.sqlpp new file mode 100644 index 0000000..9668a90 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.03.query.sqlpp @@ -0,0 +1,22 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +USE test; + +SELECT median(m) AS med FROM odd_ds; \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.04.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.04.query.sqlpp new file mode 100644 index 0000000..5b52dcd --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.04.query.sqlpp @@ -0,0 +1,22 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +USE test; + +SELECT median(m) AS med FROM even_ds; \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.05.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.05.query.sqlpp new file mode 100644 index 0000000..9773b9f --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.05.query.sqlpp @@ -0,0 +1,23 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +USE test; + +SELECT median(v) AS med +FROM [0,1,0,1,3,3,2,1,0,3,7,9,6,9,3] AS v; \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.06.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.06.query.sqlpp new file mode 100644 index 0000000..68b7be0 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.06.query.sqlpp @@ -0,0 +1,23 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +USE test; + +SELECT median(v) AS med +FROM [0,1,0,1,3,3,2,1,0,3,7,9,6,9] AS v; \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.07.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.07.query.sqlpp new file mode 100644 index 0000000..95dc135 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.07.query.sqlpp @@ -0,0 +1,23 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +USE test; + +SELECT median(m) AS med +FROM empty_ds; \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.08.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.08.query.sqlpp new file mode 100644 index 0000000..daac19b --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.08.query.sqlpp @@ -0,0 +1,23 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +USE test; + +SELECT median(m) AS med +FROM one_item_ds; \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.09.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.09.query.sqlpp new file mode 100644 index 0000000..9927bab --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.09.query.sqlpp @@ -0,0 +1,23 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +USE test; + +SELECT median(m) AS med +FROM large_ds; \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.10.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.10.query.sqlpp new file mode 100644 index 0000000..e4b1d11 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.10.query.sqlpp @@ -0,0 +1,25 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +USE test; + +SELECT g, median(m) AS med +FROM large_ds +GROUP BY g +ORDER BY g; \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.11.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.11.query.sqlpp new file mode 100644 index 0000000..4880e98 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.11.query.sqlpp @@ -0,0 +1,23 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +USE test; +SET `compiler.sortmemory` "130KB"; +SELECT median(m) AS med +FROM large_ds; \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.99.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.99.ddl.sqlpp new file mode 100644 index 0000000..36b2bab --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/median/median.99.ddl.sqlpp @@ -0,0 +1,20 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +DROP DATAVERSE test IF EXISTS; \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/median/median.03.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/median/median.03.adm new file mode 100644 index 0000000..6d291ab --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/median/median.03.adm @@ -0,0 +1 @@ +{ "med": 3.0 } \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/median/median.04.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/median/median.04.adm new file mode 100644 index 0000000..9506c5b --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/median/median.04.adm @@ -0,0 +1 @@ +{ "med": 2.5 } \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/median/median.05.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/median/median.05.adm new file mode 100644 index 0000000..6d291ab --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/median/median.05.adm @@ -0,0 +1 @@ +{ "med": 3.0 } \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/median/median.06.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/median/median.06.adm new file mode 100644 index 0000000..9506c5b --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/median/median.06.adm @@ -0,0 +1 @@ +{ "med": 2.5 } \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/median/median.07.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/median/median.07.adm new file mode 100644 index 0000000..dd133f3 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/median/median.07.adm @@ -0,0 +1 @@ +{ "med": null } \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/median/median.08.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/median/median.08.adm new file mode 100644 index 0000000..a4a8fa5 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/median/median.08.adm @@ -0,0 +1 @@ +{ "med": 9.0 } \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/median/median.09.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/median/median.09.adm new file mode 100644 index 0000000..8f23892 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/median/median.09.adm @@ -0,0 +1 @@ +{ "med": 50.1 } \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/median/median.10.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/median/median.10.adm new file mode 100644 index 0000000..d5f5227 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/median/median.10.adm @@ -0,0 +1,15 @@ +{ "med": 50.0, "g": 0 } +{ "med": 49.8, "g": 1 } +{ "med": 50.0, "g": 2 } +{ "med": 50.2, "g": 3 } +{ "med": 50.3, "g": 4 } +{ "med": 50.0, "g": 5 } +{ "med": 50.1, "g": 6 } +{ "med": 50.1, "g": 7 } +{ "med": 50.2, "g": 8 } +{ "med": 50.1, "g": 9 } +{ "med": 50.3, "g": 10 } +{ "med": 49.8, "g": 11 } +{ "med": 49.9, "g": 12 } +{ "med": 49.9, "g": 13 } +{ "med": 50.2, "g": 14 } \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/median/median.11.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/median/median.11.adm new file mode 100644 index 0000000..8f23892 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/median/median.11.adm @@ -0,0 +1 @@ +{ "med": 50.1 } \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml b/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml index a9fbbe3..6abac01 100644 --- a/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml +++ b/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml @@ -1625,6 +1625,11 @@ </test-group> <test-group name="aggregate-sql"> <test-case FilePath="aggregate-sql"> + <compilation-unit name="median"> + <output-dir compare="Text">median</output-dir> + </compilation-unit> + </test-case> + <test-case FilePath="aggregate-sql"> <compilation-unit name="min_max_arrays"> <output-dir compare="Text">min_max_arrays</output-dir> </compilation-unit> diff --git a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractMedianAggregateFunction.java b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractMedianAggregateFunction.java index a695c23..207eaa3 100644 --- a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractMedianAggregateFunction.java +++ b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractMedianAggregateFunction.java @@ -83,9 +83,13 @@ import org.apache.hyracks.dataflow.std.collectors.InputChannelFrameReader; import org.apache.hyracks.dataflow.std.sort.RunMergingFrameReader; import org.apache.hyracks.util.string.UTF8StringUtil; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; public abstract class AbstractMedianAggregateFunction extends AbstractAggregateFunction { + private static final Logger LOGGER = LogManager.getLogger(); + protected static final String MEDIAN = "median"; private static final int COUNT_FIELD_ID = 0; private static final int HANDLE_FIELD_ID = 1; @@ -215,17 +219,21 @@ return; } try { - double medianVal = findMedian(); + boolean medianFound = findMedian(); resultStorage.reset(); - aDouble.setValue(medianVal); - doubleSerde.serialize(aDouble, resultStorage.getDataOutput()); - result.set(resultStorage); + if (medianFound) { + doubleSerde.serialize(aDouble, resultStorage.getDataOutput()); + result.set(resultStorage); + } else { + PointableHelper.setNull(result); + LOGGER.warn("median was not found"); + } } catch (IOException e) { throw HyracksDataException.create(e); } } - private double findMedian() throws HyracksDataException { + private boolean findMedian() throws HyracksDataException { RunMergingFrameReader merger = createRunsMergingFrameReader(); return getMedian(merger); } @@ -248,11 +256,11 @@ doubleNkComputerFactory.createNormalizedKeyComputer(), recordDesc); } - private double getMedian(RunMergingFrameReader merger) throws HyracksDataException { + private boolean getMedian(RunMergingFrameReader merger) throws HyracksDataException { boolean isOdd = count % 2 != 0; long medianPosition = isOdd ? count / 2 : (count - 1) / 2; long currentTupleCount = 0; - double medianVal = -1; + boolean found = false; merger.open(); try { while (merger.nextFrame(frame)) { @@ -261,7 +269,8 @@ if (currentTupleCount + tupleCount > medianPosition) { int firstMedian = (int) (medianPosition - currentTupleCount); ftr.reset(fta, firstMedian); - medianVal = ADoubleSerializerDeserializer.getDouble(ftr.getFieldData(0), ftr.getFieldStart(0) + 1); + double medianVal = + ADoubleSerializerDeserializer.getDouble(ftr.getFieldData(0), ftr.getFieldStart(0) + 1); if (!isOdd) { if (firstMedian + 1 < tupleCount) { // second median is in the same frame @@ -276,14 +285,19 @@ (ADoubleSerializerDeserializer.getDouble(ftr.getFieldData(0), ftr.getFieldStart(0) + 1) + medianVal) / 2; } + aDouble.setValue(medianVal); + found = true; break; } currentTupleCount += tupleCount; } + while (merger.nextFrame(frame)) { + // consume the remaining frames to close the network channels gracefully + } } finally { merger.close(); } - return medianVal; + return found; } protected void setPartialResult(IPointable result, long fileId, String address, int port) diff --git a/hyracks-fullstack/hyracks/hyracks-comm/src/main/java/org/apache/hyracks/comm/channels/NetworkInputChannel.java b/hyracks-fullstack/hyracks/hyracks-comm/src/main/java/org/apache/hyracks/comm/channels/NetworkInputChannel.java index 5ae81fb..18cab49 100644 --- a/hyracks-fullstack/hyracks/hyracks-comm/src/main/java/org/apache/hyracks/comm/channels/NetworkInputChannel.java +++ b/hyracks-fullstack/hyracks/hyracks-comm/src/main/java/org/apache/hyracks/comm/channels/NetworkInputChannel.java @@ -37,6 +37,7 @@ public class NetworkInputChannel implements IInputChannel { private static final Logger LOGGER = LogManager.getLogger(); + private static final int INITIAL_MSG_FILLER = -1; public static final int INITIAL_MESSAGE_SIZE = 24; private final IChannelConnectionFactory netManager; @@ -106,6 +107,7 @@ writeBuffer.putInt(partitionId.getConnectorDescriptorId().getId()); writeBuffer.putInt(partitionId.getSenderIndex()); writeBuffer.putInt(partitionId.getReceiverIndex()); + writeBuffer.putInt(INITIAL_MSG_FILLER); writeBuffer.flip(); if (LOGGER.isTraceEnabled()) { LOGGER.trace("Sending partition request: " + partitionId + " on channel: " + ccb); diff --git a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/partitions/MaterializedPartition.java b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/partitions/MaterializedPartition.java index 218e557..1471453 100644 --- a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/partitions/MaterializedPartition.java +++ b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/partitions/MaterializedPartition.java @@ -57,7 +57,7 @@ @Override public void writeTo(final IFrameWriter writer) { - executor.execute(new PartitionFileReader(ctx, partitionFile, ioManager, writer)); + executor.execute(new PartitionFileReader(ctx, partitionFile, ioManager, writer, false)); } @Override diff --git a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/partitions/PartitionFileReader.java b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/partitions/PartitionFileReader.java index a0d8dcd..609b32a 100644 --- a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/partitions/PartitionFileReader.java +++ b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/partitions/PartitionFileReader.java @@ -20,6 +20,7 @@ import java.nio.ByteBuffer; +import org.apache.commons.io.FileUtils; import org.apache.hyracks.api.comm.IFrameWriter; import org.apache.hyracks.api.context.IHyracksCommonContext; import org.apache.hyracks.api.exceptions.HyracksDataException; @@ -33,13 +34,15 @@ private final FileReference partitionFile; private final IIOManager ioManager; private final IFrameWriter writer; + private final boolean deleteFile; public PartitionFileReader(IHyracksCommonContext ctx, FileReference partitionFile, IIOManager ioManager, - IFrameWriter writer) { + IFrameWriter writer, boolean deleteFile) { this.ctx = ctx; this.partitionFile = partitionFile; this.ioManager = ioManager; this.writer = writer; + this.deleteFile = deleteFile; } @Override @@ -73,7 +76,13 @@ writer.close(); } } finally { - ioManager.close(fh); + try { + ioManager.close(fh); + } finally { + if (deleteFile) { + FileUtils.deleteQuietly(partitionFile.getFile()); + } + } } } catch (HyracksDataException e) { throw new RuntimeException(e); diff --git a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/partitions/PartitionFileReaderUtil.java b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/partitions/PartitionFileReaderUtil.java index 2675c27..2a105d9 100644 --- a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/partitions/PartitionFileReaderUtil.java +++ b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/partitions/PartitionFileReaderUtil.java @@ -53,6 +53,6 @@ } ExecutorService executor = ncs.getExecutor(); noc.setFrameSize(joblet.getInitialFrameSize()); - executor.execute(new PartitionFileReader(joblet, fileRef, ncs.getIoManager(), noc)); + executor.execute(new PartitionFileReader(joblet, fileRef, ncs.getIoManager(), noc, true)); } } -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18215 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: neo Gerrit-Change-Id: Ica618a51a0c577cc3e68c48d6cab1624e5a1ff4b Gerrit-Change-Number: 18215 Gerrit-PatchSet: 3 Gerrit-Owner: Ali Alsuliman <[email protected]> Gerrit-Reviewer: Ali Alsuliman <[email protected]> Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins <[email protected]> Gerrit-Reviewer: Michael Blow <[email protected]> Gerrit-Reviewer: Murtadha Hubail <[email protected]> Gerrit-Reviewer: Wail Alkowaileet <[email protected]> Gerrit-MessageType: merged
