Dmitry Lychagin has submitted this change and it was merged. Change subject: [ASTERIXDB-2483][COMP][FUN] Eliminate listify for distinct aggregates ......................................................................
[ASTERIXDB-2483][COMP][FUN] Eliminate listify for distinct aggregates - user model changes: no - storage format changes: no - interface changes: no Details: - Move distinct aggregate rewriting from SqlppQueryRewriter to RewriteDistinctAggregateRule in the optimizer - Add runtime for scalar distinct aggregates - Fix ExtractCommonOperatorsRule handling of binary operators - Additional tests for distinct aggregates Change-Id: If13ea2696e9e0a8a639db684656e5642991c1f99 Reviewed-on: https://asterix-gerrit.ics.uci.edu/3293 Reviewed-by: Ali Alsuliman <[email protected]> Tested-by: Dmitry Lychagin <[email protected]> --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java A asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/RewriteDistinctAggregateRule.java A asterixdb/asterix-app/src/test/resources/optimizerts/queries/aggregate-sql-sugar/distinct_mixed/distinct_mixed.10.sqlpp A asterixdb/asterix-app/src/test/resources/optimizerts/queries/aggregate-sql-sugar/distinct_mixed/distinct_mixed.11.sqlpp A asterixdb/asterix-app/src/test/resources/optimizerts/queries/aggregate-sql-sugar/distinct_mixed/distinct_mixed.12.sqlpp A asterixdb/asterix-app/src/test/resources/optimizerts/queries/aggregate-sql-sugar/distinct_mixed/distinct_mixed.13.sqlpp A asterixdb/asterix-app/src/test/resources/optimizerts/queries/aggregate-sql-sugar/distinct_mixed/distinct_mixed.14.sqlpp A asterixdb/asterix-app/src/test/resources/optimizerts/queries/aggregate-sql-sugar/distinct_mixed/distinct_mixed.15.sqlpp A asterixdb/asterix-app/src/test/resources/optimizerts/queries/aggregate-sql-sugar/distinct_mixed/distinct_mixed.16.sqlpp A asterixdb/asterix-app/src/test/resources/optimizerts/queries/aggregate-sql-sugar/distinct_mixed/distinct_mixed.17.sqlpp A asterixdb/asterix-app/src/test/resources/optimizerts/queries/aggregate-sql-sugar/distinct_mixed/distinct_mixed.3.sqlpp A asterixdb/asterix-app/src/test/resources/optimizerts/queries/aggregate-sql-sugar/distinct_mixed/distinct_mixed.4.sqlpp A asterixdb/asterix-app/src/test/resources/optimizerts/queries/aggregate-sql-sugar/distinct_mixed/distinct_mixed.5.sqlpp A asterixdb/asterix-app/src/test/resources/optimizerts/queries/aggregate-sql-sugar/distinct_mixed/distinct_mixed.6.sqlpp A asterixdb/asterix-app/src/test/resources/optimizerts/queries/aggregate-sql-sugar/distinct_mixed/distinct_mixed.7.sqlpp A asterixdb/asterix-app/src/test/resources/optimizerts/queries/aggregate-sql-sugar/distinct_mixed/distinct_mixed.8.sqlpp A asterixdb/asterix-app/src/test/resources/optimizerts/queries/aggregate-sql-sugar/distinct_mixed/distinct_mixed.9.sqlpp A asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.10.plan A asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.11.plan A asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.12.plan A asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.13.plan A asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.14.plan A asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.15.plan A asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.16.plan A asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.17.plan A asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.3.plan A asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.4.plan A asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.5.plan A asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.6.plan A asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.7.plan A asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.8.plan A asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.9.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/group-by/sugar-06-distinct.plan A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql-sugar/distinct_mixed/distinct_mixed.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql-sugar/distinct_mixed/distinct_mixed.10.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql-sugar/distinct_mixed/distinct_mixed.11.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql-sugar/distinct_mixed/distinct_mixed.12.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql-sugar/distinct_mixed/distinct_mixed.13.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql-sugar/distinct_mixed/distinct_mixed.14.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql-sugar/distinct_mixed/distinct_mixed.15.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql-sugar/distinct_mixed/distinct_mixed.16.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql-sugar/distinct_mixed/distinct_mixed.17.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql-sugar/distinct_mixed/distinct_mixed.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql-sugar/distinct_mixed/distinct_mixed.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql-sugar/distinct_mixed/distinct_mixed.4.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql-sugar/distinct_mixed/distinct_mixed.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql-sugar/distinct_mixed/distinct_mixed.6.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql-sugar/distinct_mixed/distinct_mixed.7.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql-sugar/distinct_mixed/distinct_mixed.8.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql-sugar/distinct_mixed/distinct_mixed.9.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.10.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.11.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.12.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.13.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.14.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.15.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.16.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.17.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.3.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.4.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.5.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.6.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.7.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.8.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.9.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/skewness_distinct/skewness_distinct.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml M asterixdb/asterix-geo/src/main/java/org/apache/asterix/geo/aggregates/ScalarSTUnionAggregateDescriptor.java A asterixdb/asterix-geo/src/main/java/org/apache/asterix/geo/aggregates/ScalarSTUnionDistinctAggregateDescriptor.java M asterixdb/asterix-geo/src/main/java/org/apache/asterix/geo/evaluators/GeoFunctionRegistrant.java M asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/SqlppQueryRewriter.java D asterixdb/asterix-lang-sqlpp/src/main/java/org/apache/asterix/lang/sqlpp/rewrites/visitor/SqlppDistinctAggregationSugarVisitor.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/BuiltinFunctions.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/AbstractScalarAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/AbstractScalarDistinctAggregateDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/GenericScalarAggregateFunction.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/GenericScalarDistinctAggregateFunction.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarAvgAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarAvgDistinctAggregateDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarCountAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarCountDistinctAggregateDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarKurtosisAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarKurtosisDistinctAggregateDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarMaxAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarMaxDistinctAggregateDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarMinAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarMinDistinctAggregateDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarSkewnessAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarSkewnessDistinctAggregateDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarSqlAvgAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarSqlAvgDistinctAggregateDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarSqlCountAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarSqlCountDistinctAggregateDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarSqlKurtosisAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarSqlKurtosisDistinctAggregateDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarSqlMaxAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarSqlMaxDistinctAggregateDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarSqlMinAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarSqlMinDistinctAggregateDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarSqlSkewnessAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarSqlSkewnessDistinctAggregateDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarSqlStddevAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarSqlStddevDistinctAggregateDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarSqlStddevPopAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarSqlStddevPopDistinctAggregateDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarSqlSumAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarSqlSumDistinctAggregateDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarSqlVarAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarSqlVarDistinctAggregateDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarSqlVarPopAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarSqlVarPopDistinctAggregateDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarStddevAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarStddevDistinctAggregateDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarStddevPopAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarStddevPopDistinctAggregateDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarSumAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarSumDistinctAggregateDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarVarAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarVarDistinctAggregateDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarVarPopAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarVarPopDistinctAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/utils/PointableHashSet.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/ArrayDistinctDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/ArrayIntersectDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/ArraySymDiffEval.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/ArrayUnionDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/PointableHelper.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/functions/FunctionCollection.java D asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/utils/ArrayFunctionsUtil.java M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/util/OperatorManipulationUtil.java M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/ExtractCommonOperatorsRule.java 131 files changed, 4,160 insertions(+), 526 deletions(-) Approvals: Anon. E. Moose #1000171: Ali Alsuliman: Looks good to me, approved Dmitry Lychagin: Verified -- To view, visit https://asterix-gerrit.ics.uci.edu/3293 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: merged Gerrit-Change-Id: If13ea2696e9e0a8a639db684656e5642991c1f99 Gerrit-PatchSet: 6 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Dmitry Lychagin <[email protected]> Gerrit-Reviewer: Ali Alsuliman <[email protected]> Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Dmitry Lychagin <[email protected]> Gerrit-Reviewer: Till Westmann <[email protected]>
