Change in asterixdb[master]: [ASTERIXDB-2575][RT][FUN] Remove usages of ComparisonHelper
Hello Anon. E. Moose (1000171), Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3470 to look at the new patch set (#4). Change subject: [ASTERIXDB-2575][RT][FUN] Remove usages of ComparisonHelper .. [ASTERIXDB-2575][RT][FUN] Remove usages of ComparisonHelper - user model changes: no - storage format changes: no - interface changes: no Details: Remove usages of ComparisonHelper and switch to logical or physical comparators. - updated RecordReplace test cases that used to throw an exception when comparing complex types. - cleaned up interval classes. - fixed getStartOffset() in AIntervalPointable. - removed ComparisonHelper Change-Id: Ib2a05f35c752e1d2ecfdbabbf5d303ea73c7c9db --- M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/ObjectsQueries.xml M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/object_replace/object_replace.4.query.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/object_replace/object_replace.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/objects/object_replace/object_replace.4.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/objects/object_replace/object_replace.5.adm M asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/printers/adm/AIntervalPrinterFactory.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/printers/json/clean/AIntervalPrinterFactory.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/printers/json/lossless/AIntervalPrinterFactory.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/serde/AIntervalSerializerDeserializer.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/om/pointables/nonvisitor/AIntervalPointable.java D asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/comparisons/ComparisonHelper.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/records/RecordReplaceDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/records/RecordReplaceEvaluator.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/temporal/AbstractIntervalLogicFuncDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/temporal/GetOverlappingIntervalDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/temporal/IntervalLogic.java 16 files changed, 202 insertions(+), 712 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/70/3470/4 -- To view, visit https://asterix-gerrit.ics.uci.edu/3470 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib2a05f35c752e1d2ecfdbabbf5d303ea73c7c9db Gerrit-Change-Number: 3470 Gerrit-PatchSet: 4 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2575][RT][FUN] Remove usages of ComparisonHelper
Hello Anon. E. Moose (1000171), Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3470 to look at the new patch set (#3). Change subject: [ASTERIXDB-2575][RT][FUN] Remove usages of ComparisonHelper .. [ASTERIXDB-2575][RT][FUN] Remove usages of ComparisonHelper - user model changes: no - storage format changes: no - interface changes: no Details: Remove usages of ComparisonHelper and switch to logical or physical comparators. - updated RecordReplace test cases that used to throw an exception when comparing complex types. - cleaned up interval classes. - fixed getStartOffset() in AIntervalPointable. Change-Id: Ib2a05f35c752e1d2ecfdbabbf5d303ea73c7c9db --- M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/ObjectsQueries.xml M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/object_replace/object_replace.4.query.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/object_replace/object_replace.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/objects/object_replace/object_replace.4.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/objects/object_replace/object_replace.5.adm M asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/printers/adm/AIntervalPrinterFactory.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/printers/json/clean/AIntervalPrinterFactory.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/printers/json/lossless/AIntervalPrinterFactory.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/serde/AIntervalSerializerDeserializer.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/om/pointables/nonvisitor/AIntervalPointable.java D asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/comparisons/ComparisonHelper.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/records/RecordReplaceDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/records/RecordReplaceEvaluator.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/temporal/AbstractIntervalLogicFuncDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/temporal/GetOverlappingIntervalDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/temporal/IntervalLogic.java 16 files changed, 202 insertions(+), 712 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/70/3470/3 -- To view, visit https://asterix-gerrit.ics.uci.edu/3470 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib2a05f35c752e1d2ecfdbabbf5d303ea73c7c9db Gerrit-Change-Number: 3470 Gerrit-PatchSet: 3 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2575][RT][FUN] Remove usages of ComparisonHelper
Ali Alsuliman has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/3470 Change subject: [ASTERIXDB-2575][RT][FUN] Remove usages of ComparisonHelper .. [ASTERIXDB-2575][RT][FUN] Remove usages of ComparisonHelper - user model changes: no - storage format changes: no - interface changes: no Details: Remove usages of ComparisonHelper and switch to logical or physical comparators. - updated RecordReplace test cases that used to throw an exception when comparing complex types. - cleaned up interval classes. - fixed getStartOffset() in AIntervalPointable. Change-Id: Ib2a05f35c752e1d2ecfdbabbf5d303ea73c7c9db --- M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/ObjectsQueries.xml M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/object_replace/object_replace.4.query.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/object_replace/object_replace.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/objects/object_replace/object_replace.4.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/objects/object_replace/object_replace.5.adm M asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/printers/adm/AIntervalPrinterFactory.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/printers/json/clean/AIntervalPrinterFactory.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/printers/json/lossless/AIntervalPrinterFactory.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/serde/AIntervalSerializerDeserializer.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/om/pointables/nonvisitor/AIntervalPointable.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/records/RecordReplaceDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/records/RecordReplaceEvaluator.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/temporal/AbstractIntervalLogicFuncDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/temporal/GetOverlappingIntervalDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/temporal/IntervalLogic.java 15 files changed, 202 insertions(+), 231 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/70/3470/1 diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/ObjectsQueries.xml b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/ObjectsQueries.xml index 150a37f..e37dee9c 100644 --- a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/ObjectsQueries.xml +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/ObjectsQueries.xml @@ -147,8 +147,6 @@ object_replace - function object-replace expects its 2nd input parameter to be of type primitive - function object-replace expects its 2nd input parameter to be of type primitive false diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/object_replace/object_replace.4.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/object_replace/object_replace.4.query.sqlpp index 15c0bce..9e0b525 100644 --- a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/object_replace/object_replace.4.query.sqlpp +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/object_replace/object_replace.4.query.sqlpp @@ -19,9 +19,9 @@ /* * Description : Testing object_replace under different queries. - * Expected Res : Failure + * Expected Res : Success */ use TinySocial; -select value object_replace({"a":1}, {"b":2}, "z"); \ No newline at end of file +select value object_replace({"a":1, "a2": {"b":2}, "a3": {"b2":2}, "a4": {"b":2}}, {"b":2}, "z"); \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/object_replace/object_replace.5.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/object_replace/object_replace.5.query.sqlpp index a46a727..63008dd 100644 --- a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/object_replace/object_replace.5.query.sqlpp +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/object_replace/object_replace.5.query.sqlpp @@ -19,9 +19,9 @@ /* * Description : Testing object_replace under different queries. - * Expected Res : Failure + * Expected Res : Success */ use TinySocial; -select value object_replace({"a":1},
Change in asterixdb[master]: [NO ISSUE][COMP] Fix variable handling in Intersect operator
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3460 ) Change subject: [NO ISSUE][COMP] Fix variable handling in Intersect operator .. Patch Set 1: (1 comment) https://asterix-gerrit.ics.uci.edu/#/c/3460/1/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/IntersectOperator.java File hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/IntersectOperator.java: https://asterix-gerrit.ics.uci.edu/#/c/3460/1/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/IntersectOperator.java@137 PS1, Line 137: inputExtraVars0 I wonder why they check type consistency for compare variables but not extra variables -- To view, visit https://asterix-gerrit.ics.uci.edu/3460 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4f436566e743d7e908a72bfa502c138ae7b99c6c Gerrit-Change-Number: 3460 Gerrit-PatchSet: 1 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Comment-Date: Wed, 26 Jun 2019 17:51:36 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [NO ISSUE][COMP] Fix variable handling in Intersect operator
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3460 ) Change subject: [NO ISSUE][COMP] Fix variable handling in Intersect operator .. Patch Set 1: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3460 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4f436566e743d7e908a72bfa502c138ae7b99c6c Gerrit-Change-Number: 3460 Gerrit-PatchSet: 1 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Comment-Date: Wed, 26 Jun 2019 17:51:38 + Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][COMP] Stop reusing variables of Intersect operator
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3450 ) Change subject: [NO ISSUE][COMP] Stop reusing variables of Intersect operator .. Patch Set 1: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3450 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I551819b828450760797f6b29af02aad4e124646c Gerrit-Change-Number: 3450 Gerrit-PatchSet: 1 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Comment-Date: Fri, 21 Jun 2019 17:56:37 + Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE] Fix error message thrown by ATypeHierarchy
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3441 ) Change subject: [NO ISSUE] Fix error message thrown by ATypeHierarchy .. Patch Set 2: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3441 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I830e93056d18123610b933a0e36fd1b54c6e140b Gerrit-Change-Number: 3441 Gerrit-PatchSet: 2 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Anon. E. Moose Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Comment-Date: Tue, 18 Jun 2019 20:11:48 + Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][FUN] Allow frame in RATIO_TO_REPORT()
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3449 ) Change subject: [NO ISSUE][FUN] Allow frame in RATIO_TO_REPORT() .. Patch Set 1: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3449 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibc3686bebaf791aa84ef34d8fd143e53edc7856a Gerrit-Change-Number: 3449 Gerrit-PatchSet: 1 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Comment-Date: Tue, 18 Jun 2019 04:44:43 + Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][OTH] Use Request Stats When Printing Results on NC
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3444 ) Change subject: [NO ISSUE][OTH] Use Request Stats When Printing Results on NC .. Patch Set 2: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3444 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I54c3b3e72cfb0c2499a95bfc0f514dd5d6774293 Gerrit-Change-Number: 3444 Gerrit-PatchSet: 2 Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Hussain Towaileb Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Murtadha Hubail Gerrit-Comment-Date: Tue, 18 Jun 2019 04:43:39 + Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][COMP] Do not inline non-functional LET clauses
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3436 ) Change subject: [NO ISSUE][COMP] Do not inline non-functional LET clauses .. Patch Set 2: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3436 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3740745e2e8c8a0bb11aa7908e68c682dd9a5553 Gerrit-Change-Number: 3436 Gerrit-PatchSet: 2 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Comment-Date: Mon, 17 Jun 2019 21:31:48 + Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][FUN] Allow ORDER BY in RATIO_TO_REPORT()
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3442 ) Change subject: [NO ISSUE][FUN] Allow ORDER BY in RATIO_TO_REPORT() .. Patch Set 1: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3442 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I031b3eff1aee8be12f1943d39f5f1c640476e4e8 Gerrit-Change-Number: 3442 Gerrit-PatchSet: 1 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Comment-Date: Sat, 15 Jun 2019 20:28:33 + Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][FUN] Remove unused scalar aggregates
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3429 ) Change subject: [NO ISSUE][FUN] Remove unused scalar aggregates .. Patch Set 5: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3429 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2c1274876c884f16bcbee0e3f7aaba4d91b271ce Gerrit-Change-Number: 3429 Gerrit-PatchSet: 5 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Comment-Date: Tue, 11 Jun 2019 21:53:31 + Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2458][COMP] Fix min/max functions with group by
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3434 ) Change subject: [ASTERIXDB-2458][COMP] Fix min/max functions with group by .. Patch Set 2: Code-Review+1 -- To view, visit https://asterix-gerrit.ics.uci.edu/3434 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie0551b091b9adbbbd51158dbd36124a7184bdce0 Gerrit-Change-Number: 3434 Gerrit-PatchSet: 2 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Mon, 10 Jun 2019 21:52:45 + Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2458][COMP] Fix min/max functions with group by
Hello Anon. E. Moose (1000171), Till Westmann, Jenkins, Dmitry Lychagin, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3434 to look at the new patch set (#2). Change subject: [ASTERIXDB-2458][COMP] Fix min/max functions with group by .. [ASTERIXDB-2458][COMP] Fix min/max functions with group by - user model changes: no - storage format changes: no - interface changes: no Details: In a sort-group-by operator, an aggregate function has a merging aggregate function that will be used when merging the run files if they were generated. The merging aggregate function of the local min/max aggregate function should be different from the merging aggregate function of the global min/max. For local min/max, the merging aggregate function should propagate system_null if the aggregation is system_null. Also, the global min/max should handle finishPartial() different from finish(). finishPartial() should not output NULL if the aggregation is system_null since the aggregation is still going on. - added functions to be intermediate steps for local aggregation. - implemented logic for finishPartial() - added test cases for strict_min/max with group by Change-Id: Ie0551b091b9adbbbd51158dbd36124a7184bdce0 --- M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.01.ddl.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.02.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.31.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.32.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.33.ddl.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate/min_max_arrays/min_max_arrays.01.ddl.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate/min_max_arrays/min_max_arrays.02.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate/min_max_arrays/min_max_arrays.23.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate/min_max_arrays/min_max_arrays.24.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate/min_max_arrays/min_max_arrays.25.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate/min_max_arrays/min_max_arrays.26.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate/min_max_arrays/min_max_arrays.27.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate/min_max_arrays/min_max_arrays.28.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate/min_max_arrays/min_max_arrays.29.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate/min_max_arrays/min_max_arrays.30.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate/min_max_arrays/min_max_arrays.31.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate/min_max_arrays/min_max_arrays.32.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate/min_max_arrays/min_max_arrays.33.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate/min_max_arrays/min_max_arrays.34.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate/min_max_arrays/min_max_arrays.35.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate/min_max_arrays/min_max_arrays.36.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/min_max_arrays/min_max_arrays.31.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/min_max_arrays/min_max_arrays.32.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/min_max_arrays/min_max_arrays.23.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/min_max_arrays/min_max_arrays.24.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/min_max_arrays/min_max_arrays.25.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/min_max_arrays/min_max_arrays.26.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/min_max_arrays/min_max_arrays.27.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/min_max_arrays/min_max_arrays.28.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/min_max_arrays/min_max_arrays.29.adm A
Change in asterixdb[master]: [ASTERIXDB-2458][COMP] Fix min/max functions with group by
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3434 ) Change subject: [ASTERIXDB-2458][COMP] Fix min/max functions with group by .. Patch Set 1: Code-Review+1 -- To view, visit https://asterix-gerrit.ics.uci.edu/3434 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie0551b091b9adbbbd51158dbd36124a7184bdce0 Gerrit-Change-Number: 3434 Gerrit-PatchSet: 1 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Comment-Date: Mon, 10 Jun 2019 17:08:15 + Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2458][COMP] Fix min/max functions with group by
Ali Alsuliman has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/3434 Change subject: [ASTERIXDB-2458][COMP] Fix min/max functions with group by .. [ASTERIXDB-2458][COMP] Fix min/max functions with group by - user model changes: no - storage format changes: no - interface changes: no Details: In a sort-group-by operator, an aggregate function has a merging aggregate function that will be used when merging the run files if they were generated. The merging aggregate function of the local min/max aggregate function should be different from the merging aggregate function of the global min/max. For local min/max, the merging aggregate function should propagate system_null if the aggregation is system_null. Also, the global min/max should handle finishPartial() different from finish(). finishPartial() should not output NULL if the aggregation is system_null since the aggregation is still going on. - added functions to be intermediate steps for local aggregation. - implemented logic for finishPartial() Change-Id: Ie0551b091b9adbbbd51158dbd36124a7184bdce0 --- M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.01.ddl.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.02.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.31.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.32.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.33.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/min_max_arrays/min_max_arrays.31.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/min_max_arrays/min_max_arrays.32.adm M asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/BuiltinFunctions.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractMinMaxAggregateFunction.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/IntermediateMaxAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/IntermediateMinAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/IntermediateSqlMaxAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/IntermediateSqlMinAggregateDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/MinMaxAggregateFunction.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/SqlMinMaxAggregateFunction.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/functions/FunctionCollection.java M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/AbstractIntroduceCombinerRule.java 17 files changed, 12,878 insertions(+), 21 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/34/3434/1 -- To view, visit https://asterix-gerrit.ics.uci.edu/3434 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ie0551b091b9adbbbd51158dbd36124a7184bdce0 Gerrit-Change-Number: 3434 Gerrit-PatchSet: 1 Gerrit-Owner: Ali Alsuliman
Change in asterixdb[master]: [ASTERIXDB-2574][COMP] Fix min/max functions
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3427 ) Change subject: [ASTERIXDB-2574][COMP] Fix min/max functions .. Patch Set 2: (1 comment) https://asterix-gerrit.ics.uci.edu/#/c/3427/2/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/types/AbstractCollectionType.java File asterixdb/asterix-om/src/main/java/org/apache/asterix/om/types/AbstractCollectionType.java: https://asterix-gerrit.ics.uci.edu/#/c/3427/2/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/types/AbstractCollectionType.java@42 PS2, Line 42: // TODO(ali): this should be removed. itemType is already enforced to be NonNull > I think we need to investigate the intent of this method. May be we should agreed. -- To view, visit https://asterix-gerrit.ics.uci.edu/3427 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1231cfe558099d167bae0b2fa7fc4879b756baf0 Gerrit-Change-Number: 3427 Gerrit-PatchSet: 2 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Thu, 06 Jun 2019 21:24:52 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [NO ISSUE][SQLPP] Support SELECT LET
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3428 ) Change subject: [NO ISSUE][SQLPP] Support SELECT LET .. Patch Set 2: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3428 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1226dca83e756075608232c642851f646a9bee3b Gerrit-Change-Number: 3428 Gerrit-PatchSet: 2 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Thu, 06 Jun 2019 20:56:21 + Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2574][COMP] Fix min/max functions
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3427 ) Change subject: [ASTERIXDB-2574][COMP] Fix min/max functions .. Patch Set 2: (1 comment) https://asterix-gerrit.ics.uci.edu/#/c/3427/2/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarArrayAggAggregateDescriptor.java File asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarArrayAggAggregateDescriptor.java: https://asterix-gerrit.ics.uci.edu/#/c/3427/2/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarArrayAggAggregateDescriptor.java@47 PS2, Line 47: itemType); check this just to make sure. -- To view, visit https://asterix-gerrit.ics.uci.edu/3427 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1231cfe558099d167bae0b2fa7fc4879b756baf0 Gerrit-Change-Number: 3427 Gerrit-PatchSet: 2 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Thu, 06 Jun 2019 02:00:20 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [ASTERIXDB-2574][COMP] Fix min/max functions
Hello Anon. E. Moose (1000171), Till Westmann, Jenkins, Dmitry Lychagin, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3427 to look at the new patch set (#2). Change subject: [ASTERIXDB-2574][COMP] Fix min/max functions .. [ASTERIXDB-2574][COMP] Fix min/max functions - user model changes: no - storage format changes: no - interface changes: no This change is mainly for 2 things. The first thing is to not throw an exception when the type of the aggregated field is invalid for min/max (e.g. record or rectange) or min/max get incompatible data like string and int. The result in this case would be NULL. The second thing is to enable comparing ARRAYs correctly by using logical comparison. When a partition runs into type invalidity, it will output NULL. The global aggregator interprets NULL received from a partition as type invalidity and outputs NULL as the final result. Both SQL and SQL++ will do that. Special treatment is needed for scalar and distinct version of SQL since SQL min/max ignores NULL values and continue aggregation and the scalar and distinct version of SQL are normally setup as a global aggregator since they behave like the global aggregator in a two-step aggregation. Currently, there is only a local min and max functions. The other min/max functions are used for everything, the global function of two-step aggregation, and for scalar and distinct min/max. In order to differentiate, a global min/max functions are added that will be used for the two-step aggregation. Details: - fixed listify to open up elements when adding them to the collection and the collection item type is of type ANY and changed the type inferer of listify to enable that. - fixed AbstractCollectionType to make sure itemType is never null. - changed MinMaxAggTypeComputer to not throw an exception but return NULL for invalid types. - changed min/max descriptors to implement inferer to propagate the type of the field and pass that when getting a comparator. - switched min/max comparison to the logical comparison. - refactored method inequalityUndefined to be shared by logical comparison and min/max functions. - added global max/min functions to enable differentiating between scalar min/max, distinct min/max and two-step min/max (global & local). - code clean-up for LogicalScalarBinaryComparator; created two INSTANCES and re-used. Change-Id: I1231cfe558099d167bae0b2fa7fc4879b756baf0 --- A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.01.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.02.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.03.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.04.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.05.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.06.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.07.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.08.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.09.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.10.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.11.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.12.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.13.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.14.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.15.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.16.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.17.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.18.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.19.query.sqlpp A
Change in asterixdb[master]: [NO ISSUE][SQLPP] Optional field name in object constructor
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3424 ) Change subject: [NO ISSUE][SQLPP] Optional field name in object constructor .. Patch Set 3: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3424 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0dfba8f8137f88d934a5be8ffb5fbc1c1db0bb58 Gerrit-Change-Number: 3424 Gerrit-PatchSet: 3 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Wed, 05 Jun 2019 15:44:45 + Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][SQLPP] Optional field name in object constructor
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3424 ) Change subject: [NO ISSUE][SQLPP] Optional field name in object constructor .. Patch Set 3: (1 comment) https://asterix-gerrit.ics.uci.edu/#/c/3424/3/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/no_fieldname_constr/no_fieldname_constr.1.query.sqlpp File asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/no_fieldname_constr/no_fieldname_constr.1.query.sqlpp: https://asterix-gerrit.ics.uci.edu/#/c/3424/3/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/no_fieldname_constr/no_fieldname_constr.1.query.sqlpp@25 PS3, Line 25: from range(1, 2) x, range(3, 4) y it would be nice to have a test where there is one nested field access whose last identifier matches another field access. -- To view, visit https://asterix-gerrit.ics.uci.edu/3424 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0dfba8f8137f88d934a5be8ffb5fbc1c1db0bb58 Gerrit-Change-Number: 3424 Gerrit-PatchSet: 3 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Wed, 05 Jun 2019 15:41:56 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [NO ISSUE][FUN] Support FROM LAST in NTH_VALUE()
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3419 ) Change subject: [NO ISSUE][FUN] Support FROM LAST in NTH_VALUE() .. Patch Set 2: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3419 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iceac579bd5a3e651bcd7707e324148690e020cf5 Gerrit-Change-Number: 3419 Gerrit-PatchSet: 2 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Tue, 04 Jun 2019 06:39:25 + Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][SQLPP] Improve error message for unsupported EXPLAIN
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3420 ) Change subject: [NO ISSUE][SQLPP] Improve error message for unsupported EXPLAIN .. Patch Set 2: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3420 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idd7844ca3d9e3c2c0de0b070e4a76282cb96e278 Gerrit-Change-Number: 3420 Gerrit-PatchSet: 2 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Tue, 04 Jun 2019 03:29:09 + Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][FUN] Support IGNORE NULLS in window functions
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3407 ) Change subject: [NO ISSUE][FUN] Support IGNORE NULLS in window functions .. Patch Set 2: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3407 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I400c9c95bac9159efa17fca4e97047fd089eb931 Gerrit-Change-Number: 3407 Gerrit-PatchSet: 2 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Wed, 29 May 2019 20:00:32 + Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][FUN] Support IGNORE NULLS in window functions
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3407 ) Change subject: [NO ISSUE][FUN] Support IGNORE NULLS in window functions .. Patch Set 2: (1 comment) https://asterix-gerrit.ics.uci.edu/#/c/3407/2/asterixdb/asterix-common/src/main/resources/asx_errormsg/en.properties File asterixdb/asterix-common/src/main/resources/asx_errormsg/en.properties: https://asterix-gerrit.ics.uci.edu/#/c/3407/2/asterixdb/asterix-common/src/main/resources/asx_errormsg/en.properties@182 PS2, Line 182: 1104 = Invalid modifier %1$s for function %2$s let's check if we need to add a mapping. -- To view, visit https://asterix-gerrit.ics.uci.edu/3407 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I400c9c95bac9159efa17fca4e97047fd089eb931 Gerrit-Change-Number: 3407 Gerrit-PatchSet: 2 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Wed, 29 May 2019 20:00:30 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [NO ISSUE][CONF] Mark KeyStore Pass As Hidden Config
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3413 ) Change subject: [NO ISSUE][CONF] Mark KeyStore Pass As Hidden Config .. Patch Set 1: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3413 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1c27c3b229ca7d8dd44717c21cd096f0090a7278 Gerrit-Change-Number: 3413 Gerrit-PatchSet: 1 Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Wed, 29 May 2019 16:35:12 + Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2547][COMP] Disallow passing UNION tag to get comparator
Ali Alsuliman has submitted this change and it was merged. ( https://asterix-gerrit.ics.uci.edu/3404 ) Change subject: [ASTERIXDB-2547][COMP] Disallow passing UNION tag to get comparator .. [ASTERIXDB-2547][COMP] Disallow passing UNION tag to get comparator - user model changes: no - storage format changes: no - interface changes: no Details: UNION should not be allowed when getting a comparator by tag. The comparator provider returns a generic comparator with types ANY when UNION is passed as a tag. This could cause problems if the actual type is a complex type. UNION should not be allowed. Change-Id: Id8816a0dc5584f0a27410c512f3a44ccfc6c3151 Reviewed-on: https://asterix-gerrit.ics.uci.edu/3404 Contrib: Jenkins Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Dmitry Lychagin --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/LangExpressionToPlanTranslator.java M asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/util/RangeMapBuilder.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/nontagged/BinaryComparatorFactoryProvider.java 3 files changed, 23 insertions(+), 12 deletions(-) Approvals: Jenkins: Verified; ; Verified Anon. E. Moose (1000171): Dmitry Lychagin: Looks good to me, approved Objections: Jenkins: Violations found diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/LangExpressionToPlanTranslator.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/LangExpressionToPlanTranslator.java index b8bfffd..c6c8a2f 100644 --- a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/LangExpressionToPlanTranslator.java +++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/LangExpressionToPlanTranslator.java @@ -1252,7 +1252,7 @@ if (oc.getRangeMap() != null) { Iterator orderModifIter = oc.getModifierList().iterator(); boolean ascending = orderModifIter.next() == OrderModifier.ASC; -RangeMapBuilder.verifyRangeOrder(oc.getRangeMap(), ascending); +RangeMapBuilder.verifyRangeOrder(oc.getRangeMap(), ascending, sourceLoc); ord.getAnnotations().put(OperatorAnnotations.USE_STATIC_RANGE, oc.getRangeMap()); } return new Pair<>(ord, null); diff --git a/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/util/RangeMapBuilder.java b/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/util/RangeMapBuilder.java index c505c1c..a26c94d 100644 --- a/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/util/RangeMapBuilder.java +++ b/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/util/RangeMapBuilder.java @@ -23,6 +23,7 @@ import org.apache.asterix.common.exceptions.CompilationException; import org.apache.asterix.common.exceptions.ErrorCode; +import org.apache.asterix.common.exceptions.RuntimeDataException; import org.apache.asterix.formats.nontagged.BinaryComparatorFactoryProvider; import org.apache.asterix.formats.nontagged.SerializerDeserializerProvider; import org.apache.asterix.lang.common.base.Expression; @@ -50,6 +51,7 @@ import org.apache.hyracks.api.dataflow.value.IBinaryComparatorFactory; import org.apache.hyracks.api.dataflow.value.ISerializerDeserializer; import org.apache.hyracks.api.exceptions.HyracksDataException; +import org.apache.hyracks.api.exceptions.SourceLocation; import org.apache.hyracks.data.std.util.ArrayBackedValueStorage; import org.apache.hyracks.dataflow.common.data.partition.range.RangeMap; @@ -145,19 +147,25 @@ } } -public static void verifyRangeOrder(RangeMap rangeMap, boolean ascending) throws CompilationException { +public static void verifyRangeOrder(RangeMap rangeMap, boolean ascending, SourceLocation sourceLoc) +throws CompilationException { // TODO Add support for composite fields. int fieldIndex = 0; int fieldType = rangeMap.getTag(0, 0); BinaryComparatorFactoryProvider comparatorFactory = BinaryComparatorFactoryProvider.INSTANCE; -IBinaryComparatorFactory bcf = - comparatorFactory.getBinaryComparatorFactory(ATypeTag.VALUE_TYPE_MAPPING[fieldType], ascending); +IBinaryComparatorFactory bcf; +try { +bcf = comparatorFactory.getBinaryComparatorFactory(ATypeTag.VALUE_TYPE_MAPPING[fieldType], ascending); +} catch (RuntimeDataException e) { +throw new CompilationException(ErrorCode.COMPILATION_ERROR, sourceLoc, e.getMessage()); +} IBinaryComparator comparator = bcf.createBinaryComparator(); int c = 0; for (int split = 1; split < rangeMap.getSplitCount(); ++split) { if (fieldType != rangeMap.getTag(fieldIndex, split)) { -
Change in asterixdb[master]: [ASTERIXDB-2547][COMP] Disallow passing UNION tag to get comparator
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3404 ) Change subject: [ASTERIXDB-2547][COMP] Disallow passing UNION tag to get comparator .. Patch Set 2: (1 comment) https://asterix-gerrit.ics.uci.edu/#/c/3404/2/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/util/RangeMapBuilder.java File asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/util/RangeMapBuilder.java: https://asterix-gerrit.ics.uci.edu/#/c/3404/2/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/util/RangeMapBuilder.java@180 PS2, Line 180: " + " > Can we remove this string concatenation? Done -- To view, visit https://asterix-gerrit.ics.uci.edu/3404 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id8816a0dc5584f0a27410c512f3a44ccfc6c3151 Gerrit-Change-Number: 3404 Gerrit-PatchSet: 2 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Tue, 28 May 2019 03:07:37 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [ASTERIXDB-2547][COMP] Disallow passing UNION tag to get comparator
Hello Anon. E. Moose (1000171), Till Westmann, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3404 to look at the new patch set (#3). Change subject: [ASTERIXDB-2547][COMP] Disallow passing UNION tag to get comparator .. [ASTERIXDB-2547][COMP] Disallow passing UNION tag to get comparator - user model changes: no - storage format changes: no - interface changes: no Details: UNION should not be allowed when getting a comparator by tag. The comparator provider returns a generic comparator with types ANY when UNION is passed as a tag. This could cause problems if the actual type is a complex type. UNION should not be allowed. Change-Id: Id8816a0dc5584f0a27410c512f3a44ccfc6c3151 --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/LangExpressionToPlanTranslator.java M asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/util/RangeMapBuilder.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/nontagged/BinaryComparatorFactoryProvider.java 3 files changed, 23 insertions(+), 12 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/04/3404/3 -- To view, visit https://asterix-gerrit.ics.uci.edu/3404 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id8816a0dc5584f0a27410c512f3a44ccfc6c3151 Gerrit-Change-Number: 3404 Gerrit-PatchSet: 3 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann
Change in asterixdb[master]: [ASTERIXDB-2458][COMP] Fix InjectTypeCastForFunctionArgumentsRule
Ali Alsuliman has submitted this change and it was merged. ( https://asterix-gerrit.ics.uci.edu/3406 ) Change subject: [ASTERIXDB-2458][COMP] Fix InjectTypeCastForFunctionArgumentsRule .. [ASTERIXDB-2458][COMP] Fix InjectTypeCastForFunctionArgumentsRule - user model changes: no - storage format changes: no - interface changes: no Details: InjectTypeCastForFunctionArgumentsRule is for functions that can potentially return any of their arguments. switch and if_null(expr1, expr2, ...) are examples. All the arguments need to be casted (opened) to the type that the function will return which is the generalized type of all arguments. Some functions like if_null can determine the exact expression they will return, e.g. if_null(1, {"id": 3}) in which case the return type is always integer. The rule tries to cast th 2nd argument, the record, to integer and fails. In such cases, these functions do not need to cast their arguments. If the function determines its output type to be ANY, then all arguments need to be casted (opened). If the function determines its output to be a dervied type, then casting is also needed since that output type should be the generalized type of all arguments. Change-Id: I2fee234d883b59319e4ec4df58d61ecd498373fd Reviewed-on: https://asterix-gerrit.ics.uci.edu/3406 Contrib: Jenkins Sonar-Qube: Jenkins Tested-by: Jenkins Integration-Tests: Jenkins Reviewed-by: Dmitry Lychagin --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/InjectTypeCastForFunctionArgumentsRule.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/cast-ASTERIXDB-2458/cast-ASTERIXDB-2458.1.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/cast-ASTERIXDB-2458/cast-ASTERIXDB-2458.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml M asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/common/TypeResolverUtil.java 5 files changed, 64 insertions(+), 22 deletions(-) Approvals: Jenkins: Verified; No violations found; ; Verified Anon. E. Moose (1000171): Dmitry Lychagin: Looks good to me, approved diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/InjectTypeCastForFunctionArgumentsRule.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/InjectTypeCastForFunctionArgumentsRule.java index 3678826..77d63f1 100644 --- a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/InjectTypeCastForFunctionArgumentsRule.java +++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/InjectTypeCastForFunctionArgumentsRule.java @@ -24,12 +24,15 @@ import java.util.HashMap; import java.util.List; import java.util.Map; +import java.util.function.IntBinaryOperator; import java.util.function.IntPredicate; import org.apache.asterix.dataflow.data.common.TypeResolverUtil; import org.apache.asterix.lang.common.util.FunctionUtil; import org.apache.asterix.om.functions.BuiltinFunctions; import org.apache.asterix.om.typecomputer.base.TypeCastUtils; +import org.apache.asterix.om.typecomputer.impl.TypeComputeUtils; +import org.apache.asterix.om.types.ATypeTag; import org.apache.asterix.om.types.IAType; import org.apache.commons.lang3.mutable.Mutable; import org.apache.commons.lang3.mutable.MutableObject; @@ -103,39 +106,27 @@ } FunctionIdentifier funcId = func.getFunctionIdentifier(); if (funcId.equals(BuiltinFunctions.SWITCH_CASE)) { -rewritten |= rewriteSwitchCase(op, func, context); +rewritten |= rewriteFunction(op, func, null, context, 2, +InjectTypeCastForFunctionArgumentsRule::switchIncrement); } else if (FUN_TO_ARG_CHECKER.containsKey(funcId)) { -rewritten |= rewriteFunction(op, func, FUN_TO_ARG_CHECKER.get(funcId), context); -} -return rewritten; -} - -// Injects casts that cast types for different "THEN" and "ELSE" branches. -private boolean rewriteSwitchCase(ILogicalOperator op, AbstractFunctionCallExpression func, -IOptimizationContext context) throws AlgebricksException { -IVariableTypeEnvironment env = op.computeInputTypeEnvironment(context); -IAType producedType = (IAType) env.getType(func); -List> argRefs = func.getArguments(); -int argSize = argRefs.size(); -boolean rewritten = false; -for (int argIndex = 2; argIndex < argSize; argIndex += (argIndex + 2 == argSize) ? 1 : 2) { -Mutable argRef = argRefs.get(argIndex); -if (rewriteFunctionArgument(argRef, producedType, env)) { -rewritten = true; -} +rewritten |= rewriteFunction(op, func, FUN_TO_ARG_CHECKER.get(funcId), context, 0, +InjectTypeCastForFunctionArgumentsRule::increment); }
Change in asterixdb[master]: [ASTERIXDB-2547][COMP] Disallow passing UNION tag to get comparator
Hello Anon. E. Moose (1000171), Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3404 to look at the new patch set (#2). Change subject: [ASTERIXDB-2547][COMP] Disallow passing UNION tag to get comparator .. [ASTERIXDB-2547][COMP] Disallow passing UNION tag to get comparator - user model changes: no - storage format changes: no - interface changes: no Details: UNION should not be allowed when getting a comparator by tag. The comparator provider returns a generic comparator with types ANY when UNION is passed as a tag. This could cause problems if the actual type is a complex type. UNION should not be allowed. Change-Id: Id8816a0dc5584f0a27410c512f3a44ccfc6c3151 --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/LangExpressionToPlanTranslator.java M asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/util/RangeMapBuilder.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/nontagged/BinaryComparatorFactoryProvider.java 3 files changed, 23 insertions(+), 12 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/04/3404/2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3404 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id8816a0dc5584f0a27410c512f3a44ccfc6c3151 Gerrit-Change-Number: 3404 Gerrit-PatchSet: 2 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2458][COMP] Fix InjectTypeCastForFunctionArgumentsRule
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3406 ) Change subject: [ASTERIXDB-2458][COMP] Fix InjectTypeCastForFunctionArgumentsRule .. Patch Set 3: (1 comment) https://asterix-gerrit.ics.uci.edu/#/c/3406/2/asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/common/TypeResolverUtil.java File asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/common/TypeResolverUtil.java: https://asterix-gerrit.ics.uci.edu/#/c/3406/2/asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/common/TypeResolverUtil.java@98 PS2, Line 98: // Generalizes two input types. > This method's name can be confusing. Primitive types are in general castabl Done -- To view, visit https://asterix-gerrit.ics.uci.edu/3406 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2fee234d883b59319e4ec4df58d61ecd498373fd Gerrit-Change-Number: 3406 Gerrit-PatchSet: 3 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Fri, 24 May 2019 19:02:02 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [ASTERIXDB-2458][COMP] Fix InjectTypeCastForFunctionArgumentsRule
Hello Anon. E. Moose (1000171), Till Westmann, Jenkins, Dmitry Lychagin, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3406 to look at the new patch set (#3). Change subject: [ASTERIXDB-2458][COMP] Fix InjectTypeCastForFunctionArgumentsRule .. [ASTERIXDB-2458][COMP] Fix InjectTypeCastForFunctionArgumentsRule - user model changes: no - storage format changes: no - interface changes: no Details: InjectTypeCastForFunctionArgumentsRule is for functions that can potentially return any of their arguments. switch and if_null(expr1, expr2, ...) are examples. All the arguments need to be casted (opened) to the type that the function will return which is the generalized type of all arguments. Some functions like if_null can determine the exact expression they will return, e.g. if_null(1, {"id": 3}) in which case the return type is always integer. The rule tries to cast th 2nd argument, the record, to integer and fails. In such cases, these functions do not need to cast their arguments. If the function determines its output type to be ANY, then all arguments need to be casted (opened). If the function determines its output to be a dervied type, then casting is also needed since that output type should be the generalized type of all arguments. Change-Id: I2fee234d883b59319e4ec4df58d61ecd498373fd --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/InjectTypeCastForFunctionArgumentsRule.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/cast-ASTERIXDB-2458/cast-ASTERIXDB-2458.1.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/cast-ASTERIXDB-2458/cast-ASTERIXDB-2458.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml M asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/common/TypeResolverUtil.java 5 files changed, 64 insertions(+), 22 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/06/3406/3 -- To view, visit https://asterix-gerrit.ics.uci.edu/3406 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I2fee234d883b59319e4ec4df58d61ecd498373fd Gerrit-Change-Number: 3406 Gerrit-PatchSet: 3 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann
Change in asterixdb[master]: [ASTERIXDB-2458][COMP] Fix InjectTypeCastForFunctionArgumentsRule
Hello Anon. E. Moose (1000171), Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3406 to look at the new patch set (#2). Change subject: [ASTERIXDB-2458][COMP] Fix InjectTypeCastForFunctionArgumentsRule .. [ASTERIXDB-2458][COMP] Fix InjectTypeCastForFunctionArgumentsRule - user model changes: no - storage format changes: no - interface changes: no Details: InjectTypeCastForFunctionArgumentsRule is for functions that can potentially return any of their arguments. switch and if_null(expr1, expr2, ...) are examples. All the arguments need to be casted (opened) to the type that the function will return which is the generalized type of all arguments. Some functions like if_null can determine the exact expression they will return, e.g. if_null(1, {"id": 3}) in which case the return type is always integer. The rule tries to cast th 2nd argument, the record, to integer and fails. In such cases, these functions do not need to cast their arguments. If the function determines its output type to be ANY, then all arguments need to be casted (opened). If the function determines its output to be a dervied type, then casting is also needed since that output type should be the generalized type of all arguments. Change-Id: I2fee234d883b59319e4ec4df58d61ecd498373fd --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/InjectTypeCastForFunctionArgumentsRule.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/cast-ASTERIXDB-2458/cast-ASTERIXDB-2458.1.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/cast-ASTERIXDB-2458/cast-ASTERIXDB-2458.1.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml M asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/common/TypeResolverUtil.java 5 files changed, 71 insertions(+), 22 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/06/3406/2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3406 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I2fee234d883b59319e4ec4df58d61ecd498373fd Gerrit-Change-Number: 3406 Gerrit-PatchSet: 2 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2547][COMP] Disallow passing UNION tag to get comparator
Ali Alsuliman has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/3404 Change subject: [ASTERIXDB-2547][COMP] Disallow passing UNION tag to get comparator .. [ASTERIXDB-2547][COMP] Disallow passing UNION tag to get comparator - user model changes: no - storage format changes: no - interface changes: no Details: UNION should not be allowed when getting a comparator by tag. The comparator provider returns a generic comparator with types ANY when UNION is passed as a tag. This could cause problems if the actual type is a complex type. UNION should not be allowed. Change-Id: Id8816a0dc5584f0a27410c512f3a44ccfc6c3151 --- M asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/nontagged/BinaryComparatorFactoryProvider.java 1 file changed, 3 insertions(+), 3 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/04/3404/1 diff --git a/asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/nontagged/BinaryComparatorFactoryProvider.java b/asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/nontagged/BinaryComparatorFactoryProvider.java index 396bf3b..5f59a41 100644 --- a/asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/nontagged/BinaryComparatorFactoryProvider.java +++ b/asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/nontagged/BinaryComparatorFactoryProvider.java @@ -35,6 +35,7 @@ import org.apache.asterix.om.types.ATypeTag; import org.apache.asterix.om.types.BuiltinType; import org.apache.asterix.om.types.IAType; +import org.apache.hyracks.algebricks.common.exceptions.NotImplementedException; import org.apache.hyracks.algebricks.data.IBinaryComparatorFactoryProvider; import org.apache.hyracks.api.dataflow.value.IBinaryComparatorFactory; import org.apache.hyracks.data.std.accessors.BooleanBinaryComparatorFactory; @@ -115,10 +116,9 @@ public IBinaryComparatorFactory getBinaryComparatorFactory(ATypeTag type, boolean ascending) { switch (type) { case ANY: -case UNION: -// i think UNION shouldn't be allowed. the actual type could be closed array or record. ANY would fail. -// we could do smth better for nullable fields return createGenericBinaryComparatorFactory(BuiltinType.ANY, BuiltinType.ANY, ascending); +case UNION: +throw new NotImplementedException("No comparator for type UNION"); case NULL: case MISSING: return new AnyBinaryComparatorFactory(); -- To view, visit https://asterix-gerrit.ics.uci.edu/3404 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Id8816a0dc5584f0a27410c512f3a44ccfc6c3151 Gerrit-Change-Number: 3404 Gerrit-PatchSet: 1 Gerrit-Owner: Ali Alsuliman
Change in asterixdb[master]: [ASTERIXDB-2288][COMP] Fix field access with UNION
Ali Alsuliman has submitted this change and it was merged. ( https://asterix-gerrit.ics.uci.edu/3400 ) Change subject: [ASTERIXDB-2288][COMP] Fix field access with UNION .. [ASTERIXDB-2288][COMP] Fix field access with UNION - user model changes: no - storage format changes: no - interface changes: no Details: For UNION, if there is an assign operator above UNION, the optimizer will move the assign operator under UNION into both branches. If the assign operator has field access by index, the index is based on the output record of the union operator which could be different from the field index in left and right branches. The fix is to map the index to the correct index of the input operator where the assign is copied to. Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 Reviewed-on: https://asterix-gerrit.ics.uci.edu/3400 Contrib: Jenkins Tested-by: Jenkins Integration-Tests: Jenkins Reviewed-by: Dmitry Lychagin --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java A asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/AsterixPushAssignBelowUnionAllRule.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.7.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/PushAssignBelowUnionAllRule.java 15 files changed, 397 insertions(+), 21 deletions(-) Approvals: Jenkins: Verified; ; Verified Anon. E. Moose (1000171): Dmitry Lychagin: Looks good to me, approved Objections: Jenkins: Violations found diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java index e4c35e4..f9aecab 100644 --- a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java +++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java @@ -30,6 +30,7 @@ import org.apache.asterix.optimizer.rules.AsterixExtractFunctionsFromJoinConditionRule; import org.apache.asterix.optimizer.rules.AsterixInlineVariablesRule; import org.apache.asterix.optimizer.rules.AsterixIntroduceGroupByCombinerRule; +import org.apache.asterix.optimizer.rules.AsterixPushAssignBelowUnionAllRule; import org.apache.asterix.optimizer.rules.ByNameToByIndexFieldAccessRule; import org.apache.asterix.optimizer.rules.CancelUnnestWithNestedListifyRule; import org.apache.asterix.optimizer.rules.CheckFilterExpressionTypeRule; @@ -117,7 +118,6 @@ import org.apache.hyracks.algebricks.rewriter.rules.IntroduceProjectsRule; import org.apache.hyracks.algebricks.rewriter.rules.IsolateHyracksOperatorsRule; import org.apache.hyracks.algebricks.rewriter.rules.PullSelectOutOfEqJoin; -import org.apache.hyracks.algebricks.rewriter.rules.PushAssignBelowUnionAllRule; import org.apache.hyracks.algebricks.rewriter.rules.PushGroupByIntoSortRule; import org.apache.hyracks.algebricks.rewriter.rules.PushMapOperatorDownThroughProductRule; import org.apache.hyracks.algebricks.rewriter.rules.PushNestedOrderByUnderPreSortedGroupByRule; @@ -330,7 +330,7 @@ public static final List
Change in asterixdb[master]: [ASTERIXDB-2288][COMP] Fix field access with UNION
Hello Anon. E. Moose (1000171), Till Westmann, Jenkins, Dmitry Lychagin, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3400 to look at the new patch set (#10). Change subject: [ASTERIXDB-2288][COMP] Fix field access with UNION .. [ASTERIXDB-2288][COMP] Fix field access with UNION - user model changes: no - storage format changes: no - interface changes: no Details: For UNION, if there is an assign operator above UNION, the optimizer will move the assign operator under UNION into both branches. If the assign operator has field access by index, the index is based on the output record of the union operator which could be different from the field index in left and right branches. The fix is to map the index to the correct index of the input operator where the assign is copied to. Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java A asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/AsterixPushAssignBelowUnionAllRule.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.7.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/PushAssignBelowUnionAllRule.java 15 files changed, 397 insertions(+), 21 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/00/3400/10 -- To view, visit https://asterix-gerrit.ics.uci.edu/3400 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 Gerrit-Change-Number: 3400 Gerrit-PatchSet: 10 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann
Change in asterixdb[master]: [ASTERIXDB-2288][COMP] Fix field access with UNION
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3400 ) Change subject: [ASTERIXDB-2288][COMP] Fix field access with UNION .. Patch Set 9: (2 comments) https://asterix-gerrit.ics.uci.edu/#/c/3400/8/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushAssignBelowUnionAllRule.java File asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushAssignBelowUnionAllRule.java: https://asterix-gerrit.ics.uci.edu/#/c/3400/8/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushAssignBelowUnionAllRule.java@196 PS8, Line 196: > I'd move this 'if' outside of this method into modifyExpression loop. So li Done. a bit different now with the new logic. https://asterix-gerrit.ics.uci.edu/#/c/3400/8/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushAssignBelowUnionAllRule.java@220 PS8, Line 220: > what if it returns -1 (not found)? hmm. i don't think the assign can be pushed in this case (even though it should not happen). i changed the logic now to not push the assign. -- To view, visit https://asterix-gerrit.ics.uci.edu/3400 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 Gerrit-Change-Number: 3400 Gerrit-PatchSet: 9 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Tue, 21 May 2019 01:43:09 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [ASTERIXDB-2288][COMP] Fix field access with UNION
Hello Anon. E. Moose (1000171), Till Westmann, Jenkins, Dmitry Lychagin, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3400 to look at the new patch set (#9). Change subject: [ASTERIXDB-2288][COMP] Fix field access with UNION .. [ASTERIXDB-2288][COMP] Fix field access with UNION - user model changes: no - storage format changes: no - interface changes: no Details: For UNION, if there is an assign operator above UNION, the optimizer will move the assign operator under UNION into both branches. If the assign operator has field access by index, the index is based on the output record of the union operator which could be different from the field index in left and right branches. The fix is to map the index to the correct index of the input operator where the assign is copied to. Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java A asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/AsterixPushAssignBelowUnionAllRule.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.7.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/PushAssignBelowUnionAllRule.java 15 files changed, 396 insertions(+), 21 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/00/3400/9 -- To view, visit https://asterix-gerrit.ics.uci.edu/3400 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 Gerrit-Change-Number: 3400 Gerrit-PatchSet: 9 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann
Change in asterixdb[master]: [ASTERIXDB-2288][COMP] Fix field access with UNION
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3400 ) Change subject: [ASTERIXDB-2288][COMP] Fix field access with UNION .. Patch Set 8: never mind the multiple patches. I was fighting gerrit to convince it that PushAssignBelowUnionAllRule is just renamed and not a new file. -- To view, visit https://asterix-gerrit.ics.uci.edu/3400 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 Gerrit-Change-Number: 3400 Gerrit-PatchSet: 8 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Fri, 17 May 2019 01:33:38 + Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2289][COMP] Fix field access with CASE
Ali Alsuliman has submitted this change and it was merged. ( https://asterix-gerrit.ics.uci.edu/3399 ) Change subject: [ASTERIXDB-2289][COMP] Fix field access with CASE .. [ASTERIXDB-2289][COMP] Fix field access with CASE - user model changes: no - storage format changes: no - interface changes: no Details: This patch fixes field access in the presense CASE and JOIN. This is a scenario where push-down-field-access rule throws an exception if the field access has potentially two sources and it could not push down the field access to left or right branch. Don't throw an exception and just return false (i.e. field access was not pushed) instead of throwing an exception. Change-Id: I911e4e9018c15e8f226e46fa610d222eb2301fcd Reviewed-on: https://asterix-gerrit.ics.uci.edu/3399 Contrib: Jenkins Tested-by: Jenkins Integration-Tests: Jenkins Reviewed-by: Dmitry Lychagin --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushFieldAccessRule.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access-ASTERIXDB-2289/field_access-ASTERIXDB-2289.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access-ASTERIXDB-2289/field_access-ASTERIXDB-2289.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access-ASTERIXDB-2289/field_access-ASTERIXDB-2289.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access-ASTERIXDB-2289/field_access-ASTERIXDB-2289.4.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access-ASTERIXDB-2289/field_access-ASTERIXDB-2289.3.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/util/OperatorPropertiesUtil.java 8 files changed, 162 insertions(+), 24 deletions(-) Approvals: Jenkins: Verified; ; Verified Dmitry Lychagin: Looks good to me, approved Objections: Jenkins: Violations found Anon. E. Moose (1000171): Violations found diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushFieldAccessRule.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushFieldAccessRule.java index 421b00d..c82aa33 100644 --- a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushFieldAccessRule.java +++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushFieldAccessRule.java @@ -84,6 +84,9 @@ if (op.getOperatorTag() != LogicalOperatorTag.ASSIGN) { return false; } +if (!OperatorPropertiesUtil.isMovable(op)) { +return false; +} AssignOperator access = (AssignOperator) op; ILogicalExpression expr = getFirstExpr(access); String finalAnnot; @@ -196,17 +199,17 @@ pushDownFieldAccessRec(opRef2, context, finalAnnot); return true; } -List usedInAccess = new LinkedList<>(); +HashSet usedInAccess = new HashSet<>(); VariableUtilities.getUsedVariables(assignOp, usedInAccess); -List produced2 = new LinkedList<>(); +HashSet produced2 = new HashSet<>(); if (inputOp.getOperatorTag() == LogicalOperatorTag.GROUP) { VariableUtilities.getLiveVariables(inputOp, produced2); } else { VariableUtilities.getProducedVariables(inputOp, produced2); } boolean pushItDown = false; -List inter = new ArrayList<>(usedInAccess); +HashSet inter = new HashSet<>(usedInAccess); if (inter.isEmpty()) { // ground value return false; } @@ -234,8 +237,7 @@ LogicalVariable oldVar = assignOp.getVariables().get(0); VariableReferenceExpression v2Ref = new VariableReferenceExpression(v2); v2Ref.setSourceLocation(g.getSourceLocation()); -g.getDecorList().add(new Pair>(oldVar, -new MutableObject(v2Ref))); +g.getDecorList().add(new Pair<>(oldVar, new MutableObject<>(v2Ref))); changed = true; assignOp.getVariables().set(0, v2); VariableUtilities.substituteVariables(assignOp, m.first, m.second, context); @@ -281,8 +283,7 @@ } } } -throw new CompilationException(ErrorCode.COMPILATION_ERROR, assignOp.getSourceLocation(), -"Field access " + assignOp.getExpressions().get(0).getValue() + " doesn't correspond to any input"); +return false; } else { // check if the accessed field is one of the partitioning key fields. If yes, we can equate the 2
Change in asterixdb[master]: [ASTERIXDB-2288][COMP] Fix field access with UNION
Hello Anon. E. Moose (1000171), Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3400 to look at the new patch set (#8). Change subject: [ASTERIXDB-2288][COMP] Fix field access with UNION .. [ASTERIXDB-2288][COMP] Fix field access with UNION - user model changes: no - storage format changes: no - interface changes: no Details: For UNION, if there is an assign operator above UNION, the optimizer will move the assign operator under UNION into both branches. If the assign operator has field access by index, the index is based on the output record of the union operator which could be different from the field index in left and right branches. The fix is to map the index to the correct index of the input operator where the assign is copied to. Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java R asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushAssignBelowUnionAllRule.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.7.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml 14 files changed, 334 insertions(+), 8 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/00/3400/8 -- To view, visit https://asterix-gerrit.ics.uci.edu/3400 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 Gerrit-Change-Number: 3400 Gerrit-PatchSet: 8 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2289][COMP] Fix field access with CASE
Hello Anon. E. Moose (1000171), Till Westmann, Jenkins, Michael Blow, Dmitry Lychagin, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3399 to look at the new patch set (#4). Change subject: [ASTERIXDB-2289][COMP] Fix field access with CASE .. [ASTERIXDB-2289][COMP] Fix field access with CASE - user model changes: no - storage format changes: no - interface changes: no Details: This patch fixes field access in the presense CASE and JOIN. This is a scenario where push-down-field-access rule throws an exception if the field access has potentially two sources and it could not push down the field access to left or right branch. Don't throw an exception and just return false (i.e. field access was not pushed) instead of throwing an exception. Change-Id: I911e4e9018c15e8f226e46fa610d222eb2301fcd --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushFieldAccessRule.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access-ASTERIXDB-2289/field_access-ASTERIXDB-2289.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access-ASTERIXDB-2289/field_access-ASTERIXDB-2289.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access-ASTERIXDB-2289/field_access-ASTERIXDB-2289.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access-ASTERIXDB-2289/field_access-ASTERIXDB-2289.4.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access-ASTERIXDB-2289/field_access-ASTERIXDB-2289.3.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/util/OperatorPropertiesUtil.java 8 files changed, 162 insertions(+), 24 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/99/3399/4 -- To view, visit https://asterix-gerrit.ics.uci.edu/3399 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I911e4e9018c15e8f226e46fa610d222eb2301fcd Gerrit-Change-Number: 3399 Gerrit-PatchSet: 4 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Till Westmann
Change in asterixdb[master]: [ASTERIXDB-2288][COMP] Fix field access with UNION
Hello Anon. E. Moose (1000171), Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3400 to look at the new patch set (#7). Change subject: [ASTERIXDB-2288][COMP] Fix field access with UNION .. [ASTERIXDB-2288][COMP] Fix field access with UNION - user model changes: no - storage format changes: no - interface changes: no Details: For UNION, if there is an assign operator above UNION, the optimizer will move the assign operator under UNION into both branches. If the assign operator has field access by index, the index is based on the output record of the union operator which could be different from the field index in left and right branches. The fix is to convert the index into a named field, and then move the assign operator. The optimizer will fire by-name-to-by-index rule after this and replace the name with the correct index for each record of left and right branch. by-name-to-by-index rule is already a required rule to be fired after push-assign-under-union rule. Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java R asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushAssignBelowUnionAllRule.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.7.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml 14 files changed, 345 insertions(+), 7 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/00/3400/7 -- To view, visit https://asterix-gerrit.ics.uci.edu/3400 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 Gerrit-Change-Number: 3400 Gerrit-PatchSet: 7 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2288][COMP] Fix field access with UNION
Ali Alsuliman has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/3400 Change subject: [ASTERIXDB-2288][COMP] Fix field access with UNION .. [ASTERIXDB-2288][COMP] Fix field access with UNION - user model changes: no - storage format changes: no - interface changes: no Details: For UNION, if there is an assign operator above UNION, the optimizer will move the assign operator under UNION into both branches. If the assign operator has field access by index, the index is based on the output record of the union operator which could be different from the field index in left and right branches. The fix is to convert the index into a named field, and then move the assign operator. The optimizer will fire by-name-to-by-index rule after this and replace the name with the correct index for each record of left and right branch. by-name-to-by-index rule is already a required rule to be fired after push-assign-under-union rule. Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java A asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushAssignBelowUnionAllRule.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.7.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml D hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/PushAssignBelowUnionAllRule.java 15 files changed, 515 insertions(+), 171 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/00/3400/1 diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java index e4c35e4..9d5fc2f 100644 --- a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java +++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java @@ -117,7 +117,7 @@ import org.apache.hyracks.algebricks.rewriter.rules.IntroduceProjectsRule; import org.apache.hyracks.algebricks.rewriter.rules.IsolateHyracksOperatorsRule; import org.apache.hyracks.algebricks.rewriter.rules.PullSelectOutOfEqJoin; -import org.apache.hyracks.algebricks.rewriter.rules.PushAssignBelowUnionAllRule; +import org.apache.asterix.optimizer.rules.PushAssignBelowUnionAllRule; import org.apache.hyracks.algebricks.rewriter.rules.PushGroupByIntoSortRule; import org.apache.hyracks.algebricks.rewriter.rules.PushMapOperatorDownThroughProductRule; import org.apache.hyracks.algebricks.rewriter.rules.PushNestedOrderByUnderPreSortedGroupByRule; diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushAssignBelowUnionAllRule.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushAssignBelowUnionAllRule.java new file mode 100644 index 000..552b691 --- /dev/null +++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushAssignBelowUnionAllRule.java @@ -0,0 +1,245 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with
Change in asterixdb[master]: [ASTERIXDB-2288][COMP] Fix field access with UNION
Hello Anon. E. Moose (1000171), Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3400 to look at the new patch set (#6). Change subject: [ASTERIXDB-2288][COMP] Fix field access with UNION .. [ASTERIXDB-2288][COMP] Fix field access with UNION - user model changes: no - storage format changes: no - interface changes: no Details: For UNION, if there is an assign operator above UNION, the optimizer will move the assign operator under UNION into both branches. If the assign operator has field access by index, the index is based on the output record of the union operator which could be different from the field index in left and right branches. The fix is to convert the index into a named field, and then move the assign operator. The optimizer will fire by-name-to-by-index rule after this and replace the name with the correct index for each record of left and right branch. by-name-to-by-index rule is already a required rule to be fired after push-assign-under-union rule. Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java R asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushAssignBelowUnionAllRule.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.7.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml 14 files changed, 345 insertions(+), 7 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/00/3400/6 -- To view, visit https://asterix-gerrit.ics.uci.edu/3400 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 Gerrit-Change-Number: 3400 Gerrit-PatchSet: 6 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2288][COMP] Fix field access with UNION
Hello Anon. E. Moose (1000171), Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3400 to look at the new patch set (#5). Change subject: [ASTERIXDB-2288][COMP] Fix field access with UNION .. [ASTERIXDB-2288][COMP] Fix field access with UNION - user model changes: no - storage format changes: no - interface changes: no Details: For UNION, if there is an assign operator above UNION, the optimizer will move the assign operator under UNION into both branches. If the assign operator has field access by index, the index is based on the output record of the union operator which could be different from the field index in left and right branches. The fix is to convert the index into a named field, and then move the assign operator. The optimizer will fire by-name-to-by-index rule after this and replace the name with the correct index for each record of left and right branch. by-name-to-by-index rule is already a required rule to be fired after push-assign-under-union rule. Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java R asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushAssignBelowUnionAllRule.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.7.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml 14 files changed, 271 insertions(+), 2 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/00/3400/5 -- To view, visit https://asterix-gerrit.ics.uci.edu/3400 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 Gerrit-Change-Number: 3400 Gerrit-PatchSet: 5 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2288][COMP] Fix field access with UNION
Hello Anon. E. Moose (1000171), Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3400 to look at the new patch set (#3). Change subject: [ASTERIXDB-2288][COMP] Fix field access with UNION .. [ASTERIXDB-2288][COMP] Fix field access with UNION - user model changes: no - storage format changes: no - interface changes: no Details: For UNION, if there is an assign operator above UNION, the optimizer will move the assign operator under UNION into both branches. If the assign operator has field access by index, the index is based on the output record of the union operator which could be different from the field index in left and right branches. The fix is to convert the index into a named field, and then move the assign operator. The optimizer will fire by-name-to-by-index rule after this and replace the name with the correct index for each record of left and right branch. by-name-to-by-index rule is already a required rule to be fired after push-assign-under-union rule. Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 --- A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.7.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml 12 files changed, 269 insertions(+), 0 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/00/3400/3 -- To view, visit https://asterix-gerrit.ics.uci.edu/3400 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 Gerrit-Change-Number: 3400 Gerrit-PatchSet: 3 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2288][COMP] Fix field access with UNION
Hello Anon. E. Moose (1000171), Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3400 to look at the new patch set (#2). Change subject: [ASTERIXDB-2288][COMP] Fix field access with UNION .. [ASTERIXDB-2288][COMP] Fix field access with UNION - user model changes: no - storage format changes: no - interface changes: no Details: For UNION, if there is an assign operator above UNION, the optimizer will move the assign operator under UNION into both branches. If the assign operator has field access by index, the index is based on the output record of the union operator which could be different from the field index in left and right branches. The fix is to convert the index into a named field, and then move the assign operator. The optimizer will fire by-name-to-by-index rule after this and replace the name with the correct index for each record of left and right branch. by-name-to-by-index rule is already a required rule to be fired after push-assign-under-union rule. Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java A asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushAssignBelowUnionAllRule.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.7.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml D hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/PushAssignBelowUnionAllRule.java 15 files changed, 515 insertions(+), 171 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/00/3400/2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3400 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 Gerrit-Change-Number: 3400 Gerrit-PatchSet: 2 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2289][COMP] Fix field access with CASE
Hello Anon. E. Moose (1000171), Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3399 to look at the new patch set (#3). Change subject: [ASTERIXDB-2289][COMP] Fix field access with CASE .. [ASTERIXDB-2289][COMP] Fix field access with CASE - user model changes: no - storage format changes: no - interface changes: no Details: This patch fixes field access in the presense CASE and JOIN. This is a scenario where push-down-field-access rule throws an exception if the field access has potentially two sources and it could not push down the field access to left or right branch. The exception thinks that the field access has no source when in reality it has but there were two potential sources because of CASE. Don't throw an exception if the field access is a valid one and return false (i.e. field access was not push) instead of throwing an exception. Change-Id: I911e4e9018c15e8f226e46fa610d222eb2301fcd --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushFieldAccessRule.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access-ASTERIXDB-2289/field_access-ASTERIXDB-2289.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access-ASTERIXDB-2289/field_access-ASTERIXDB-2289.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access-ASTERIXDB-2289/field_access-ASTERIXDB-2289.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access-ASTERIXDB-2289/field_access-ASTERIXDB-2289.3.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml 6 files changed, 143 insertions(+), 3 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/99/3399/3 -- To view, visit https://asterix-gerrit.ics.uci.edu/3399 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I911e4e9018c15e8f226e46fa610d222eb2301fcd Gerrit-Change-Number: 3399 Gerrit-PatchSet: 3 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2289][COMP] Fix field access with CASE
Hello Anon. E. Moose (1000171), Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3399 to look at the new patch set (#2). Change subject: [ASTERIXDB-2289][COMP] Fix field access with CASE .. [ASTERIXDB-2289][COMP] Fix field access with CASE - user model changes: no - storage format changes: no - interface changes: no Details: This patch fixes field access in the presense CASE and JOIN. This is a scenario where push-down-field-access rule throws an exception if the field access has potentially two sources and it could not push down the field access to left or right branch. The exception thinks that the field access has no source when in reality it has but there were two potential sources because of CASE. Don't throw an exception if the field access is a valid one and return false (i.e. field access was not push) instead of throwing an exception. Change-Id: I911e4e9018c15e8f226e46fa610d222eb2301fcd --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushFieldAccessRule.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access-ASTERIXDB-2289/field_access-ASTERIXDB-2289.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access-ASTERIXDB-2289/field_access-ASTERIXDB-2289.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access-ASTERIXDB-2289/field_access-ASTERIXDB-2289.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access-ASTERIXDB-2289/field_access-ASTERIXDB-2289.3.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml 6 files changed, 143 insertions(+), 3 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/99/3399/2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3399 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I911e4e9018c15e8f226e46fa610d222eb2301fcd Gerrit-Change-Number: 3399 Gerrit-PatchSet: 2 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2289][COMP] Fix field access with UNION and CASE
Ali Alsuliman has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/3399 Change subject: [ASTERIXDB-2289][COMP] Fix field access with UNION and CASE .. [ASTERIXDB-2289][COMP] Fix field access with UNION and CASE - user model changes: no - storage format changes: no - interface changes: no Details: This patch fixes field access in the presense of UNION and CASE. For CASE scenario, push-down-field-access rule throws an exception if the field access has potentially two sources and it could not push down the field access to left or right branch. The exception thinks that the field access has no source when in reality it has but there were two potential sources because of CASE. Don't throw an exception if the field access is a valid one and return false (i.e. field access was not push) instead of throwing an exception. For UNION, if there is an assign operator above UNION, the optimizer will move the assign operator under UNION into both branches. If the assign operator has field access by index, the index is based on the output record of the union operator which could be different from the field index in left and right branches. The fix is to convert the index into a named field, and then move the assign operator. The optimizer will fire by-name-to-by-index rule after this and replace the name with the correct index for each record of left and right branch. by-name-to-by-index rule is already a required rule to be fired after push-assign-under-union rule. Change-Id: I911e4e9018c15e8f226e46fa610d222eb2301fcd --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java A asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushAssignBelowUnionAllRule.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushFieldAccessRule.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access/field_access.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access/field_access.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access/field_access.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access/field_access.4.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access/field_access.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access/field_access.6.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access/field_access.7.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access/field_access.3.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access/field_access.4.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access/field_access.5.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access/field_access.6.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access/field_access.7.adm D hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/PushAssignBelowUnionAllRule.java 16 files changed, 537 insertions(+), 174 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/99/3399/1 diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java index e4c35e4..79f565f 100644 --- a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java +++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java @@ -66,6 +66,7 @@ import org.apache.asterix.optimizer.rules.NestGroupByRule; import org.apache.asterix.optimizer.rules.PushAggFuncIntoStandaloneAggregateRule; import org.apache.asterix.optimizer.rules.PushAggregateIntoNestedSubplanRule; +import org.apache.asterix.optimizer.rules.PushAssignBelowUnionAllRule; import org.apache.asterix.optimizer.rules.PushFieldAccessRule; import org.apache.asterix.optimizer.rules.PushGroupByThroughProduct; import org.apache.asterix.optimizer.rules.PushLimitIntoOrderByRule; @@ -117,7 +118,6 @@ import org.apache.hyracks.algebricks.rewriter.rules.IntroduceProjectsRule; import org.apache.hyracks.algebricks.rewriter.rules.IsolateHyracksOperatorsRule; import org.apache.hyracks.algebricks.rewriter.rules.PullSelectOutOfEqJoin; -import org.apache.hyracks.algebricks.rewriter.rules.PushAssignBelowUnionAllRule; import org.apache.hyracks.algebricks.rewriter.rules.PushGroupByIntoSortRule; import org.apache.hyracks.algebricks.rewriter.rules.PushMapOperatorDownThroughProductRule; import
Change in asterixdb[master]: [ASTERIXDB-2567][RT] Fix serializing nullable record fields
Ali Alsuliman has submitted this change and it was merged. ( https://asterix-gerrit.ics.uci.edu/3398 ) Change subject: [ASTERIXDB-2567][RT] Fix serializing nullable record fields .. [ASTERIXDB-2567][RT] Fix serializing nullable record fields - user model changes: no - storage format changes: no - interface changes: no Details: When serializing a record, a field should be checked if it's nullable. The value to be serialized should be checked if it's null or value to serialize it correctly. Exceptions could happen when the record is in an array if record fields are not checked. Change-Id: I42105965e69c60bc4b123337296476b4d75f0ad8 Reviewed-on: https://asterix-gerrit.ics.uci.edu/3398 Contrib: Jenkins Tested-by: Jenkins Reviewed-by: Murtadha Hubail Integration-Tests: Jenkins --- A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.1.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.2.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.4.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.6.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.7.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.8.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.2.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.3.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.4.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.5.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.6.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.7.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.8.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml M asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/serde/ARecordSerializerDeserializer.java 18 files changed, 207 insertions(+), 9 deletions(-) Approvals: Jenkins: Verified; ; Verified Murtadha Hubail: Looks good to me, approved Anon. E. Moose (1000171): Objections: Jenkins: Violations found diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.1.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.1.query.sqlpp new file mode 100644 index 000..8d50408 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.1.query.sqlpp @@ -0,0 +1,21 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the
Change in asterixdb[master]: [ASTERIXDB-2567][RT] Fix serializing nullable record fields
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3398 ) Change subject: [ASTERIXDB-2567][RT] Fix serializing nullable record fields .. Patch Set 4: (1 comment) https://asterix-gerrit.ics.uci.edu/#/c/3398/3/asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/serde/ARecordSerializerDeserializer.java File asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/serde/ARecordSerializerDeserializer.java: https://asterix-gerrit.ics.uci.edu/#/c/3398/3/asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/serde/ARecordSerializerDeserializer.java@185 PS3, Line 185: for (int fieldIndex > not changed in your patch, but why not move this into the for-loop initiali right. done since i had to rebase anyway. -- To view, visit https://asterix-gerrit.ics.uci.edu/3398 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I42105965e69c60bc4b123337296476b4d75f0ad8 Gerrit-Change-Number: 3398 Gerrit-PatchSet: 4 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Wed, 15 May 2019 18:31:05 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [ASTERIXDB-2567][RT] Fix serializing nullable record fields
Hello Anon. E. Moose (1000171), Till Westmann, Jenkins, Michael Blow, Dmitry Lychagin, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3398 to look at the new patch set (#4). Change subject: [ASTERIXDB-2567][RT] Fix serializing nullable record fields .. [ASTERIXDB-2567][RT] Fix serializing nullable record fields - user model changes: no - storage format changes: no - interface changes: no Details: When serializing a record, a field should be checked if it's nullable. The value to be serialized should be checked if it's null or value to serialize it correctly. Exceptions could happen when the record is in an array if record fields are not checked. Change-Id: I42105965e69c60bc4b123337296476b4d75f0ad8 --- A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.1.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.2.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.4.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.6.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.7.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.8.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.2.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.3.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.4.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.5.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.6.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.7.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.8.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml M asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/serde/ARecordSerializerDeserializer.java 18 files changed, 207 insertions(+), 9 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/98/3398/4 -- To view, visit https://asterix-gerrit.ics.uci.edu/3398 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I42105965e69c60bc4b123337296476b4d75f0ad8 Gerrit-Change-Number: 3398 Gerrit-PatchSet: 4 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Till Westmann
Change in asterixdb[master]: [NO ISSUE][COMP] Change plan printing for window operators
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3397 ) Change subject: [NO ISSUE][COMP] Change plan printing for window operators .. Patch Set 2: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3397 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1446238a7c550abf618a72650c2a811fbc4dd710 Gerrit-Change-Number: 3397 Gerrit-PatchSet: 2 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Wed, 15 May 2019 18:16:47 + Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2562][FUN] Add support for bitwise functions
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3377 ) Change subject: [ASTERIXDB-2562][FUN] Add support for bitwise functions .. Patch Set 12: (14 comments) https://asterix-gerrit.ics.uci.edu/#/c/3377/11/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/BitwiseValueCountFlagTypeComputer.java File asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/BitwiseValueCountFlagTypeComputer.java: https://asterix-gerrit.ics.uci.edu/#/c/3377/11/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/BitwiseValueCountFlagTypeComputer.java@45 PS11, Line 45: BitwiseValueCountFlagTypeComputer this class and BitwiseValuePositionFlagTypeComputer and BitwiseValuePositionTypeComputer have a lot in common. you could either create instances with several flags (return type, whether to include ARRAY), or create instances with an enum that differentiates each one. Based on the enum (or flags) you check and configure the return type. https://asterix-gerrit.ics.uci.edu/#/c/3377/11/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/bitwise/AbstractBitMultipleValuesEvaluator.java File asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/bitwise/AbstractBitMultipleValuesEvaluator.java: https://asterix-gerrit.ics.uci.edu/#/c/3377/11/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/bitwise/AbstractBitMultipleValuesEvaluator.java@94 PS11, Line 94: resultStorage.reset(); resultStorage is already reset down, right? https://asterix-gerrit.ics.uci.edu/#/c/3377/11/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/bitwise/AbstractBitMultipleValuesEvaluator.java@102 PS11, Line 102: 0 better to use getStartOffset https://asterix-gerrit.ics.uci.edu/#/c/3377/11/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/bitwise/AbstractBitMultipleValuesEvaluator.java@122 PS11, Line 122: if (!ATypeHierarchy.canPromote(typeTag, ATypeTag.DOUBLE)) { : PointableHelper.setNull(result); : return; : } : : // Value validity check : if (!isValidLongValue(bytes, startOffset + 1, typeTag)) { : PointableHelper.setNull(result); : return; : } we could refactor this code and reuse below (and for other bit functions, too). The refactored method would return boolean (true to return, false to continue). It would also set the result if it says true. https://asterix-gerrit.ics.uci.edu/#/c/3377/11/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/bitwise/AbstractBitMultipleValuesEvaluator.java@154 PS11, Line 154: getLongValue(bytes, startOffset + 1, typeTag, i) you could plug it directly in the method if you want. https://asterix-gerrit.ics.uci.edu/#/c/3377/11/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/bitwise/AbstractBitMultipleValuesEvaluator.java@185 PS11, Line 185: isValidLongValue The returned result is always inverted. we could call it "isInvalidLongValue" and change accordingly. https://asterix-gerrit.ics.uci.edu/#/c/3377/11/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/bitwise/AbstractBitMultipleValuesEvaluator.java@189 PS11, Line 189: float value what is the behaviour if the float is NaN or INF/-INF? https://asterix-gerrit.ics.uci.edu/#/c/3377/11/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/bitwise/AbstractBitMultipleValuesEvaluator.java@192 PS11, Line 192: value > Long.MAX_VALUE what if the value is < MIN_VALUE? do these functions accept negative arguments? https://asterix-gerrit.ics.uci.edu/#/c/3377/11/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/bitwise/AbstractBitMultipleValuesEvaluator.java@219 PS11, Line 219: private long getLongValue(byte[] bytes, int startOffset, ATypeTag typeTag, int argIndex) we should refactor this and reuse in other bit functions. you could put it in a util class. https://asterix-gerrit.ics.uci.edu/#/c/3377/11/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/bitwise/AbstractBitValueCountFlagEvaluator.java File asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/bitwise/AbstractBitValueCountFlagEvaluator.java: https://asterix-gerrit.ics.uci.edu/#/c/3377/11/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/bitwise/AbstractBitValueCountFlagEvaluator.java@174 PS11, Line 174: (int) count) are we losing precision here?
Change in asterixdb[master]: [ASTERIXDB-2567][RT] Fix serializing nullable record fields
Hello Anon. E. Moose (1000171), Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3398 to look at the new patch set (#2). Change subject: [ASTERIXDB-2567][RT] Fix serializing nullable record fields .. [ASTERIXDB-2567][RT] Fix serializing nullable record fields - user model changes: no - storage format changes: no - interface changes: no Details: When serializing a record, a field should be checked if it's nullable. The value to be serialized should be checked if it's null or value to serialize it correctly. Exceptions could happen when the record is in an array if record fields are not checked. Change-Id: I42105965e69c60bc4b123337296476b4d75f0ad8 --- A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.1.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.2.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.4.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.6.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.7.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.8.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.2.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.3.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.4.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.5.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.6.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.7.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.8.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml M asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/serde/ARecordSerializerDeserializer.java 18 files changed, 206 insertions(+), 7 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/98/3398/2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3398 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I42105965e69c60bc4b123337296476b4d75f0ad8 Gerrit-Change-Number: 3398 Gerrit-PatchSet: 2 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2567][RT] Fix serializing nullable record fields
Ali Alsuliman has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/3398 Change subject: [ASTERIXDB-2567][RT] Fix serializing nullable record fields .. [ASTERIXDB-2567][RT] Fix serializing nullable record fields - user model changes: no - storage format changes: no - interface changes: no Details: When serializing a record, a field should be checked if it's nullable. The value to be serialized should be checked if it's null or value to serialize it correctly. Exceptions could happen when the record is in an array if record fields are not checked. Change-Id: I42105965e69c60bc4b123337296476b4d75f0ad8 --- A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.1.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.2.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.4.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.6.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.7.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.2.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.3.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.4.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.5.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.6.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.7.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml M asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/serde/ARecordSerializerDeserializer.java 16 files changed, 181 insertions(+), 7 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/98/3398/1 diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.1.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.1.query.sqlpp new file mode 100644 index 000..8d50408 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.1.query.sqlpp @@ -0,0 +1,21 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +// testing fix for record serialization with nullable fields where the record is in an array and will be constant folded +[{"f1": "fieldStr", "f2": 1 < "text"}]; \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.2.query.sqlpp
Change in asterixdb[master]: [ASTERIXDB-2535][COMP] Fix uuid present in insert/upsert statement
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3370 ) Change subject: [ASTERIXDB-2535][COMP] Fix uuid present in insert/upsert statement .. Patch Set 11: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3370 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I22100d3ff29864b8bfd54b0decb183e5056fdb4a Gerrit-Change-Number: 3370 Gerrit-PatchSet: 11 Gerrit-Owner: Hussain Towaileb Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Hussain Towaileb Gerrit-Reviewer: Hussain Towaileb Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Tue, 14 May 2019 18:04:30 + Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2535][COMP] Fix uuid present in insert/upsert statement
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3370 ) Change subject: [ASTERIXDB-2535][COMP] Fix uuid present in insert/upsert statement .. Patch Set 11: (1 comment) https://asterix-gerrit.ics.uci.edu/#/c/3370/11/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/records/RecordMergeIgnoreDuplicatesDescriptor.java File asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/records/RecordMergeIgnoreDuplicatesDescriptor.java: https://asterix-gerrit.ics.uci.edu/#/c/3370/11/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/records/RecordMergeIgnoreDuplicatesDescriptor.java@89 PS11, Line 89: @Override : protected void handleDuplicateFields() { : // Don't throw an exception, just ignore the duplicate field merging, this will take the : // leftRecord field and ignore the one on the rightRecord. : } we could just pass a flag to RecordMergeEvaluator that would either throw or not throw. The inheritance seems simple here. -- To view, visit https://asterix-gerrit.ics.uci.edu/3370 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I22100d3ff29864b8bfd54b0decb183e5056fdb4a Gerrit-Change-Number: 3370 Gerrit-PatchSet: 11 Gerrit-Owner: Hussain Towaileb Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Hussain Towaileb Gerrit-Reviewer: Hussain Towaileb Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Tue, 14 May 2019 18:08:53 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [NO ISSUE] Additional constructor for TaggedValuePointable
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3242 ) Change subject: [NO ISSUE] Additional constructor for TaggedValuePointable .. Patch Set 5: Code-Review+2 (1 comment) https://asterix-gerrit.ics.uci.edu/#/c/3242/5/hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/src/main/java/org/apache/hyracks/data/std/primitive/TaggedValuePointable.java File hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/src/main/java/org/apache/hyracks/data/std/primitive/TaggedValuePointable.java: https://asterix-gerrit.ics.uci.edu/#/c/3242/5/hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/src/main/java/org/apache/hyracks/data/std/primitive/TaggedValuePointable.java@47 PS5, Line 47: public should we make it private? -- To view, visit https://asterix-gerrit.ics.uci.edu/3242 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3cad369ecb2f7cafd901f11e757272dce7e1e6b5 Gerrit-Change-Number: 3242 Gerrit-PatchSet: 5 Gerrit-Owner: Till Westmann Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Tue, 14 May 2019 00:41:23 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [NO ISSUE] Additional constructor for TaggedValuePointable
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3242 ) Change subject: [NO ISSUE] Additional constructor for TaggedValuePointable .. Patch Set 5: (1 comment) https://asterix-gerrit.ics.uci.edu/#/c/3242/5/hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/src/main/java/org/apache/hyracks/data/std/primitive/TaggedValuePointable.java File hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/src/main/java/org/apache/hyracks/data/std/primitive/TaggedValuePointable.java: https://asterix-gerrit.ics.uci.edu/#/c/3242/5/hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/src/main/java/org/apache/hyracks/data/std/primitive/TaggedValuePointable.java@39 PS5, Line 39: value.set(bytes, start + 1, length - 1); > Yes, we would need to adjust the pointers when setting the pointable. given the assumption that the value passed to "set" should be tagged, we should keep this as is. -- To view, visit https://asterix-gerrit.ics.uci.edu/3242 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3cad369ecb2f7cafd901f11e757272dce7e1e6b5 Gerrit-Change-Number: 3242 Gerrit-PatchSet: 5 Gerrit-Owner: Till Westmann Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Tue, 14 May 2019 00:40:47 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [NO ISSUE][FUN] Add ARRAY_AGG() SQL aggregate function
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3390 ) Change subject: [NO ISSUE][FUN] Add ARRAY_AGG() SQL aggregate function .. Patch Set 4: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3390 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I704e031a1252493e83ad8d45c38b75e0b15c1896 Gerrit-Change-Number: 3390 Gerrit-PatchSet: 4 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Mon, 13 May 2019 19:50:10 + Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2564][RT] Too many objects created in min() and max()
Ali Alsuliman has submitted this change and it was merged. ( https://asterix-gerrit.ics.uci.edu/3391 ) Change subject: [ASTERIXDB-2564][RT] Too many objects created in min() and max() .. [ASTERIXDB-2564][RT] Too many objects created in min() and max() - user model changes: no - storage format changes: no - interface changes: no Details: During min() and max() aggregation, the functions keep track of the aggregation type in order to handle heterogeneous lists. It promotes the aggregation type if needed (e.g. encountered double). Don't switch to new aggregation type and create a new comparator when the new input value type is the same as the previously aggregated values. That is because canPromote(agg_type, new_val_type) will always return true for same types. Change-Id: I0bb9f0715985ae555de00bbf3173c80371d8968b Reviewed-on: https://asterix-gerrit.ics.uci.edu/3391 Contrib: Jenkins Sonar-Qube: Jenkins Tested-by: Jenkins Integration-Tests: Jenkins Reviewed-by: Dmitry Lychagin --- M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractMinMaxAggregateFunction.java 1 file changed, 43 insertions(+), 60 deletions(-) Approvals: Jenkins: Verified; No violations found; ; Verified Anon. E. Moose (1000171): Dmitry Lychagin: Looks good to me, approved diff --git a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractMinMaxAggregateFunction.java b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractMinMaxAggregateFunction.java index 86ae924..90f006d 100644 --- a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractMinMaxAggregateFunction.java +++ b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractMinMaxAggregateFunction.java @@ -30,7 +30,6 @@ import org.apache.hyracks.algebricks.runtime.base.IScalarEvaluatorFactory; import org.apache.hyracks.api.context.IHyracksTaskContext; import org.apache.hyracks.api.dataflow.value.IBinaryComparator; -import org.apache.hyracks.api.dataflow.value.IBinaryComparatorFactory; import org.apache.hyracks.api.exceptions.HyracksDataException; import org.apache.hyracks.api.exceptions.SourceLocation; import org.apache.hyracks.data.std.api.IPointable; @@ -39,18 +38,16 @@ import org.apache.hyracks.dataflow.common.data.accessors.IFrameTupleReference; public abstract class AbstractMinMaxAggregateFunction extends AbstractAggregateFunction { -private IPointable inputVal = new VoidPointable(); -private ArrayBackedValueStorage outputVal = new ArrayBackedValueStorage(); -private ArrayBackedValueStorage tempValForCasting = new ArrayBackedValueStorage(); - -protected ArrayBackedValueStorage resultStorage = new ArrayBackedValueStorage(); -private IScalarEvaluator eval; +protected final ArrayBackedValueStorage resultStorage = new ArrayBackedValueStorage(); +private final IPointable inputVal = new VoidPointable(); +private final ArrayBackedValueStorage outputVal = new ArrayBackedValueStorage(); +private final ArrayBackedValueStorage tempValForCasting = new ArrayBackedValueStorage(); +private final IScalarEvaluator eval; +private final boolean isMin; protected ATypeTag aggType; private IBinaryComparator cmp; -private ITypeConvertComputer tpc; -private final boolean isMin; -public AbstractMinMaxAggregateFunction(IScalarEvaluatorFactory[] args, IHyracksTaskContext context, boolean isMin, +AbstractMinMaxAggregateFunction(IScalarEvaluatorFactory[] args, IHyracksTaskContext context, boolean isMin, SourceLocation sourceLoc) throws HyracksDataException { super(sourceLoc); eval = args[0].createScalarEvaluator(context); @@ -82,9 +79,8 @@ // First value encountered. Set type, comparator, and initial value. aggType = typeTag; // Set comparator. -IBinaryComparatorFactory cmpFactory = - BinaryComparatorFactoryProvider.INSTANCE.getBinaryComparatorFactory(aggType, isMin); -cmp = cmpFactory.createBinaryComparator(); +cmp = BinaryComparatorFactoryProvider.INSTANCE.getBinaryComparatorFactory(aggType, isMin) +.createBinaryComparator(); // Initialize min value. outputVal.assign(inputVal); } else if (typeTag != ATypeTag.SYSTEM_NULL && !ATypeHierarchy.isCompatible(typeTag, aggType)) { @@ -94,56 +90,27 @@ throw new IncompatibleTypeException(sourceLoc, "min/max", aggType.serialize(), typeTag.serialize()); } } else { - // If a system_null is encountered locally, it would be an error; otherwise if it is seen // by a global aggregator, it is simple ignored. if (typeTag == ATypeTag.SYSTEM_NULL) {
Change in asterixdb[master]: [ASTERIXDB-2564][RT] Too many objects created in min() and max()
Hello Anon. E. Moose (1000171), Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3391 to look at the new patch set (#2). Change subject: [ASTERIXDB-2564][RT] Too many objects created in min() and max() .. [ASTERIXDB-2564][RT] Too many objects created in min() and max() - user model changes: no - storage format changes: no - interface changes: no Details: During min() and max() aggregation, the functions keep track of the aggregation type in order to handle heterogeneous lists. It promotes the aggregation type if needed (e.g. encountered double). Don't switch to new aggregation type and create a new comparator when the new input value type is the same as the previously aggregated values. That is because canPromote(agg_type, new_val_type) will always return true for same types. Change-Id: I0bb9f0715985ae555de00bbf3173c80371d8968b --- M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractMinMaxAggregateFunction.java 1 file changed, 43 insertions(+), 60 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/91/3391/2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3391 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I0bb9f0715985ae555de00bbf3173c80371d8968b Gerrit-Change-Number: 3391 Gerrit-PatchSet: 2 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2564][RT] Too many objects created in min() and max()
Ali Alsuliman has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/3391 Change subject: [ASTERIXDB-2564][RT] Too many objects created in min() and max() .. [ASTERIXDB-2564][RT] Too many objects created in min() and max() - user model changes: no - storage format changes: no - interface changes: no Details: During min() and max() aggregation, the functions keep track of the aggregation type in order to handle heterogeneous lists. It promotes the aggregation type if needed (e.g. encountered double). Don't switch to new aggregation type and create a new comparator when the new input value type is the same as the previously aggregated values. That is because canPromote(agg_type, new_val_type) will always return true for same types. Change-Id: I0bb9f0715985ae555de00bbf3173c80371d8968b --- M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractMinMaxAggregateFunction.java 1 file changed, 40 insertions(+), 54 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/91/3391/1 diff --git a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractMinMaxAggregateFunction.java b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractMinMaxAggregateFunction.java index 86ae924..616bb5a 100644 --- a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractMinMaxAggregateFunction.java +++ b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractMinMaxAggregateFunction.java @@ -30,7 +30,6 @@ import org.apache.hyracks.algebricks.runtime.base.IScalarEvaluatorFactory; import org.apache.hyracks.api.context.IHyracksTaskContext; import org.apache.hyracks.api.dataflow.value.IBinaryComparator; -import org.apache.hyracks.api.dataflow.value.IBinaryComparatorFactory; import org.apache.hyracks.api.exceptions.HyracksDataException; import org.apache.hyracks.api.exceptions.SourceLocation; import org.apache.hyracks.data.std.api.IPointable; @@ -39,18 +38,16 @@ import org.apache.hyracks.dataflow.common.data.accessors.IFrameTupleReference; public abstract class AbstractMinMaxAggregateFunction extends AbstractAggregateFunction { -private IPointable inputVal = new VoidPointable(); -private ArrayBackedValueStorage outputVal = new ArrayBackedValueStorage(); -private ArrayBackedValueStorage tempValForCasting = new ArrayBackedValueStorage(); - -protected ArrayBackedValueStorage resultStorage = new ArrayBackedValueStorage(); -private IScalarEvaluator eval; +protected final ArrayBackedValueStorage resultStorage = new ArrayBackedValueStorage(); +private final IPointable inputVal = new VoidPointable(); +private final ArrayBackedValueStorage outputVal = new ArrayBackedValueStorage(); +private final ArrayBackedValueStorage tempValForCasting = new ArrayBackedValueStorage(); +private final IScalarEvaluator eval; +private final boolean isMin; protected ATypeTag aggType; private IBinaryComparator cmp; -private ITypeConvertComputer tpc; -private final boolean isMin; -public AbstractMinMaxAggregateFunction(IScalarEvaluatorFactory[] args, IHyracksTaskContext context, boolean isMin, +AbstractMinMaxAggregateFunction(IScalarEvaluatorFactory[] args, IHyracksTaskContext context, boolean isMin, SourceLocation sourceLoc) throws HyracksDataException { super(sourceLoc); eval = args[0].createScalarEvaluator(context); @@ -82,9 +79,8 @@ // First value encountered. Set type, comparator, and initial value. aggType = typeTag; // Set comparator. -IBinaryComparatorFactory cmpFactory = - BinaryComparatorFactoryProvider.INSTANCE.getBinaryComparatorFactory(aggType, isMin); -cmp = cmpFactory.createBinaryComparator(); +cmp = BinaryComparatorFactoryProvider.INSTANCE.getBinaryComparatorFactory(aggType, isMin) +.createBinaryComparator(); // Initialize min value. outputVal.assign(inputVal); } else if (typeTag != ATypeTag.SYSTEM_NULL && !ATypeHierarchy.isCompatible(typeTag, aggType)) { @@ -94,56 +90,27 @@ throw new IncompatibleTypeException(sourceLoc, "min/max", aggType.serialize(), typeTag.serialize()); } } else { - // If a system_null is encountered locally, it would be an error; otherwise if it is seen // by a global aggregator, it is simple ignored. if (typeTag == ATypeTag.SYSTEM_NULL) { processSystemNull(); return; } - +if (aggType == typeTag) { +compareAndUpdate(cmp, inputVal, outputVal); +return; +} if
Change in asterixdb[master]: [ASTERIXDB-2535][COMP] Fix uuid present in insert/upsert statement
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3370 ) Change subject: [ASTERIXDB-2535][COMP] Fix uuid present in insert/upsert statement .. Patch Set 8: (1 comment) https://asterix-gerrit.ics.uci.edu/#/c/3370/7/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/RecordMergeTypeComputer.java File asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/RecordMergeTypeComputer.java: https://asterix-gerrit.ics.uci.edu/#/c/3370/7/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/RecordMergeTypeComputer.java@115 PS7, Line 115: : else { > I agree with you. But my concern is, if a user (say by mistake) tries input You should think of both merge functions as standalone functions. They could appear anywhere in any query. There is no guarantee that casting will be injected. Casting is not specific to insert/upsert. If you are worried about the error message being confusing for uuid case, there are workarounds. But we shouldn't change the generic case to fit the special case. -- To view, visit https://asterix-gerrit.ics.uci.edu/3370 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I22100d3ff29864b8bfd54b0decb183e5056fdb4a Gerrit-Change-Number: 3370 Gerrit-PatchSet: 8 Gerrit-Owner: Hussain Towaileb Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Hussain Towaileb Gerrit-Reviewer: Hussain Towaileb Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Sat, 11 May 2019 05:30:27 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [ASTERIXDB-2535][COMP] Fix uuid present in insert/upsert statement
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3370 ) Change subject: [ASTERIXDB-2535][COMP] Fix uuid present in insert/upsert statement .. Patch Set 8: (5 comments) https://asterix-gerrit.ics.uci.edu/#/c/3370/8/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/BuiltinFunctions.java File asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/BuiltinFunctions.java: https://asterix-gerrit.ics.uci.edu/#/c/3370/8/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/BuiltinFunctions.java@2186 PS8, Line 2186: addFunction(RECORD_MERGE_IGNORE_DUPLICATES, RecordMergeTypeComputer.INSTANCE_IGNORE_DUPLICATES, true); with this now being a new function and not related to UUID, we should add few test cases to make sure you are covered. You could add them in a later change. https://asterix-gerrit.ics.uci.edu/#/c/3370/7/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/RecordMergeTypeComputer.java File asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/RecordMergeTypeComputer.java: https://asterix-gerrit.ics.uci.edu/#/c/3370/7/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/RecordMergeTypeComputer.java@115 PS7, Line 115: : else { > Short answer: I think relying on ARecordCaster being injected isn't a good idea. We should maintain that the duplicate fields must have the same types. That's at compile-time here in the type computer. At runtime, you get to choose what to do for duplicates; whether to 1) throw duplicate exception when left and right have different types (normal current behaviour) (actually it throws exception when they have different values, i.e. if "id": 2, "id": 3. For "id":2, "id":2, it picks left) 2) ignore and pick the one on left input record (but also making sure left type is the same as the corresponding field in output record. This will be consistent with the type computer since it's picking the left type) In your example, if the "id" is part of the output record schema, then we should check that the input value type is the same as the one in the schema. The semantics of the record-merge-ignore-duplicates "runtime" says if there are duplicate fields, it is going to pick the left value (that's also the semantics of normal record merge for same-value duplicates that the compiler couldn't figure out because they are open fields). https://asterix-gerrit.ics.uci.edu/#/c/3370/8/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/RecordMergeTypeComputer.java File asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/RecordMergeTypeComputer.java: https://asterix-gerrit.ics.uci.edu/#/c/3370/8/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/RecordMergeTypeComputer.java@49 PS8, Line 49: private RecordMergeTypeComputer() { : this(false); : } : : private RecordMergeTypeComputer(boolean isIgnoreDuplicates) { : this.isIgnoreDuplicates = isIgnoreDuplicates; : } it's better to have one constructor with a boolean parameter) https://asterix-gerrit.ics.uci.edu/#/c/3370/8/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/records/RecordMergeIgnoreDuplicatesDescriptor.java File asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/records/RecordMergeIgnoreDuplicatesDescriptor.java: https://asterix-gerrit.ics.uci.edu/#/c/3370/8/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/records/RecordMergeIgnoreDuplicatesDescriptor.java@36 PS8, Line 36: @MissingNullInOutFunction just to make sure; is this annotation supposed to be for descriptors or evaluators? https://asterix-gerrit.ics.uci.edu/#/c/3370/8/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/records/RecordMergeIgnoreDuplicatesDescriptor.java@68 PS8, Line 68: RecordMergeIgnoreDuplicatesEvaluator we could remove this class and just extend RecordMergeEvaluator right here. -- To view, visit https://asterix-gerrit.ics.uci.edu/3370 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I22100d3ff29864b8bfd54b0decb183e5056fdb4a Gerrit-Change-Number: 3370 Gerrit-PatchSet: 8 Gerrit-Owner: Hussain Towaileb Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Hussain Towaileb Gerrit-Reviewer: Hussain Towaileb Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Fri, 10 May 2019 19:55:33 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [ASTERIXDB-2555][RT][COMP] Make hash join use logical comparison
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3387 ) Change subject: [ASTERIXDB-2555][RT][COMP] Make hash join use logical comparison .. Patch Set 4: (6 comments) https://asterix-gerrit.ics.uci.edu/#/c/3387/2/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/evaluators/TuplePairEvaluatorFactory.java File hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/evaluators/TuplePairEvaluatorFactory.java: https://asterix-gerrit.ics.uci.edu/#/c/3387/2/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/evaluators/TuplePairEvaluatorFactory.java@52 PS2, Line 52: ITuplePairCo > agreed, I think synchronization is not required here Done https://asterix-gerrit.ics.uci.edu/#/c/3387/2/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/evaluators/TuplePairEvaluatorFactory.java@78 PS2, Line 78: reseter.reset(tuplePairRef, leftAccessor, leftIndex, rightAccessor, rightIndex); : conditionEvaluator.evaluate(tuplePairRef, res); : return booleanInspector.getBooleanValue(res.getByteArray(), res.getStartOffset(), res.getLength()) ? 0 : 1; : } : > I think I prefer patch set 1 here- but not with a ton of conviction Done https://asterix-gerrit.ics.uci.edu/#/c/3387/2/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/evaluators/TuplePairEvaluatorFactory.java@84 PS2, Line 84: int leftTupleIndex, IFrameTupleAccessor rightAccessor, int rightTupleIndex) { : ref.reset(leftAccessor, leftTupleIndex, rightAccessor, rightTupleIndex); : } : : priva > should we use a ternary op here? Done https://asterix-gerrit.ics.uci.edu/#/c/3387/2/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/evaluators/TuplePairEvaluatorFactory.java@116 PS2, Line 116: rride : public int getFieldCount() { : return refLeft.getFieldCount() + refRight.getFieldCount(); : } : > ternary? Done https://asterix-gerrit.ics.uci.edu/#/c/3387/2/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/dataflow/value/ITuplePairComparator.java File hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/dataflow/value/ITuplePairComparator.java: https://asterix-gerrit.ics.uci.edu/#/c/3387/2/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/dataflow/value/ITuplePairComparator.java@27 PS2, Line 27: CriticalPath > What does this do? It does nothing. It's just a reminder that this method will be executed a lot, not only once per tuple, but every time you join a tuple with a bunch of other tuples. https://asterix-gerrit.ics.uci.edu/#/c/3387/2/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/util/JoinUtil.java File hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/util/JoinUtil.java: https://asterix-gerrit.ics.uci.edu/#/c/3387/2/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/util/JoinUtil.java@36 PS2, Line 36: > ok. let's move it to the "join" package and may be rename to HybridHashJoin Done -- To view, visit https://asterix-gerrit.ics.uci.edu/3387 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If1834967fdd913fdc76003f09636b2450d07cd5e Gerrit-Change-Number: 3387 Gerrit-PatchSet: 4 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Fri, 10 May 2019 11:30:34 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [ASTERIXDB-2555][RT][COMP] Make hash join use logical comparison
Hello Anon. E. Moose (1000171), Till Westmann, Jenkins, Michael Blow, Dmitry Lychagin, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3387 to look at the new patch set (#4). Change subject: [ASTERIXDB-2555][RT][COMP] Make hash join use logical comparison .. [ASTERIXDB-2555][RT][COMP] Make hash join use logical comparison - user model changes: no - storage format changes: no - interface changes: no Details: This patch changes the hash join operator to use the join condition to evaluate if tuples are equal when joining. Binary physical comparators have been removed. The join condition evaluator is in TuplePairEvaluator. - extraced TuplePairEvaluatorFactory out of nested loop join class into a separate class so that it is shared among nested loop and hash join. - switched from FrameTuplePairComparator to ITuplePairComparator in in OptimizedHybridHashJoin and InMemoryHashJoin. - moved debugging code from OptimizedHybridHashJoin into a separate class, JoinUtil. - temporarily made the logical comparison of multisets use raw binary comparison instead of returning null until the logic is implemented. - made IBinaryBooleanInspector a functional interface and updated the implementations. - updated record and array test cases to reflect the new behaviour of hash join where logical comparison could produce null. Also, updated sorting, group by and distinct test cases since the input data has been modified. - added two new input files arrays1nulls.adm & arrays2nulls.adm to be used by the open dataset. previous arrays1.adm & arrays2.adm are used by the closed dataset since it cannot accept arrays with null values. Change-Id: If1834967fdd913fdc76003f09636b2450d07cd5e --- A asterixdb/asterix-app/data/complex/arrays1nulls.adm A asterixdb/asterix-app/data/complex/arrays2nulls.adm M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/distinct/array/array.2.update.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/group-by/gby-array/gby-array.2.update.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/join/hash_join_array/hash_join_array.2.update.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/sorting/arrays/arrays.2.update.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/results/distinct/array/array.3.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/group-by/gby-array/gby-array.3.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/group-by/gby-array/gby-array.5.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_array/hash_join_array.7.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_array/hash_join_array.8.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.03.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.04.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.05.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.06.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.07.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.08.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.09.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.10.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.11.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.12.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.13.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.14.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.15.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.16.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.17.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.18.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.19.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.20.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/sorting/arrays/arrays.3.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/sorting/arrays/arrays.5.adm M
Change in asterixdb[master]: [ASTERIXDB-2555][RT][COMP] Make hash join use logical comparison
Hello Anon. E. Moose (1000171), Till Westmann, Jenkins, Michael Blow, Dmitry Lychagin, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3387 to look at the new patch set (#3). Change subject: [ASTERIXDB-2555][RT][COMP] Make hash join use logical comparison .. [ASTERIXDB-2555][RT][COMP] Make hash join use logical comparison - user model changes: no - storage format changes: no - interface changes: no Details: This patch changes the hash join operator to use the join condition to evaluate if tuples are equal when joining. Binary physical comparators have been removed. The join condition evaluator is in TuplePairEvaluator. - extraced TuplePairEvaluatorFactory out of nested loop join class into a separate class so that it is shared among nested loop and hash join. - switched from FrameTuplePairComparator to ITuplePairComparator in in OptimizedHybridHashJoin and InMemoryHashJoin. - moved debugging code from OptimizedHybridHashJoin into a separate class, JoinUtil. - temporarily made the logical comparison of multisets use raw binary comparison instead of returning null until the logic is implemented. - made IBinaryBooleanInspector a functional interface and updated the implementations. - updated record and array test cases to reflect the new behaviour of hash join where logical comparison could produce null. Also, updated sorting, group by and distinct test cases since the input data has been modified. - added two new input files arrays1nulls.adm & arrays2nulls.adm to be used by the open dataset. previous arrays1.adm & arrays2.adm are used by the closed dataset since it cannot accept arrays with null values. Change-Id: If1834967fdd913fdc76003f09636b2450d07cd5e --- A asterixdb/asterix-app/data/complex/arrays1nulls.adm A asterixdb/asterix-app/data/complex/arrays2nulls.adm M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/distinct/array/array.2.update.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/group-by/gby-array/gby-array.2.update.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/join/hash_join_array/hash_join_array.2.update.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/sorting/arrays/arrays.2.update.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/results/distinct/array/array.3.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/group-by/gby-array/gby-array.3.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/group-by/gby-array/gby-array.5.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_array/hash_join_array.7.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_array/hash_join_array.8.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.03.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.04.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.05.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.06.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.07.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.08.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.09.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.10.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.11.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.12.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.13.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.14.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.15.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.16.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.17.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.18.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.19.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.20.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/sorting/arrays/arrays.3.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/sorting/arrays/arrays.5.adm M
Change in asterixdb[master]: [ASTERIXDB-2555][RT][COMP] Make hash join use logical comparison
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3387 ) Change subject: [ASTERIXDB-2555][RT][COMP] Make hash join use logical comparison .. Patch Set 2: (2 comments) https://asterix-gerrit.ics.uci.edu/#/c/3387/2/asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.03.adm File asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.03.adm: https://asterix-gerrit.ics.uci.edu/#/c/3387/2/asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.03.adm@a16 PS2, Line 16: > why these records no longer join? "address" values seem to be the same on b it's because of the "apt" field being null. Logical comparison will render the whole comparison as null which will render the comparison as false. This is inner join. https://asterix-gerrit.ics.uci.edu/#/c/3387/2/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/util/JoinUtil.java File hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/util/JoinUtil.java: https://asterix-gerrit.ics.uci.edu/#/c/3387/2/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/util/JoinUtil.java@36 PS2, Line 36: public String printPartitionInfo(BitSet spilledStatus, OptimizedHybridHashJoin.SIDE whichSide, int numOfPartitions, > This method is specific to OptimizedHybridHashJoin, so why did we move into it's a debugging print which is not used in OptimizedHybridHashJoin. Having it there in production code isn't suitable, I think. But I can move it back if you see otherwise. If it's about the package, I can keep this Util class in the same package as the hybrid join. Let me know. -- To view, visit https://asterix-gerrit.ics.uci.edu/3387 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If1834967fdd913fdc76003f09636b2450d07cd5e Gerrit-Change-Number: 3387 Gerrit-PatchSet: 2 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Thu, 09 May 2019 18:35:31 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [NO ISSUE][COMP] Reduce memory requirements for metadata only queries
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3385 ) Change subject: [NO ISSUE][COMP] Reduce memory requirements for metadata only queries .. Patch Set 2: Code-Review+1 -- To view, visit https://asterix-gerrit.ics.uci.edu/3385 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0d8902f88f61953e839e2fa0b06a77c324edbc57 Gerrit-Change-Number: 3385 Gerrit-PatchSet: 2 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Thu, 09 May 2019 18:27:01 + Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2555][RT][COMP] Make hash join use logical comparison
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3387 ) Change subject: [ASTERIXDB-2555][RT][COMP] Make hash join use logical comparison .. Patch Set 2: (1 comment) https://asterix-gerrit.ics.uci.edu/#/c/3387/2/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/evaluators/TuplePairEvaluatorFactory.java File hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/evaluators/TuplePairEvaluatorFactory.java: https://asterix-gerrit.ics.uci.edu/#/c/3387/2/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/evaluators/TuplePairEvaluatorFactory.java@52 PS2, Line 52: synchronized not sure why the old code needed this to be synchronized. -- To view, visit https://asterix-gerrit.ics.uci.edu/3387 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If1834967fdd913fdc76003f09636b2450d07cd5e Gerrit-Change-Number: 3387 Gerrit-PatchSet: 2 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Comment-Date: Thu, 09 May 2019 15:57:50 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [ASTERIXDB-2555][RT][COMP] Make hash join use logical comparison
Hello Anon. E. Moose (1000171), Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3387 to look at the new patch set (#2). Change subject: [ASTERIXDB-2555][RT][COMP] Make hash join use logical comparison .. [ASTERIXDB-2555][RT][COMP] Make hash join use logical comparison - user model changes: no - storage format changes: no - interface changes: no Details: This patch changes the hash join operator to use the join condition to evaluate if tuples are equal when joining. Binary physical comparators have been removed. The join condition evaluator is in TuplePairEvaluator. - extraced TuplePairEvaluatorFactory out of nested loop join class into a separate class so that it is shared among nested loop and hash join. - switched from FrameTuplePairComparator to ITuplePairComparator in in OptimizedHybridHashJoin and InMemoryHashJoin. - moved debugging code from OptimizedHybridHashJoin into a separate class, JoinUtil. - temporarily made the logical comparison of multisets use raw binary comparison instead of returning null until the logic is implemented. - made IBinaryBooleanInspector a functional interface and updated the implementations. - updated record and array test cases to reflect the new behaviour of hash join where logical comparison could produce null. Also, updated sorting, group by and distinct test cases since the input data has been modified. - added two new input files arrays1nulls.adm & arrays2nulls.adm to be used by the open dataset. previous arrays1.adm & arrays2.adm are used by the closed dataset since it cannot accept arrays with null values. Change-Id: If1834967fdd913fdc76003f09636b2450d07cd5e --- A asterixdb/asterix-app/data/complex/arrays1nulls.adm A asterixdb/asterix-app/data/complex/arrays2nulls.adm M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/distinct/array/array.2.update.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/group-by/gby-array/gby-array.2.update.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/join/hash_join_array/hash_join_array.2.update.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/sorting/arrays/arrays.2.update.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/results/distinct/array/array.3.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/group-by/gby-array/gby-array.3.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/group-by/gby-array/gby-array.5.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_array/hash_join_array.7.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_array/hash_join_array.8.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.03.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.04.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.05.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.06.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.07.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.08.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.09.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.10.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.11.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.12.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.13.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.14.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.15.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.16.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.17.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.18.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.19.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.20.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/sorting/arrays/arrays.3.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/sorting/arrays/arrays.5.adm M
Change in asterixdb[master]: [NO ISSUE][COMP] Make memory requirements an operator property
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3374 ) Change subject: [NO ISSUE][COMP] Make memory requirements an operator property .. Patch Set 3: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3374 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3481ddfe163c6ce786290c540cbd05db16a7f64f Gerrit-Change-Number: 3374 Gerrit-PatchSet: 3 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Wed, 08 May 2019 17:55:34 + Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2535][COMP] Fix uuid present in insert/upsert statement
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3370 ) Change subject: [ASTERIXDB-2535][COMP] Fix uuid present in insert/upsert statement .. Patch Set 4: (1 comment) https://asterix-gerrit.ics.uci.edu/#/c/3370/4/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/records/RecordMergeUuidIgnoreDuplicateEvaluator.java File asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/records/RecordMergeUuidIgnoreDuplicateEvaluator.java: https://asterix-gerrit.ics.uci.edu/#/c/3370/4/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/records/RecordMergeUuidIgnoreDuplicateEvaluator.java@37 PS4, Line 37: isUuidIgnoreDuplicate; do we need this? this function should always ignore uuid duplicates. -- To view, visit https://asterix-gerrit.ics.uci.edu/3370 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I22100d3ff29864b8bfd54b0decb183e5056fdb4a Gerrit-Change-Number: 3370 Gerrit-PatchSet: 4 Gerrit-Owner: Hussain Towaileb Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Hussain Towaileb Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Tue, 07 May 2019 03:35:36 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [NO ISSUE][COMP] Make memory requirements an operator property
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3374 ) Change subject: [NO ISSUE][COMP] Make memory requirements an operator property .. Patch Set 2: Code-Review+1 -- To view, visit https://asterix-gerrit.ics.uci.edu/3374 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3481ddfe163c6ce786290c540cbd05db16a7f64f Gerrit-Change-Number: 3374 Gerrit-PatchSet: 2 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Tue, 07 May 2019 03:24:05 + Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][COMP] Make memory requirements an operator property
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3374 ) Change subject: [NO ISSUE][COMP] Make memory requirements an operator property .. Patch Set 2: (2 comments) https://asterix-gerrit.ics.uci.edu/#/c/3374/2/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/SetAsterixPhysicalOperatorsRule.java File asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/SetAsterixPhysicalOperatorsRule.java: https://asterix-gerrit.ics.uci.edu/#/c/3374/2/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/SetAsterixPhysicalOperatorsRule.java@135 PS2, Line 135: inputFileSize should we remove this too? it seems that the inputFileSize depends on the frame limit which we can calculate later at code gen when we get the frame limit from the local memory requirements object. no? https://asterix-gerrit.ics.uci.edu/#/c/3374/2/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/HybridToInMemoryHashJoinRule.java File hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/HybridToInMemoryHashJoinRule.java: https://asterix-gerrit.ics.uci.edu/#/c/3374/2/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/HybridToInMemoryHashJoinRule.java@37 PS2, Line 37: rewritePre I think with "Pre" we will not descend to inputs if the currently visited operator returns true (i.e. changed from hybrid to in memory). no? -- To view, visit https://asterix-gerrit.ics.uci.edu/3374 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3481ddfe163c6ce786290c540cbd05db16a7f64f Gerrit-Change-Number: 3374 Gerrit-PatchSet: 2 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Tue, 07 May 2019 03:23:58 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [ASTERIXDB-2552][RT] Reuse objects on reseting buffer manager
Ali Alsuliman has submitted this change and it was merged. ( https://asterix-gerrit.ics.uci.edu/3372 ) Change subject: [ASTERIXDB-2552][RT] Reuse objects on reseting buffer manager .. [ASTERIXDB-2552][RT] Reuse objects on reseting buffer manager - user model changes: no - storage format changes: no - interface changes: yes Details: The "VariableFrameMemoryManager" releases the physical frames objects, the logical frames objects and the free slots objects on reset(). This patch is to retain those objects and only release them on close() (as also stated in the interface). - added close() to IFrameFreeSlotPolicy to make the interface similar to the other interfaces (IFrameBufferManager, IFramePool, ... etc) where close() is supposed to release while reset() is supposed to only reset but retain resources. - renamed "InMemorySortRuntimeFactory" to "MicroSortRuntimeFactory". Change-Id: I88bea27e5024c621412ef609475e2a7ba1913afa Reviewed-on: https://asterix-gerrit.ics.uci.edu/3372 Contrib: Jenkins Tested-by: Jenkins Integration-Tests: Jenkins Reviewed-by: Dmitry Lychagin Reviewed-by: Till Westmann --- M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/MicroStableSortPOperator.java R hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/sort/MicroSortRuntimeFactory.java M hyracks-fullstack/algebricks/algebricks-tests/src/test/java/org/apache/hyracks/algebricks/tests/pushruntime/PushRuntimeTest.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/FrameFreeSlotBiggestFirst.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/FrameFreeSlotLastFit.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/FrameFreeSlotSmallestFit.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/IFrameBufferManager.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/IFrameFreeSlotPolicy.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/VariableDeletableTupleMemoryManager.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/VariableFrameMemoryManager.java 10 files changed, 90 insertions(+), 43 deletions(-) Approvals: Jenkins: Verified; ; Verified Anon. E. Moose (1000171): Dmitry Lychagin: Looks good to me, but someone else must approve Till Westmann: Looks good to me, approved Objections: Jenkins: Violations found diff --git a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/MicroStableSortPOperator.java b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/MicroStableSortPOperator.java index c6a7d9d..413c1a4 100644 --- a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/MicroStableSortPOperator.java +++ b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/MicroStableSortPOperator.java @@ -32,7 +32,7 @@ import org.apache.hyracks.algebricks.data.IBinaryComparatorFactoryProvider; import org.apache.hyracks.algebricks.data.INormalizedKeyComputerFactoryProvider; import org.apache.hyracks.algebricks.runtime.base.IPushRuntimeFactory; -import org.apache.hyracks.algebricks.runtime.operators.sort.InMemorySortRuntimeFactory; +import org.apache.hyracks.algebricks.runtime.operators.sort.MicroSortRuntimeFactory; import org.apache.hyracks.api.dataflow.value.IBinaryComparatorFactory; import org.apache.hyracks.api.dataflow.value.INormalizedKeyComputerFactory; import org.apache.hyracks.api.dataflow.value.RecordDescriptor; @@ -80,7 +80,7 @@ i++; } -IPushRuntimeFactory runtime = new InMemorySortRuntimeFactory(sortFields, nkcf, comps, null, maxNumberOfFrames); +IPushRuntimeFactory runtime = new MicroSortRuntimeFactory(sortFields, nkcf, comps, null, maxNumberOfFrames); builder.contributeMicroOperator(op, runtime, recDescriptor); ILogicalOperator src = op.getInputs().get(0).getValue(); builder.contributeGraphEdge(src, 0, op, 0); diff --git a/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/sort/InMemorySortRuntimeFactory.java b/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/sort/MicroSortRuntimeFactory.java similarity index 96%
Change in asterixdb[master]: [ASTERIXDB-2552][RT] Reuse objects on reseting buffer manager
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3372 ) Change subject: [ASTERIXDB-2552][RT] Reuse objects on reseting buffer manager .. Patch Set 2: (1 comment) https://asterix-gerrit.ics.uci.edu/#/c/3372/2/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/FrameFreeSlotBiggestFirst.java File hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/FrameFreeSlotBiggestFirst.java: https://asterix-gerrit.ics.uci.edu/#/c/3372/2/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/FrameFreeSlotBiggestFirst.java@104 PS2, Line 104: // TODO(ali): fix to not release resources > In this change? If you insist. But I would prefer to do it in a later change to keep this one small and manageable. -- To view, visit https://asterix-gerrit.ics.uci.edu/3372 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I88bea27e5024c621412ef609475e2a7ba1913afa Gerrit-Change-Number: 3372 Gerrit-PatchSet: 2 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Fri, 03 May 2019 17:57:10 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [NO ISSUE][HYR][RT] Handle calls to close() properly in hash join
Ali Alsuliman has submitted this change and it was merged. ( https://asterix-gerrit.ics.uci.edu/3375 ) Change subject: [NO ISSUE][HYR][RT] Handle calls to close() properly in hash join .. [NO ISSUE][HYR][RT] Handle calls to close() properly in hash join - user model changes: no - storage format changes: no - interface changes: no Details: Hash join build activity assumes that upon calling close() the join state must have been allocated in open() and therefore, it will directly call "close build" using the join state. However, if the build activity throws an exception in open(), then it could happen that the join state has not been initialized. Calling close() on the build activity will lead to a NPE since it will use the join state directly. This patch checks if the join state has been initialized. If so, it will close the build as usual. Change-Id: I41fff7deed5e56818bfefb4d36b8018a9f512cbb Reviewed-on: https://asterix-gerrit.ics.uci.edu/3375 Contrib: Jenkins Tested-by: Jenkins Integration-Tests: Jenkins Reviewed-by: Murtadha Hubail --- M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoinOperatorDescriptor.java 1 file changed, 9 insertions(+), 7 deletions(-) Approvals: Jenkins: Verified; ; Verified Anon. E. Moose (1000171): Murtadha Hubail: Looks good to me, approved Objections: Jenkins: Violations found diff --git a/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoinOperatorDescriptor.java b/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoinOperatorDescriptor.java index 403c492..81d08b2 100644 --- a/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoinOperatorDescriptor.java +++ b/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoinOperatorDescriptor.java @@ -315,13 +315,15 @@ @Override public void close() throws HyracksDataException { -state.hybridHJ.closeBuild(); -if (isFailed) { -state.hybridHJ.clearBuildTempFiles(); -} else { -ctx.setStateObject(state); -if (LOGGER.isTraceEnabled()) { -LOGGER.trace("OptimizedHybridHashJoin closed its build phase"); +if (state.hybridHJ != null) { +state.hybridHJ.closeBuild(); +if (isFailed) { +state.hybridHJ.clearBuildTempFiles(); +} else { +ctx.setStateObject(state); +if (LOGGER.isTraceEnabled()) { +LOGGER.trace("OptimizedHybridHashJoin closed its build phase"); +} } } } -- To view, visit https://asterix-gerrit.ics.uci.edu/3375 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I41fff7deed5e56818bfefb4d36b8018a9f512cbb Gerrit-Change-Number: 3375 Gerrit-PatchSet: 2 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Till Westmann
Change in asterixdb[master]: [NO ISSUE][OTR] GlobalConfig Cleanup
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3376 ) Change subject: [NO ISSUE][OTR] GlobalConfig Cleanup .. Patch Set 1: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3376 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie59a646d7d2eedaf1360c97c355f827d80917adf Gerrit-Change-Number: 3376 Gerrit-PatchSet: 1 Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Thu, 02 May 2019 19:10:17 + Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][FUN] Move negative tests to seperate test unit
Ali Alsuliman has submitted this change and it was merged. ( https://asterix-gerrit.ics.uci.edu/3371 ) Change subject: [NO ISSUE][FUN] Move negative tests to seperate test unit .. [NO ISSUE][FUN] Move negative tests to seperate test unit - user model changes: no - storage format changes: no - interface changes: no Details: Moved array functions negative tests to a seperate test unit. Change-Id: I0a3bfce402e4b7091c40483b5802db7b27afa717 Reviewed-on: https://asterix-gerrit.ics.uci.edu/3371 Contrib: Jenkins Sonar-Qube: Jenkins Tested-by: Jenkins Integration-Tests: Jenkins Reviewed-by: Dmitry Lychagin --- R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_concat/array_concat.4.ddl.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.01.query.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.02.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.03.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.04.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.05.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.06.query.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.07.query.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.08.query.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.09.query.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_intersect/array_intersect.4.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_intersect/array_intersect.5.ddl.sqlpp D asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_intersect/array_intersect.5.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_range/array_range.4.ddl.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_replace/array_replace.5.ddl.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_symdiff/array_symdiff.4.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_symdiff/array_symdiff.5.ddl.sqlpp D asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_symdiff/array_symdiff.5.query.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_symdiffn/array_symdiffn.4.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_symdiffn/array_symdiffn.5.ddl.sqlpp D asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_symdiffn/array_symdiffn.5.query.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_union/array_union.4.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_union/array_union.5.ddl.sqlpp D asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_union/array_union.5.query.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml 25 files changed, 58 insertions(+), 179 deletions(-) Approvals: Jenkins: Verified; No violations found; ; Verified Anon. E. Moose (1000171): Dmitry Lychagin: Looks good to me, approved diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_concat/array_concat.5.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_concat/array_concat.4.ddl.sqlpp similarity index 100% rename from asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_concat/array_concat.5.ddl.sqlpp rename to asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_concat/array_concat.4.ddl.sqlpp diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_concat/array_concat.4.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.01.query.sqlpp similarity index 97% rename from asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_concat/array_concat.4.query.sqlpp rename to asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.01.query.sqlpp index 805dd99..a65a081 100755 ---
Change in asterixdb[master]: [NO ISSUE][HYR][RT] Handle calls to close() properly in hash join
Ali Alsuliman has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/3375 Change subject: [NO ISSUE][HYR][RT] Handle calls to close() properly in hash join .. [NO ISSUE][HYR][RT] Handle calls to close() properly in hash join - user model changes: no - storage format changes: no - interface changes: no Details: Hash join build activity assumes that upon calling close() the join state must have been allocated in open() and therefore, it will directly call "close build" using the join state. However, if the build activity throws an exception in open(), then it could happen that the join state has not been initialized. Calling close() on the build activity will lead to a NPE since it will use the join state directly. This patch checks if the join state has been initialized. If so, it will close the build as usual. Change-Id: I41fff7deed5e56818bfefb4d36b8018a9f512cbb --- M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoinOperatorDescriptor.java 1 file changed, 9 insertions(+), 7 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/75/3375/1 diff --git a/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoinOperatorDescriptor.java b/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoinOperatorDescriptor.java index 403c492..81d08b2 100644 --- a/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoinOperatorDescriptor.java +++ b/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoinOperatorDescriptor.java @@ -315,13 +315,15 @@ @Override public void close() throws HyracksDataException { -state.hybridHJ.closeBuild(); -if (isFailed) { -state.hybridHJ.clearBuildTempFiles(); -} else { -ctx.setStateObject(state); -if (LOGGER.isTraceEnabled()) { -LOGGER.trace("OptimizedHybridHashJoin closed its build phase"); +if (state.hybridHJ != null) { +state.hybridHJ.closeBuild(); +if (isFailed) { +state.hybridHJ.clearBuildTempFiles(); +} else { +ctx.setStateObject(state); +if (LOGGER.isTraceEnabled()) { +LOGGER.trace("OptimizedHybridHashJoin closed its build phase"); +} } } } -- To view, visit https://asterix-gerrit.ics.uci.edu/3375 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I41fff7deed5e56818bfefb4d36b8018a9f512cbb Gerrit-Change-Number: 3375 Gerrit-PatchSet: 1 Gerrit-Owner: Ali Alsuliman
Change in asterixdb[master]: [ASTERIXDB-2552][RT] Reuse objects on reseting buffer manager
Hello Anon. E. Moose (1000171), Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3372 to look at the new patch set (#2). Change subject: [ASTERIXDB-2552][RT] Reuse objects on reseting buffer manager .. [ASTERIXDB-2552][RT] Reuse objects on reseting buffer manager - user model changes: no - storage format changes: no - interface changes: yes Details: The "VariableFrameMemoryManager" releases the physical frames objects, the logical frames objects and the free slots objects on reset(). This patch is to retain those objects and only release them on close() (as also stated in the interface). - added close() to IFrameFreeSlotPolicy to make the interface similar to the other interfaces (IFrameBufferManager, IFramePool, ... etc) where close() is supposed to release while reset() is supposed to only reset but retain resources. - renamed "InMemorySortRuntimeFactory" to "MicroSortRuntimeFactory". Change-Id: I88bea27e5024c621412ef609475e2a7ba1913afa --- M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/MicroStableSortPOperator.java R hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/sort/MicroSortRuntimeFactory.java M hyracks-fullstack/algebricks/algebricks-tests/src/test/java/org/apache/hyracks/algebricks/tests/pushruntime/PushRuntimeTest.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/FrameFreeSlotBiggestFirst.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/FrameFreeSlotLastFit.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/FrameFreeSlotSmallestFit.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/IFrameBufferManager.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/IFrameFreeSlotPolicy.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/VariableDeletableTupleMemoryManager.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/VariableFrameMemoryManager.java 10 files changed, 90 insertions(+), 43 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/72/3372/2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3372 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I88bea27e5024c621412ef609475e2a7ba1913afa Gerrit-Change-Number: 3372 Gerrit-PatchSet: 2 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2552][RT] Reuse objects on reseting buffer manager
Ali Alsuliman has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/3372 Change subject: [ASTERIXDB-2552][RT] Reuse objects on reseting buffer manager .. [ASTERIXDB-2552][RT] Reuse objects on reseting buffer manager - user model changes: no - storage format changes: no - interface changes: yes Details: The "VariableFrameMemoryManager" releases the physical frames objects, the logical frames objects and the free slots objects on reset(). This patch is to retain those objects and only release them on close() (as also stated in the interface). - added close() to IFrameFreeSlotPolicy to make the interface similar to the other interfaces (IFrameBufferManager, IFramePool, ... etc) where close() is supposed to release while reset() is supposed to only reset but retain resources. - renamed "InMemorySortRuntimeFactory" to "MicroSortRuntimeFactory". Change-Id: I88bea27e5024c621412ef609475e2a7ba1913afa --- M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/MicroStableSortPOperator.java R hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/sort/MicroSortRuntimeFactory.java M hyracks-fullstack/algebricks/algebricks-tests/src/test/java/org/apache/hyracks/algebricks/tests/pushruntime/PushRuntimeTest.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/FrameFreeSlotBiggestFirst.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/FrameFreeSlotLastFit.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/FrameFreeSlotSmallestFit.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/IFrameBufferManager.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/IFrameFreeSlotPolicy.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/VariableDeletableTupleMemoryManager.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/VariableFrameMemoryManager.java 10 files changed, 90 insertions(+), 40 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/72/3372/1 diff --git a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/MicroStableSortPOperator.java b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/MicroStableSortPOperator.java index c6a7d9d..413c1a4 100644 --- a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/MicroStableSortPOperator.java +++ b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/MicroStableSortPOperator.java @@ -32,7 +32,7 @@ import org.apache.hyracks.algebricks.data.IBinaryComparatorFactoryProvider; import org.apache.hyracks.algebricks.data.INormalizedKeyComputerFactoryProvider; import org.apache.hyracks.algebricks.runtime.base.IPushRuntimeFactory; -import org.apache.hyracks.algebricks.runtime.operators.sort.InMemorySortRuntimeFactory; +import org.apache.hyracks.algebricks.runtime.operators.sort.MicroSortRuntimeFactory; import org.apache.hyracks.api.dataflow.value.IBinaryComparatorFactory; import org.apache.hyracks.api.dataflow.value.INormalizedKeyComputerFactory; import org.apache.hyracks.api.dataflow.value.RecordDescriptor; @@ -80,7 +80,7 @@ i++; } -IPushRuntimeFactory runtime = new InMemorySortRuntimeFactory(sortFields, nkcf, comps, null, maxNumberOfFrames); +IPushRuntimeFactory runtime = new MicroSortRuntimeFactory(sortFields, nkcf, comps, null, maxNumberOfFrames); builder.contributeMicroOperator(op, runtime, recDescriptor); ILogicalOperator src = op.getInputs().get(0).getValue(); builder.contributeGraphEdge(src, 0, op, 0); diff --git a/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/sort/InMemorySortRuntimeFactory.java b/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/sort/MicroSortRuntimeFactory.java similarity index 96% rename from hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/sort/InMemorySortRuntimeFactory.java rename to hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/sort/MicroSortRuntimeFactory.java index
Change in asterixdb[master]: [NO ISSUE][FUN] Move negative tests to seperate test unit
Ali Alsuliman has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/3371 Change subject: [NO ISSUE][FUN] Move negative tests to seperate test unit .. [NO ISSUE][FUN] Move negative tests to seperate test unit - user model changes: no - storage format changes: no - interface changes: no Details: Moved array functions negative tests to a seperate test unit. Change-Id: I0a3bfce402e4b7091c40483b5802db7b27afa717 --- R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_concat/array_concat.4.ddl.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.01.query.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.02.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.03.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.04.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.05.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.06.query.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.07.query.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.08.query.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.09.query.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_intersect/array_intersect.4.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_intersect/array_intersect.5.ddl.sqlpp D asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_intersect/array_intersect.5.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_range/array_range.4.ddl.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_replace/array_replace.5.ddl.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_symdiff/array_symdiff.4.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_symdiff/array_symdiff.5.ddl.sqlpp D asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_symdiff/array_symdiff.5.query.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_symdiffn/array_symdiffn.4.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_symdiffn/array_symdiffn.5.ddl.sqlpp D asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_symdiffn/array_symdiffn.5.query.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_union/array_union.4.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_union/array_union.5.ddl.sqlpp D asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_union/array_union.5.query.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml 25 files changed, 58 insertions(+), 179 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/71/3371/1 diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_concat/array_concat.5.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_concat/array_concat.4.ddl.sqlpp similarity index 100% rename from asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_concat/array_concat.5.ddl.sqlpp rename to asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_concat/array_concat.4.ddl.sqlpp diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_concat/array_concat.4.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.01.query.sqlpp similarity index 97% rename from asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_concat/array_concat.4.query.sqlpp rename to asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.01.query.sqlpp index 805dd99..a65a081 100755 --- a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_concat/array_concat.4.query.sqlpp +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.01.query.sqlpp @@ -22,6 +22,4 @@ * Expected Res : Error due to processing different list types */ -use TinySocial;
Change in asterixdb[master]: [NO ISSUE][COMP] Refactor physical window operator
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3369 ) Change subject: [NO ISSUE][COMP] Refactor physical window operator .. Patch Set 2: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3369 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3863fa3d298aef53d4098be9fc17b0451eb2c23e Gerrit-Change-Number: 3369 Gerrit-PatchSet: 2 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Wed, 01 May 2019 19:27:03 + Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][COMP] Refactor physical window operator
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3369 ) Change subject: [NO ISSUE][COMP] Refactor physical window operator .. Patch Set 1: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3369 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3863fa3d298aef53d4098be9fc17b0451eb2c23e Gerrit-Change-Number: 3369 Gerrit-PatchSet: 1 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Wed, 01 May 2019 17:20:20 + Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2552][RT] Implement micro external sort for subplans
Ali Alsuliman has submitted this change and it was merged. ( https://asterix-gerrit.ics.uci.edu/3353 ) Change subject: [ASTERIXDB-2552][RT] Implement micro external sort for subplans .. [ASTERIXDB-2552][RT] Implement micro external sort for subplans - user model changes: no - storage format changes: no - interface changes: no Details: This patch is to change the in memory sort used in subplans to be a micro external sort to avoid out of memory error. - added reset() to the runs merger to allow reusing the runs merger. - renamed "InMemoryStableSortPOperator" to "MicroStableSortPOperator". - changed the tag from "IN_MEMORY_STABLE_SORT" to "MICRO_STABLE_SORT". - added test cases. Change-Id: I930849d644c60d461d2869c9773b85e49b46fbdb Reviewed-on: https://asterix-gerrit.ics.uci.edu/3353 Contrib: Jenkins Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Dmitry Lychagin --- A asterixdb/asterix-app/src/test/resources/optimizerts/queries/micro_external_sort/micro_external_sort.sqlpp M asterixdb/asterix-app/src/test/resources/optimizerts/results/ASTERIXDB-2402.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.10.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.11.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.12.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.13.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.14.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.16.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.17.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.9.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/group-by/sugar-06-distinct.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/insert-return-custom-result.plan A asterixdb/asterix-app/src/test/resources/optimizerts/results/micro_external_sort/micro_external_sort.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/nest_aggregate.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/nested_loj3.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/query-ASTERIXDB-1263.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/query-issue785.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/similarity/jaccard-similarity-join-dual-order.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/similarity/jaccard-similarity-join-right-ahead.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/udfs/query-ASTERIXDB-1308-1.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/udfs/query-ASTERIXDB-1308-2.plan A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/sorting/micro_external_sort/micro_external_sort.1.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/sorting/micro_external_sort/micro_external_sort.1.adm M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/base/PhysicalOperatorTag.java M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/AbstractStableSortPOperator.java R hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/MicroStableSortPOperator.java M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/StableSortPOperator.java M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/EnforceStructuralPropertiesRule.java M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/PushNestedOrderByUnderPreSortedGroupByRule.java M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/SetAlgebricksPhysicalOperatorsRule.java M hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/sort/InMemorySortRuntimeFactory.java M hyracks-fullstack/algebricks/algebricks-tests/src/test/java/org/apache/hyracks/algebricks/tests/pushruntime/PushRuntimeTest.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/AbstractExternalSortRunMerger.java 33 files changed, 328 insertions(+), 101 deletions(-) Approvals: Jenkins:
Change in asterixdb[master]: [ASTERIXDB-2552][RT] Implement micro external sort for subplans
Hello Anon. E. Moose (1000171), Till Westmann, Jenkins, Michael Blow, Dmitry Lychagin, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3353 to look at the new patch set (#3). Change subject: [ASTERIXDB-2552][RT] Implement micro external sort for subplans .. [ASTERIXDB-2552][RT] Implement micro external sort for subplans - user model changes: no - storage format changes: no - interface changes: no Details: This patch is to change the in memory sort used in subplans to be a micro external sort to avoid out of memory error. - added reset() to the runs merger to allow reusing the runs merger. - renamed "InMemoryStableSortPOperator" to "MicroStableSortPOperator". - changed the tag from "IN_MEMORY_STABLE_SORT" to "MICRO_STABLE_SORT". - added test cases. Change-Id: I930849d644c60d461d2869c9773b85e49b46fbdb --- A asterixdb/asterix-app/src/test/resources/optimizerts/queries/micro_external_sort/micro_external_sort.sqlpp M asterixdb/asterix-app/src/test/resources/optimizerts/results/ASTERIXDB-2402.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.10.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.11.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.12.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.13.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.14.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.16.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.17.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.9.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/group-by/sugar-06-distinct.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/insert-return-custom-result.plan A asterixdb/asterix-app/src/test/resources/optimizerts/results/micro_external_sort/micro_external_sort.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/nest_aggregate.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/nested_loj3.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/query-ASTERIXDB-1263.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/query-issue785.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/similarity/jaccard-similarity-join-dual-order.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/similarity/jaccard-similarity-join-right-ahead.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/udfs/query-ASTERIXDB-1308-1.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/udfs/query-ASTERIXDB-1308-2.plan A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/sorting/micro_external_sort/micro_external_sort.1.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/sorting/micro_external_sort/micro_external_sort.1.adm M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/base/PhysicalOperatorTag.java M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/AbstractStableSortPOperator.java R hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/MicroStableSortPOperator.java M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/StableSortPOperator.java M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/EnforceStructuralPropertiesRule.java M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/PushNestedOrderByUnderPreSortedGroupByRule.java M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/SetAlgebricksPhysicalOperatorsRule.java M hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/sort/InMemorySortRuntimeFactory.java M hyracks-fullstack/algebricks/algebricks-tests/src/test/java/org/apache/hyracks/algebricks/tests/pushruntime/PushRuntimeTest.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/AbstractExternalSortRunMerger.java 33 files changed, 328 insertions(+), 101 deletions(-) git pull
Change in asterixdb[master]: [NO ISSUE][API] Return Bad Request on Insufficient Cluster Resources
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3368 ) Change subject: [NO ISSUE][API] Return Bad Request on Insufficient Cluster Resources .. Patch Set 1: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3368 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I15500bfa4051a97fe3f5277d318b5a1187da225c Gerrit-Change-Number: 3368 Gerrit-PatchSet: 1 Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Tue, 30 Apr 2019 18:59:09 + Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][COMP] Refactor physical operator assignment rules
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3367 ) Change subject: [NO ISSUE][COMP] Refactor physical operator assignment rules .. Patch Set 1: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3367 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I502f367464a6fabc595cff804722f793e052570f Gerrit-Change-Number: 3367 Gerrit-PatchSet: 1 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Tue, 30 Apr 2019 18:37:58 + Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2552][RT] Implement micro external sort for subplans
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3353 ) Change subject: [ASTERIXDB-2552][RT] Implement micro external sort for subplans .. Patch Set 1: (2 comments) https://asterix-gerrit.ics.uci.edu/#/c/3353/1/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/sort/MicroSortRuntimeFactory.java File hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/sort/MicroSortRuntimeFactory.java: https://asterix-gerrit.ics.uci.edu/#/c/3353/1/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/sort/MicroSortRuntimeFactory.java@94 PS1, Line 94: public void close() throws HyracksDataException { > Where do we close the "writer"? It's changed now with the refactoring. It's closed here explicitly. https://asterix-gerrit.ics.uci.edu/#/c/3353/1/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/sort/MicroSortRuntimeFactory.java@127 PS1, Line 127: } > should we call super.fail() to fail the writer? It's changed now. The writer is failed here in the wrappingWriter. -- To view, visit https://asterix-gerrit.ics.uci.edu/3353 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I930849d644c60d461d2869c9773b85e49b46fbdb Gerrit-Change-Number: 3353 Gerrit-PatchSet: 1 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Tue, 30 Apr 2019 06:04:46 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [ASTERIXDB-2552][RT] Implement micro external sort for subplans
Hello Anon. E. Moose (1000171), Till Westmann, Jenkins, Michael Blow, Dmitry Lychagin, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3353 to look at the new patch set (#2). Change subject: [ASTERIXDB-2552][RT] Implement micro external sort for subplans .. [ASTERIXDB-2552][RT] Implement micro external sort for subplans - user model changes: no - storage format changes: no - interface changes: no Details: This patch is to change the in memory sort used in subplans to be a micro external sort to avoid out of memory error. - added reset() to the runs merger to allow reusing the runs merger. - renamed "InMemoryStableSortPOperator" to "MicroStableSortPOperator". - changed the tag from "IN_MEMORY_STABLE_SORT" to "MICRO_STABLE_SORT". - added test cases. Change-Id: I930849d644c60d461d2869c9773b85e49b46fbdb --- A asterixdb/asterix-app/src/test/resources/optimizerts/queries/micro_external_sort/micro_external_sort.sqlpp M asterixdb/asterix-app/src/test/resources/optimizerts/results/ASTERIXDB-2402.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.10.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.11.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.12.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.13.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.14.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.16.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.17.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.9.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/group-by/sugar-06-distinct.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/insert-return-custom-result.plan A asterixdb/asterix-app/src/test/resources/optimizerts/results/micro_external_sort/micro_external_sort.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/nest_aggregate.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/nested_loj3.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/query-ASTERIXDB-1263.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/query-issue785.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/similarity/jaccard-similarity-join-dual-order.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/similarity/jaccard-similarity-join-right-ahead.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/udfs/query-ASTERIXDB-1308-1.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/udfs/query-ASTERIXDB-1308-2.plan A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/sorting/micro_external_sort/micro_external_sort.1.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/sorting/micro_external_sort/micro_external_sort.1.adm M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/base/PhysicalOperatorTag.java M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/AbstractStableSortPOperator.java R hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/MicroStableSortPOperator.java M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/StableSortPOperator.java M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/EnforceStructuralPropertiesRule.java M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/PushNestedOrderByUnderPreSortedGroupByRule.java M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/SetAlgebricksPhysicalOperatorsRule.java M hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/sort/InMemorySortRuntimeFactory.java M hyracks-fullstack/algebricks/algebricks-tests/src/test/java/org/apache/hyracks/algebricks/tests/pushruntime/PushRuntimeTest.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/AbstractExternalSortRunMerger.java 33 files changed, 329 insertions(+), 101 deletions(-) git pull
Change in asterixdb[master]: [ASTERIXDB-2552][RT] Refactor runs generator and merger
Ali Alsuliman has submitted this change and it was merged. ( https://asterix-gerrit.ics.uci.edu/3363 ) Change subject: [ASTERIXDB-2552][RT] Refactor runs generator and merger .. [ASTERIXDB-2552][RT] Refactor runs generator and merger - user model changes: no - storage format changes: no - interface changes: no Details: Moved the writer and sorter out of the merger to allow micro external sort to use the run generator and merger plus minor clean-ups. Change-Id: Idda31c92cbcddba5ebef8bbbf7855b9c8293dd51 Reviewed-on: https://asterix-gerrit.ics.uci.edu/3363 Contrib: Jenkins Tested-by: Jenkins Integration-Tests: Jenkins Reviewed-by: Dmitry Lychagin --- M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/group/sort/ExternalSortGroupByRunMerger.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/group/sort/SortGroupByOperatorDescriptor.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/AbstractExternalSortRunMerger.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/AbstractFrameSorter.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/AbstractSortRunGenerator.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/AbstractSorterOperatorDescriptor.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/ExternalSortOperatorDescriptor.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/ExternalSortRunMerger.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/FrameSorterMergeSort.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/FrameSorterQuickSort.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/HeapSortRunGenerator.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/HybridTopKSortRunGenerator.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/TopKSorterOperatorDescriptor.java M hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/unit/SortGroupbyTest.java 14 files changed, 209 insertions(+), 234 deletions(-) Approvals: Jenkins: Verified; ; Verified Anon. E. Moose (1000171): Dmitry Lychagin: Looks good to me, approved Objections: Jenkins: Violations found diff --git a/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/group/sort/ExternalSortGroupByRunMerger.java b/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/group/sort/ExternalSortGroupByRunMerger.java index 364f1c7..cb2da4c 100644 --- a/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/group/sort/ExternalSortGroupByRunMerger.java +++ b/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/group/sort/ExternalSortGroupByRunMerger.java @@ -32,7 +32,6 @@ import org.apache.hyracks.dataflow.std.group.IAggregatorDescriptorFactory; import org.apache.hyracks.dataflow.std.group.preclustered.PreclusteredGroupWriter; import org.apache.hyracks.dataflow.std.sort.AbstractExternalSortRunMerger; -import org.apache.hyracks.dataflow.std.sort.ISorter; /** * Group-by aggregation is pushed into multi-pass merge of external sort. @@ -44,28 +43,23 @@ private final RecordDescriptor inputRecordDesc; private final RecordDescriptor partialAggRecordDesc; private final RecordDescriptor outRecordDesc; - private final int[] groupFields; private final IAggregatorDescriptorFactory mergeAggregatorFactory; private final IAggregatorDescriptorFactory partialAggregatorFactory; private final boolean localSide; - private final int[] mergeSortFields; private final int[] mergeGroupFields; private final IBinaryComparator[] groupByComparators; -public ExternalSortGroupByRunMerger(IHyracksTaskContext ctx, ISorter frameSorter, List runs, -int[] sortFields, RecordDescriptor inRecordDesc, RecordDescriptor partialAggRecordDesc, -RecordDescriptor outRecordDesc, int framesLimit, IFrameWriter writer, int[] groupFields, -INormalizedKeyComputer nmk, IBinaryComparator[] comparators, +public ExternalSortGroupByRunMerger(IHyracksTaskContext ctx, List runs, int[] sortFields, +RecordDescriptor inRecordDesc, RecordDescriptor partialAggRecordDesc, RecordDescriptor outRecordDesc, +int framesLimit, int[]
Change in asterixdb[master]: [NO ISSUE][COMP] Refactor physical operators for GroupBy and DistinctBy
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3364 ) Change subject: [NO ISSUE][COMP] Refactor physical operators for GroupBy and DistinctBy .. Patch Set 4: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3364 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I93f17a6a3e0df6587e518e18eea01ca54a289b4c Gerrit-Change-Number: 3364 Gerrit-PatchSet: 4 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Fri, 26 Apr 2019 22:56:24 + Gerrit-HasComments: No