Change in asterixdb[master]: [ASTERIXDB-2575][RT][FUN] Remove usages of ComparisonHelper
Hello Anon. E. Moose (1000171), Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3470 to look at the new patch set (#4). Change subject: [ASTERIXDB-2575][RT][FUN] Remove usages of ComparisonHelper .. [ASTERIXDB-2575][RT][FUN] Remove usages of ComparisonHelper - user model changes: no - storage format changes: no - interface changes: no Details: Remove usages of ComparisonHelper and switch to logical or physical comparators. - updated RecordReplace test cases that used to throw an exception when comparing complex types. - cleaned up interval classes. - fixed getStartOffset() in AIntervalPointable. - removed ComparisonHelper Change-Id: Ib2a05f35c752e1d2ecfdbabbf5d303ea73c7c9db --- M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/ObjectsQueries.xml M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/object_replace/object_replace.4.query.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/object_replace/object_replace.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/objects/object_replace/object_replace.4.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/objects/object_replace/object_replace.5.adm M asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/printers/adm/AIntervalPrinterFactory.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/printers/json/clean/AIntervalPrinterFactory.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/printers/json/lossless/AIntervalPrinterFactory.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/serde/AIntervalSerializerDeserializer.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/om/pointables/nonvisitor/AIntervalPointable.java D asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/comparisons/ComparisonHelper.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/records/RecordReplaceDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/records/RecordReplaceEvaluator.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/temporal/AbstractIntervalLogicFuncDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/temporal/GetOverlappingIntervalDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/temporal/IntervalLogic.java 16 files changed, 202 insertions(+), 712 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/70/3470/4 -- To view, visit https://asterix-gerrit.ics.uci.edu/3470 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib2a05f35c752e1d2ecfdbabbf5d303ea73c7c9db Gerrit-Change-Number: 3470 Gerrit-PatchSet: 4 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2575][RT][FUN] Remove usages of ComparisonHelper
Hello Anon. E. Moose (1000171), Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3470 to look at the new patch set (#3). Change subject: [ASTERIXDB-2575][RT][FUN] Remove usages of ComparisonHelper .. [ASTERIXDB-2575][RT][FUN] Remove usages of ComparisonHelper - user model changes: no - storage format changes: no - interface changes: no Details: Remove usages of ComparisonHelper and switch to logical or physical comparators. - updated RecordReplace test cases that used to throw an exception when comparing complex types. - cleaned up interval classes. - fixed getStartOffset() in AIntervalPointable. Change-Id: Ib2a05f35c752e1d2ecfdbabbf5d303ea73c7c9db --- M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/ObjectsQueries.xml M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/object_replace/object_replace.4.query.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/object_replace/object_replace.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/objects/object_replace/object_replace.4.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/objects/object_replace/object_replace.5.adm M asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/printers/adm/AIntervalPrinterFactory.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/printers/json/clean/AIntervalPrinterFactory.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/printers/json/lossless/AIntervalPrinterFactory.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/serde/AIntervalSerializerDeserializer.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/om/pointables/nonvisitor/AIntervalPointable.java D asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/comparisons/ComparisonHelper.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/records/RecordReplaceDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/records/RecordReplaceEvaluator.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/temporal/AbstractIntervalLogicFuncDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/temporal/GetOverlappingIntervalDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/temporal/IntervalLogic.java 16 files changed, 202 insertions(+), 712 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/70/3470/3 -- To view, visit https://asterix-gerrit.ics.uci.edu/3470 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ib2a05f35c752e1d2ecfdbabbf5d303ea73c7c9db Gerrit-Change-Number: 3470 Gerrit-PatchSet: 3 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2575][RT][FUN] Remove usages of ComparisonHelper
Ali Alsuliman has uploaded this change for review. (
https://asterix-gerrit.ics.uci.edu/3470
Change subject: [ASTERIXDB-2575][RT][FUN] Remove usages of ComparisonHelper
..
[ASTERIXDB-2575][RT][FUN] Remove usages of ComparisonHelper
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
Remove usages of ComparisonHelper and switch to logical or physical
comparators.
- updated RecordReplace test cases that used to throw an exception
when comparing complex types.
- cleaned up interval classes.
- fixed getStartOffset() in AIntervalPointable.
Change-Id: Ib2a05f35c752e1d2ecfdbabbf5d303ea73c7c9db
---
M
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/ObjectsQueries.xml
M
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/object_replace/object_replace.4.query.sqlpp
M
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/object_replace/object_replace.5.query.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/results/objects/object_replace/object_replace.4.adm
A
asterixdb/asterix-app/src/test/resources/runtimets/results/objects/object_replace/object_replace.5.adm
M
asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/printers/adm/AIntervalPrinterFactory.java
M
asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/printers/json/clean/AIntervalPrinterFactory.java
M
asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/printers/json/lossless/AIntervalPrinterFactory.java
M
asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/serde/AIntervalSerializerDeserializer.java
M
asterixdb/asterix-om/src/main/java/org/apache/asterix/om/pointables/nonvisitor/AIntervalPointable.java
M
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/records/RecordReplaceDescriptor.java
M
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/records/RecordReplaceEvaluator.java
M
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/temporal/AbstractIntervalLogicFuncDescriptor.java
M
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/temporal/GetOverlappingIntervalDescriptor.java
M
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/temporal/IntervalLogic.java
15 files changed, 202 insertions(+), 231 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb
refs/changes/70/3470/1
diff --git
a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/ObjectsQueries.xml
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/ObjectsQueries.xml
index 150a37f..e37dee9c 100644
---
a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/ObjectsQueries.xml
+++
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/ObjectsQueries.xml
@@ -147,8 +147,6 @@
object_replace
- function object-replace expects its 2nd input parameter
to be of type primitive
- function object-replace expects its 2nd input parameter
to be of type primitive
false
diff --git
a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/object_replace/object_replace.4.query.sqlpp
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/object_replace/object_replace.4.query.sqlpp
index 15c0bce..9e0b525 100644
---
a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/object_replace/object_replace.4.query.sqlpp
+++
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/object_replace/object_replace.4.query.sqlpp
@@ -19,9 +19,9 @@
/*
* Description : Testing object_replace under different queries.
- * Expected Res : Failure
+ * Expected Res : Success
*/
use TinySocial;
-select value object_replace({"a":1}, {"b":2}, "z");
\ No newline at end of file
+select value object_replace({"a":1, "a2": {"b":2}, "a3": {"b2":2}, "a4":
{"b":2}}, {"b":2}, "z");
\ No newline at end of file
diff --git
a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/object_replace/object_replace.5.query.sqlpp
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/object_replace/object_replace.5.query.sqlpp
index a46a727..63008dd 100644
---
a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/object_replace/object_replace.5.query.sqlpp
+++
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/object_replace/object_replace.5.query.sqlpp
@@ -19,9 +19,9 @@
/*
* Description : Testing object_replace under different queries.
- * Expected Res : Failure
+ * Expected Res : Success
*/
use TinySocial;
-select value object_replace({"a":1},
Change in asterixdb[master]: [NO ISSUE][COMP] Fix variable handling in Intersect operator
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3460 ) Change subject: [NO ISSUE][COMP] Fix variable handling in Intersect operator .. Patch Set 1: (1 comment) https://asterix-gerrit.ics.uci.edu/#/c/3460/1/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/IntersectOperator.java File hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/IntersectOperator.java: https://asterix-gerrit.ics.uci.edu/#/c/3460/1/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/IntersectOperator.java@137 PS1, Line 137: inputExtraVars0 I wonder why they check type consistency for compare variables but not extra variables -- To view, visit https://asterix-gerrit.ics.uci.edu/3460 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4f436566e743d7e908a72bfa502c138ae7b99c6c Gerrit-Change-Number: 3460 Gerrit-PatchSet: 1 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Comment-Date: Wed, 26 Jun 2019 17:51:36 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [NO ISSUE][COMP] Fix variable handling in Intersect operator
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3460 ) Change subject: [NO ISSUE][COMP] Fix variable handling in Intersect operator .. Patch Set 1: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3460 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4f436566e743d7e908a72bfa502c138ae7b99c6c Gerrit-Change-Number: 3460 Gerrit-PatchSet: 1 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Comment-Date: Wed, 26 Jun 2019 17:51:38 + Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][COMP] Stop reusing variables of Intersect operator
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3450 ) Change subject: [NO ISSUE][COMP] Stop reusing variables of Intersect operator .. Patch Set 1: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3450 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I551819b828450760797f6b29af02aad4e124646c Gerrit-Change-Number: 3450 Gerrit-PatchSet: 1 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Comment-Date: Fri, 21 Jun 2019 17:56:37 + Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE] Fix error message thrown by ATypeHierarchy
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3441 ) Change subject: [NO ISSUE] Fix error message thrown by ATypeHierarchy .. Patch Set 2: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3441 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I830e93056d18123610b933a0e36fd1b54c6e140b Gerrit-Change-Number: 3441 Gerrit-PatchSet: 2 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Anon. E. Moose Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Comment-Date: Tue, 18 Jun 2019 20:11:48 + Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][FUN] Allow frame in RATIO_TO_REPORT()
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3449 ) Change subject: [NO ISSUE][FUN] Allow frame in RATIO_TO_REPORT() .. Patch Set 1: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3449 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibc3686bebaf791aa84ef34d8fd143e53edc7856a Gerrit-Change-Number: 3449 Gerrit-PatchSet: 1 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Comment-Date: Tue, 18 Jun 2019 04:44:43 + Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][OTH] Use Request Stats When Printing Results on NC
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3444 ) Change subject: [NO ISSUE][OTH] Use Request Stats When Printing Results on NC .. Patch Set 2: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3444 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I54c3b3e72cfb0c2499a95bfc0f514dd5d6774293 Gerrit-Change-Number: 3444 Gerrit-PatchSet: 2 Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Hussain Towaileb Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Murtadha Hubail Gerrit-Comment-Date: Tue, 18 Jun 2019 04:43:39 + Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][COMP] Do not inline non-functional LET clauses
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3436 ) Change subject: [NO ISSUE][COMP] Do not inline non-functional LET clauses .. Patch Set 2: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3436 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3740745e2e8c8a0bb11aa7908e68c682dd9a5553 Gerrit-Change-Number: 3436 Gerrit-PatchSet: 2 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Comment-Date: Mon, 17 Jun 2019 21:31:48 + Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][FUN] Allow ORDER BY in RATIO_TO_REPORT()
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3442 ) Change subject: [NO ISSUE][FUN] Allow ORDER BY in RATIO_TO_REPORT() .. Patch Set 1: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3442 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I031b3eff1aee8be12f1943d39f5f1c640476e4e8 Gerrit-Change-Number: 3442 Gerrit-PatchSet: 1 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Comment-Date: Sat, 15 Jun 2019 20:28:33 + Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][FUN] Remove unused scalar aggregates
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3429 ) Change subject: [NO ISSUE][FUN] Remove unused scalar aggregates .. Patch Set 5: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3429 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2c1274876c884f16bcbee0e3f7aaba4d91b271ce Gerrit-Change-Number: 3429 Gerrit-PatchSet: 5 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Comment-Date: Tue, 11 Jun 2019 21:53:31 + Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2458][COMP] Fix min/max functions with group by
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3434 ) Change subject: [ASTERIXDB-2458][COMP] Fix min/max functions with group by .. Patch Set 2: Code-Review+1 -- To view, visit https://asterix-gerrit.ics.uci.edu/3434 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie0551b091b9adbbbd51158dbd36124a7184bdce0 Gerrit-Change-Number: 3434 Gerrit-PatchSet: 2 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Mon, 10 Jun 2019 21:52:45 + Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2458][COMP] Fix min/max functions with group by
Hello Anon. E. Moose (1000171), Till Westmann, Jenkins, Dmitry Lychagin, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3434 to look at the new patch set (#2). Change subject: [ASTERIXDB-2458][COMP] Fix min/max functions with group by .. [ASTERIXDB-2458][COMP] Fix min/max functions with group by - user model changes: no - storage format changes: no - interface changes: no Details: In a sort-group-by operator, an aggregate function has a merging aggregate function that will be used when merging the run files if they were generated. The merging aggregate function of the local min/max aggregate function should be different from the merging aggregate function of the global min/max. For local min/max, the merging aggregate function should propagate system_null if the aggregation is system_null. Also, the global min/max should handle finishPartial() different from finish(). finishPartial() should not output NULL if the aggregation is system_null since the aggregation is still going on. - added functions to be intermediate steps for local aggregation. - implemented logic for finishPartial() - added test cases for strict_min/max with group by Change-Id: Ie0551b091b9adbbbd51158dbd36124a7184bdce0 --- M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.01.ddl.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.02.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.31.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.32.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.33.ddl.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate/min_max_arrays/min_max_arrays.01.ddl.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate/min_max_arrays/min_max_arrays.02.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate/min_max_arrays/min_max_arrays.23.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate/min_max_arrays/min_max_arrays.24.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate/min_max_arrays/min_max_arrays.25.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate/min_max_arrays/min_max_arrays.26.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate/min_max_arrays/min_max_arrays.27.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate/min_max_arrays/min_max_arrays.28.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate/min_max_arrays/min_max_arrays.29.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate/min_max_arrays/min_max_arrays.30.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate/min_max_arrays/min_max_arrays.31.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate/min_max_arrays/min_max_arrays.32.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate/min_max_arrays/min_max_arrays.33.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate/min_max_arrays/min_max_arrays.34.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate/min_max_arrays/min_max_arrays.35.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate/min_max_arrays/min_max_arrays.36.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/min_max_arrays/min_max_arrays.31.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/min_max_arrays/min_max_arrays.32.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/min_max_arrays/min_max_arrays.23.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/min_max_arrays/min_max_arrays.24.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/min_max_arrays/min_max_arrays.25.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/min_max_arrays/min_max_arrays.26.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/min_max_arrays/min_max_arrays.27.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/min_max_arrays/min_max_arrays.28.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate/min_max_arrays/min_max_arrays.29.adm A
Change in asterixdb[master]: [ASTERIXDB-2458][COMP] Fix min/max functions with group by
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3434 ) Change subject: [ASTERIXDB-2458][COMP] Fix min/max functions with group by .. Patch Set 1: Code-Review+1 -- To view, visit https://asterix-gerrit.ics.uci.edu/3434 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie0551b091b9adbbbd51158dbd36124a7184bdce0 Gerrit-Change-Number: 3434 Gerrit-PatchSet: 1 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Comment-Date: Mon, 10 Jun 2019 17:08:15 + Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2458][COMP] Fix min/max functions with group by
Ali Alsuliman has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/3434 Change subject: [ASTERIXDB-2458][COMP] Fix min/max functions with group by .. [ASTERIXDB-2458][COMP] Fix min/max functions with group by - user model changes: no - storage format changes: no - interface changes: no Details: In a sort-group-by operator, an aggregate function has a merging aggregate function that will be used when merging the run files if they were generated. The merging aggregate function of the local min/max aggregate function should be different from the merging aggregate function of the global min/max. For local min/max, the merging aggregate function should propagate system_null if the aggregation is system_null. Also, the global min/max should handle finishPartial() different from finish(). finishPartial() should not output NULL if the aggregation is system_null since the aggregation is still going on. - added functions to be intermediate steps for local aggregation. - implemented logic for finishPartial() Change-Id: Ie0551b091b9adbbbd51158dbd36124a7184bdce0 --- M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.01.ddl.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.02.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.31.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.32.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.33.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/min_max_arrays/min_max_arrays.31.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/aggregate-sql/min_max_arrays/min_max_arrays.32.adm M asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/BuiltinFunctions.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractMinMaxAggregateFunction.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/IntermediateMaxAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/IntermediateMinAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/IntermediateSqlMaxAggregateDescriptor.java A asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/IntermediateSqlMinAggregateDescriptor.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/MinMaxAggregateFunction.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/SqlMinMaxAggregateFunction.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/functions/FunctionCollection.java M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/AbstractIntroduceCombinerRule.java 17 files changed, 12,878 insertions(+), 21 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/34/3434/1 -- To view, visit https://asterix-gerrit.ics.uci.edu/3434 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ie0551b091b9adbbbd51158dbd36124a7184bdce0 Gerrit-Change-Number: 3434 Gerrit-PatchSet: 1 Gerrit-Owner: Ali Alsuliman
Change in asterixdb[master]: [ASTERIXDB-2574][COMP] Fix min/max functions
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3427 ) Change subject: [ASTERIXDB-2574][COMP] Fix min/max functions .. Patch Set 2: (1 comment) https://asterix-gerrit.ics.uci.edu/#/c/3427/2/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/types/AbstractCollectionType.java File asterixdb/asterix-om/src/main/java/org/apache/asterix/om/types/AbstractCollectionType.java: https://asterix-gerrit.ics.uci.edu/#/c/3427/2/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/types/AbstractCollectionType.java@42 PS2, Line 42: // TODO(ali): this should be removed. itemType is already enforced to be NonNull > I think we need to investigate the intent of this method. May be we should agreed. -- To view, visit https://asterix-gerrit.ics.uci.edu/3427 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1231cfe558099d167bae0b2fa7fc4879b756baf0 Gerrit-Change-Number: 3427 Gerrit-PatchSet: 2 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Thu, 06 Jun 2019 21:24:52 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [NO ISSUE][SQLPP] Support SELECT LET
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3428 ) Change subject: [NO ISSUE][SQLPP] Support SELECT LET .. Patch Set 2: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3428 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1226dca83e756075608232c642851f646a9bee3b Gerrit-Change-Number: 3428 Gerrit-PatchSet: 2 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Thu, 06 Jun 2019 20:56:21 + Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2574][COMP] Fix min/max functions
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3427 ) Change subject: [ASTERIXDB-2574][COMP] Fix min/max functions .. Patch Set 2: (1 comment) https://asterix-gerrit.ics.uci.edu/#/c/3427/2/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarArrayAggAggregateDescriptor.java File asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarArrayAggAggregateDescriptor.java: https://asterix-gerrit.ics.uci.edu/#/c/3427/2/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/ScalarArrayAggAggregateDescriptor.java@47 PS2, Line 47: itemType); check this just to make sure. -- To view, visit https://asterix-gerrit.ics.uci.edu/3427 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1231cfe558099d167bae0b2fa7fc4879b756baf0 Gerrit-Change-Number: 3427 Gerrit-PatchSet: 2 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Thu, 06 Jun 2019 02:00:20 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [ASTERIXDB-2574][COMP] Fix min/max functions
Hello Anon. E. Moose (1000171), Till Westmann, Jenkins, Dmitry Lychagin, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3427 to look at the new patch set (#2). Change subject: [ASTERIXDB-2574][COMP] Fix min/max functions .. [ASTERIXDB-2574][COMP] Fix min/max functions - user model changes: no - storage format changes: no - interface changes: no This change is mainly for 2 things. The first thing is to not throw an exception when the type of the aggregated field is invalid for min/max (e.g. record or rectange) or min/max get incompatible data like string and int. The result in this case would be NULL. The second thing is to enable comparing ARRAYs correctly by using logical comparison. When a partition runs into type invalidity, it will output NULL. The global aggregator interprets NULL received from a partition as type invalidity and outputs NULL as the final result. Both SQL and SQL++ will do that. Special treatment is needed for scalar and distinct version of SQL since SQL min/max ignores NULL values and continue aggregation and the scalar and distinct version of SQL are normally setup as a global aggregator since they behave like the global aggregator in a two-step aggregation. Currently, there is only a local min and max functions. The other min/max functions are used for everything, the global function of two-step aggregation, and for scalar and distinct min/max. In order to differentiate, a global min/max functions are added that will be used for the two-step aggregation. Details: - fixed listify to open up elements when adding them to the collection and the collection item type is of type ANY and changed the type inferer of listify to enable that. - fixed AbstractCollectionType to make sure itemType is never null. - changed MinMaxAggTypeComputer to not throw an exception but return NULL for invalid types. - changed min/max descriptors to implement inferer to propagate the type of the field and pass that when getting a comparator. - switched min/max comparison to the logical comparison. - refactored method inequalityUndefined to be shared by logical comparison and min/max functions. - added global max/min functions to enable differentiating between scalar min/max, distinct min/max and two-step min/max (global & local). - code clean-up for LogicalScalarBinaryComparator; created two INSTANCES and re-used. Change-Id: I1231cfe558099d167bae0b2fa7fc4879b756baf0 --- A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.01.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.02.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.03.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.04.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.05.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.06.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.07.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.08.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.09.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.10.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.11.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.12.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.13.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.14.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.15.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.16.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.17.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.18.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/aggregate-sql/min_max_arrays/min_max_arrays.19.query.sqlpp A
Change in asterixdb[master]: [NO ISSUE][SQLPP] Optional field name in object constructor
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3424 ) Change subject: [NO ISSUE][SQLPP] Optional field name in object constructor .. Patch Set 3: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3424 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0dfba8f8137f88d934a5be8ffb5fbc1c1db0bb58 Gerrit-Change-Number: 3424 Gerrit-PatchSet: 3 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Wed, 05 Jun 2019 15:44:45 + Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][SQLPP] Optional field name in object constructor
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3424 ) Change subject: [NO ISSUE][SQLPP] Optional field name in object constructor .. Patch Set 3: (1 comment) https://asterix-gerrit.ics.uci.edu/#/c/3424/3/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/no_fieldname_constr/no_fieldname_constr.1.query.sqlpp File asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/no_fieldname_constr/no_fieldname_constr.1.query.sqlpp: https://asterix-gerrit.ics.uci.edu/#/c/3424/3/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/objects/no_fieldname_constr/no_fieldname_constr.1.query.sqlpp@25 PS3, Line 25: from range(1, 2) x, range(3, 4) y it would be nice to have a test where there is one nested field access whose last identifier matches another field access. -- To view, visit https://asterix-gerrit.ics.uci.edu/3424 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0dfba8f8137f88d934a5be8ffb5fbc1c1db0bb58 Gerrit-Change-Number: 3424 Gerrit-PatchSet: 3 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Wed, 05 Jun 2019 15:41:56 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [NO ISSUE][FUN] Support FROM LAST in NTH_VALUE()
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3419 ) Change subject: [NO ISSUE][FUN] Support FROM LAST in NTH_VALUE() .. Patch Set 2: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3419 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iceac579bd5a3e651bcd7707e324148690e020cf5 Gerrit-Change-Number: 3419 Gerrit-PatchSet: 2 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Tue, 04 Jun 2019 06:39:25 + Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][SQLPP] Improve error message for unsupported EXPLAIN
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3420 ) Change subject: [NO ISSUE][SQLPP] Improve error message for unsupported EXPLAIN .. Patch Set 2: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3420 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idd7844ca3d9e3c2c0de0b070e4a76282cb96e278 Gerrit-Change-Number: 3420 Gerrit-PatchSet: 2 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Tue, 04 Jun 2019 03:29:09 + Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][FUN] Support IGNORE NULLS in window functions
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3407 ) Change subject: [NO ISSUE][FUN] Support IGNORE NULLS in window functions .. Patch Set 2: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3407 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I400c9c95bac9159efa17fca4e97047fd089eb931 Gerrit-Change-Number: 3407 Gerrit-PatchSet: 2 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Wed, 29 May 2019 20:00:32 + Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][FUN] Support IGNORE NULLS in window functions
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3407 ) Change subject: [NO ISSUE][FUN] Support IGNORE NULLS in window functions .. Patch Set 2: (1 comment) https://asterix-gerrit.ics.uci.edu/#/c/3407/2/asterixdb/asterix-common/src/main/resources/asx_errormsg/en.properties File asterixdb/asterix-common/src/main/resources/asx_errormsg/en.properties: https://asterix-gerrit.ics.uci.edu/#/c/3407/2/asterixdb/asterix-common/src/main/resources/asx_errormsg/en.properties@182 PS2, Line 182: 1104 = Invalid modifier %1$s for function %2$s let's check if we need to add a mapping. -- To view, visit https://asterix-gerrit.ics.uci.edu/3407 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I400c9c95bac9159efa17fca4e97047fd089eb931 Gerrit-Change-Number: 3407 Gerrit-PatchSet: 2 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Wed, 29 May 2019 20:00:30 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [NO ISSUE][CONF] Mark KeyStore Pass As Hidden Config
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3413 ) Change subject: [NO ISSUE][CONF] Mark KeyStore Pass As Hidden Config .. Patch Set 1: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3413 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1c27c3b229ca7d8dd44717c21cd096f0090a7278 Gerrit-Change-Number: 3413 Gerrit-PatchSet: 1 Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Wed, 29 May 2019 16:35:12 + Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2547][COMP] Disallow passing UNION tag to get comparator
Ali Alsuliman has submitted this change and it was merged. (
https://asterix-gerrit.ics.uci.edu/3404 )
Change subject: [ASTERIXDB-2547][COMP] Disallow passing UNION tag to get
comparator
..
[ASTERIXDB-2547][COMP] Disallow passing UNION tag to get comparator
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
UNION should not be allowed when getting a comparator by tag.
The comparator provider returns a generic comparator with types
ANY when UNION is passed as a tag. This could cause problems if
the actual type is a complex type. UNION should not be allowed.
Change-Id: Id8816a0dc5584f0a27410c512f3a44ccfc6c3151
Reviewed-on: https://asterix-gerrit.ics.uci.edu/3404
Contrib: Jenkins
Integration-Tests: Jenkins
Tested-by: Jenkins
Reviewed-by: Dmitry Lychagin
---
M
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/LangExpressionToPlanTranslator.java
M
asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/util/RangeMapBuilder.java
M
asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/nontagged/BinaryComparatorFactoryProvider.java
3 files changed, 23 insertions(+), 12 deletions(-)
Approvals:
Jenkins: Verified; ; Verified
Anon. E. Moose (1000171):
Dmitry Lychagin: Looks good to me, approved
Objections:
Jenkins: Violations found
diff --git
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/LangExpressionToPlanTranslator.java
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/LangExpressionToPlanTranslator.java
index b8bfffd..c6c8a2f 100644
---
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/LangExpressionToPlanTranslator.java
+++
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/LangExpressionToPlanTranslator.java
@@ -1252,7 +1252,7 @@
if (oc.getRangeMap() != null) {
Iterator orderModifIter =
oc.getModifierList().iterator();
boolean ascending = orderModifIter.next() == OrderModifier.ASC;
-RangeMapBuilder.verifyRangeOrder(oc.getRangeMap(), ascending);
+RangeMapBuilder.verifyRangeOrder(oc.getRangeMap(), ascending,
sourceLoc);
ord.getAnnotations().put(OperatorAnnotations.USE_STATIC_RANGE,
oc.getRangeMap());
}
return new Pair<>(ord, null);
diff --git
a/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/util/RangeMapBuilder.java
b/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/util/RangeMapBuilder.java
index c505c1c..a26c94d 100644
---
a/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/util/RangeMapBuilder.java
+++
b/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/util/RangeMapBuilder.java
@@ -23,6 +23,7 @@
import org.apache.asterix.common.exceptions.CompilationException;
import org.apache.asterix.common.exceptions.ErrorCode;
+import org.apache.asterix.common.exceptions.RuntimeDataException;
import org.apache.asterix.formats.nontagged.BinaryComparatorFactoryProvider;
import org.apache.asterix.formats.nontagged.SerializerDeserializerProvider;
import org.apache.asterix.lang.common.base.Expression;
@@ -50,6 +51,7 @@
import org.apache.hyracks.api.dataflow.value.IBinaryComparatorFactory;
import org.apache.hyracks.api.dataflow.value.ISerializerDeserializer;
import org.apache.hyracks.api.exceptions.HyracksDataException;
+import org.apache.hyracks.api.exceptions.SourceLocation;
import org.apache.hyracks.data.std.util.ArrayBackedValueStorage;
import org.apache.hyracks.dataflow.common.data.partition.range.RangeMap;
@@ -145,19 +147,25 @@
}
}
-public static void verifyRangeOrder(RangeMap rangeMap, boolean ascending)
throws CompilationException {
+public static void verifyRangeOrder(RangeMap rangeMap, boolean ascending,
SourceLocation sourceLoc)
+throws CompilationException {
// TODO Add support for composite fields.
int fieldIndex = 0;
int fieldType = rangeMap.getTag(0, 0);
BinaryComparatorFactoryProvider comparatorFactory =
BinaryComparatorFactoryProvider.INSTANCE;
-IBinaryComparatorFactory bcf =
-
comparatorFactory.getBinaryComparatorFactory(ATypeTag.VALUE_TYPE_MAPPING[fieldType],
ascending);
+IBinaryComparatorFactory bcf;
+try {
+bcf =
comparatorFactory.getBinaryComparatorFactory(ATypeTag.VALUE_TYPE_MAPPING[fieldType],
ascending);
+} catch (RuntimeDataException e) {
+throw new CompilationException(ErrorCode.COMPILATION_ERROR,
sourceLoc, e.getMessage());
+}
IBinaryComparator comparator = bcf.createBinaryComparator();
int c = 0;
for (int split = 1; split < rangeMap.getSplitCount(); ++split) {
if (fieldType != rangeMap.getTag(fieldIndex, split)) {
-
Change in asterixdb[master]: [ASTERIXDB-2547][COMP] Disallow passing UNION tag to get comparator
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3404 ) Change subject: [ASTERIXDB-2547][COMP] Disallow passing UNION tag to get comparator .. Patch Set 2: (1 comment) https://asterix-gerrit.ics.uci.edu/#/c/3404/2/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/util/RangeMapBuilder.java File asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/util/RangeMapBuilder.java: https://asterix-gerrit.ics.uci.edu/#/c/3404/2/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/util/RangeMapBuilder.java@180 PS2, Line 180: " + " > Can we remove this string concatenation? Done -- To view, visit https://asterix-gerrit.ics.uci.edu/3404 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id8816a0dc5584f0a27410c512f3a44ccfc6c3151 Gerrit-Change-Number: 3404 Gerrit-PatchSet: 2 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Tue, 28 May 2019 03:07:37 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [ASTERIXDB-2547][COMP] Disallow passing UNION tag to get comparator
Hello Anon. E. Moose (1000171), Till Westmann, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3404 to look at the new patch set (#3). Change subject: [ASTERIXDB-2547][COMP] Disallow passing UNION tag to get comparator .. [ASTERIXDB-2547][COMP] Disallow passing UNION tag to get comparator - user model changes: no - storage format changes: no - interface changes: no Details: UNION should not be allowed when getting a comparator by tag. The comparator provider returns a generic comparator with types ANY when UNION is passed as a tag. This could cause problems if the actual type is a complex type. UNION should not be allowed. Change-Id: Id8816a0dc5584f0a27410c512f3a44ccfc6c3151 --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/LangExpressionToPlanTranslator.java M asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/util/RangeMapBuilder.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/nontagged/BinaryComparatorFactoryProvider.java 3 files changed, 23 insertions(+), 12 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/04/3404/3 -- To view, visit https://asterix-gerrit.ics.uci.edu/3404 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id8816a0dc5584f0a27410c512f3a44ccfc6c3151 Gerrit-Change-Number: 3404 Gerrit-PatchSet: 3 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann
Change in asterixdb[master]: [ASTERIXDB-2458][COMP] Fix InjectTypeCastForFunctionArgumentsRule
Ali Alsuliman has submitted this change and it was merged. (
https://asterix-gerrit.ics.uci.edu/3406 )
Change subject: [ASTERIXDB-2458][COMP] Fix
InjectTypeCastForFunctionArgumentsRule
..
[ASTERIXDB-2458][COMP] Fix InjectTypeCastForFunctionArgumentsRule
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
InjectTypeCastForFunctionArgumentsRule is for functions that
can potentially return any of their arguments. switch and
if_null(expr1, expr2, ...) are examples. All the arguments
need to be casted (opened) to the type that the function
will return which is the generalized type of all arguments.
Some functions like if_null can determine the exact expression
they will return, e.g. if_null(1, {"id": 3}) in which case
the return type is always integer. The rule tries to cast
th 2nd argument, the record, to integer and fails. In such
cases, these functions do not need to cast their arguments.
If the function determines its output type to be ANY, then
all arguments need to be casted (opened). If the function
determines its output to be a dervied type, then casting is
also needed since that output type should be the generalized
type of all arguments.
Change-Id: I2fee234d883b59319e4ec4df58d61ecd498373fd
Reviewed-on: https://asterix-gerrit.ics.uci.edu/3406
Contrib: Jenkins
Sonar-Qube: Jenkins
Tested-by: Jenkins
Integration-Tests: Jenkins
Reviewed-by: Dmitry Lychagin
---
M
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/InjectTypeCastForFunctionArgumentsRule.java
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/cast-ASTERIXDB-2458/cast-ASTERIXDB-2458.1.query.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/results/misc/cast-ASTERIXDB-2458/cast-ASTERIXDB-2458.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M
asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/common/TypeResolverUtil.java
5 files changed, 64 insertions(+), 22 deletions(-)
Approvals:
Jenkins: Verified; No violations found; ; Verified
Anon. E. Moose (1000171):
Dmitry Lychagin: Looks good to me, approved
diff --git
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/InjectTypeCastForFunctionArgumentsRule.java
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/InjectTypeCastForFunctionArgumentsRule.java
index 3678826..77d63f1 100644
---
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/InjectTypeCastForFunctionArgumentsRule.java
+++
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/InjectTypeCastForFunctionArgumentsRule.java
@@ -24,12 +24,15 @@
import java.util.HashMap;
import java.util.List;
import java.util.Map;
+import java.util.function.IntBinaryOperator;
import java.util.function.IntPredicate;
import org.apache.asterix.dataflow.data.common.TypeResolverUtil;
import org.apache.asterix.lang.common.util.FunctionUtil;
import org.apache.asterix.om.functions.BuiltinFunctions;
import org.apache.asterix.om.typecomputer.base.TypeCastUtils;
+import org.apache.asterix.om.typecomputer.impl.TypeComputeUtils;
+import org.apache.asterix.om.types.ATypeTag;
import org.apache.asterix.om.types.IAType;
import org.apache.commons.lang3.mutable.Mutable;
import org.apache.commons.lang3.mutable.MutableObject;
@@ -103,39 +106,27 @@
}
FunctionIdentifier funcId = func.getFunctionIdentifier();
if (funcId.equals(BuiltinFunctions.SWITCH_CASE)) {
-rewritten |= rewriteSwitchCase(op, func, context);
+rewritten |= rewriteFunction(op, func, null, context, 2,
+InjectTypeCastForFunctionArgumentsRule::switchIncrement);
} else if (FUN_TO_ARG_CHECKER.containsKey(funcId)) {
-rewritten |= rewriteFunction(op, func,
FUN_TO_ARG_CHECKER.get(funcId), context);
-}
-return rewritten;
-}
-
-// Injects casts that cast types for different "THEN" and "ELSE" branches.
-private boolean rewriteSwitchCase(ILogicalOperator op,
AbstractFunctionCallExpression func,
-IOptimizationContext context) throws AlgebricksException {
-IVariableTypeEnvironment env = op.computeInputTypeEnvironment(context);
-IAType producedType = (IAType) env.getType(func);
-List> argRefs = func.getArguments();
-int argSize = argRefs.size();
-boolean rewritten = false;
-for (int argIndex = 2; argIndex < argSize; argIndex += (argIndex + 2
== argSize) ? 1 : 2) {
-Mutable argRef = argRefs.get(argIndex);
-if (rewriteFunctionArgument(argRef, producedType, env)) {
-rewritten = true;
-}
+rewritten |= rewriteFunction(op, func,
FUN_TO_ARG_CHECKER.get(funcId), context, 0,
+InjectTypeCastForFunctionArgumentsRule::increment);
}
Change in asterixdb[master]: [ASTERIXDB-2547][COMP] Disallow passing UNION tag to get comparator
Hello Anon. E. Moose (1000171), Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3404 to look at the new patch set (#2). Change subject: [ASTERIXDB-2547][COMP] Disallow passing UNION tag to get comparator .. [ASTERIXDB-2547][COMP] Disallow passing UNION tag to get comparator - user model changes: no - storage format changes: no - interface changes: no Details: UNION should not be allowed when getting a comparator by tag. The comparator provider returns a generic comparator with types ANY when UNION is passed as a tag. This could cause problems if the actual type is a complex type. UNION should not be allowed. Change-Id: Id8816a0dc5584f0a27410c512f3a44ccfc6c3151 --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/LangExpressionToPlanTranslator.java M asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/util/RangeMapBuilder.java M asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/nontagged/BinaryComparatorFactoryProvider.java 3 files changed, 23 insertions(+), 12 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/04/3404/2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3404 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id8816a0dc5584f0a27410c512f3a44ccfc6c3151 Gerrit-Change-Number: 3404 Gerrit-PatchSet: 2 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2458][COMP] Fix InjectTypeCastForFunctionArgumentsRule
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3406 ) Change subject: [ASTERIXDB-2458][COMP] Fix InjectTypeCastForFunctionArgumentsRule .. Patch Set 3: (1 comment) https://asterix-gerrit.ics.uci.edu/#/c/3406/2/asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/common/TypeResolverUtil.java File asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/common/TypeResolverUtil.java: https://asterix-gerrit.ics.uci.edu/#/c/3406/2/asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/common/TypeResolverUtil.java@98 PS2, Line 98: // Generalizes two input types. > This method's name can be confusing. Primitive types are in general castabl Done -- To view, visit https://asterix-gerrit.ics.uci.edu/3406 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2fee234d883b59319e4ec4df58d61ecd498373fd Gerrit-Change-Number: 3406 Gerrit-PatchSet: 3 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Fri, 24 May 2019 19:02:02 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [ASTERIXDB-2458][COMP] Fix InjectTypeCastForFunctionArgumentsRule
Hello Anon. E. Moose (1000171), Till Westmann, Jenkins, Dmitry Lychagin,
I'd like you to reexamine a change. Please visit
https://asterix-gerrit.ics.uci.edu/3406
to look at the new patch set (#3).
Change subject: [ASTERIXDB-2458][COMP] Fix
InjectTypeCastForFunctionArgumentsRule
..
[ASTERIXDB-2458][COMP] Fix InjectTypeCastForFunctionArgumentsRule
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
InjectTypeCastForFunctionArgumentsRule is for functions that
can potentially return any of their arguments. switch and
if_null(expr1, expr2, ...) are examples. All the arguments
need to be casted (opened) to the type that the function
will return which is the generalized type of all arguments.
Some functions like if_null can determine the exact expression
they will return, e.g. if_null(1, {"id": 3}) in which case
the return type is always integer. The rule tries to cast
th 2nd argument, the record, to integer and fails. In such
cases, these functions do not need to cast their arguments.
If the function determines its output type to be ANY, then
all arguments need to be casted (opened). If the function
determines its output to be a dervied type, then casting is
also needed since that output type should be the generalized
type of all arguments.
Change-Id: I2fee234d883b59319e4ec4df58d61ecd498373fd
---
M
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/InjectTypeCastForFunctionArgumentsRule.java
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/cast-ASTERIXDB-2458/cast-ASTERIXDB-2458.1.query.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/results/misc/cast-ASTERIXDB-2458/cast-ASTERIXDB-2458.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M
asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/common/TypeResolverUtil.java
5 files changed, 64 insertions(+), 22 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb
refs/changes/06/3406/3
--
To view, visit https://asterix-gerrit.ics.uci.edu/3406
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2fee234d883b59319e4ec4df58d61ecd498373fd
Gerrit-Change-Number: 3406
Gerrit-PatchSet: 3
Gerrit-Owner: Ali Alsuliman
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Dmitry Lychagin
Gerrit-Reviewer: Jenkins
Gerrit-Reviewer: Till Westmann
Change in asterixdb[master]: [ASTERIXDB-2458][COMP] Fix InjectTypeCastForFunctionArgumentsRule
Hello Anon. E. Moose (1000171), Jenkins,
I'd like you to reexamine a change. Please visit
https://asterix-gerrit.ics.uci.edu/3406
to look at the new patch set (#2).
Change subject: [ASTERIXDB-2458][COMP] Fix
InjectTypeCastForFunctionArgumentsRule
..
[ASTERIXDB-2458][COMP] Fix InjectTypeCastForFunctionArgumentsRule
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
InjectTypeCastForFunctionArgumentsRule is for functions that
can potentially return any of their arguments. switch and
if_null(expr1, expr2, ...) are examples. All the arguments
need to be casted (opened) to the type that the function
will return which is the generalized type of all arguments.
Some functions like if_null can determine the exact expression
they will return, e.g. if_null(1, {"id": 3}) in which case
the return type is always integer. The rule tries to cast
th 2nd argument, the record, to integer and fails. In such
cases, these functions do not need to cast their arguments.
If the function determines its output type to be ANY, then
all arguments need to be casted (opened). If the function
determines its output to be a dervied type, then casting is
also needed since that output type should be the generalized
type of all arguments.
Change-Id: I2fee234d883b59319e4ec4df58d61ecd498373fd
---
M
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/InjectTypeCastForFunctionArgumentsRule.java
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/cast-ASTERIXDB-2458/cast-ASTERIXDB-2458.1.query.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/results/misc/cast-ASTERIXDB-2458/cast-ASTERIXDB-2458.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M
asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/common/TypeResolverUtil.java
5 files changed, 71 insertions(+), 22 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb
refs/changes/06/3406/2
--
To view, visit https://asterix-gerrit.ics.uci.edu/3406
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2fee234d883b59319e4ec4df58d61ecd498373fd
Gerrit-Change-Number: 3406
Gerrit-PatchSet: 2
Gerrit-Owner: Ali Alsuliman
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2547][COMP] Disallow passing UNION tag to get comparator
Ali Alsuliman has uploaded this change for review. (
https://asterix-gerrit.ics.uci.edu/3404
Change subject: [ASTERIXDB-2547][COMP] Disallow passing UNION tag to get
comparator
..
[ASTERIXDB-2547][COMP] Disallow passing UNION tag to get comparator
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
UNION should not be allowed when getting a comparator by tag.
The comparator provider returns a generic comparator with types
ANY when UNION is passed as a tag. This could cause problems if
the actual type is a complex type. UNION should not be allowed.
Change-Id: Id8816a0dc5584f0a27410c512f3a44ccfc6c3151
---
M
asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/nontagged/BinaryComparatorFactoryProvider.java
1 file changed, 3 insertions(+), 3 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb
refs/changes/04/3404/1
diff --git
a/asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/nontagged/BinaryComparatorFactoryProvider.java
b/asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/nontagged/BinaryComparatorFactoryProvider.java
index 396bf3b..5f59a41 100644
---
a/asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/nontagged/BinaryComparatorFactoryProvider.java
+++
b/asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/nontagged/BinaryComparatorFactoryProvider.java
@@ -35,6 +35,7 @@
import org.apache.asterix.om.types.ATypeTag;
import org.apache.asterix.om.types.BuiltinType;
import org.apache.asterix.om.types.IAType;
+import org.apache.hyracks.algebricks.common.exceptions.NotImplementedException;
import org.apache.hyracks.algebricks.data.IBinaryComparatorFactoryProvider;
import org.apache.hyracks.api.dataflow.value.IBinaryComparatorFactory;
import org.apache.hyracks.data.std.accessors.BooleanBinaryComparatorFactory;
@@ -115,10 +116,9 @@
public IBinaryComparatorFactory getBinaryComparatorFactory(ATypeTag type,
boolean ascending) {
switch (type) {
case ANY:
-case UNION:
-// i think UNION shouldn't be allowed. the actual type could
be closed array or record. ANY would fail.
-// we could do smth better for nullable fields
return createGenericBinaryComparatorFactory(BuiltinType.ANY,
BuiltinType.ANY, ascending);
+case UNION:
+throw new NotImplementedException("No comparator for type
UNION");
case NULL:
case MISSING:
return new AnyBinaryComparatorFactory();
--
To view, visit https://asterix-gerrit.ics.uci.edu/3404
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Id8816a0dc5584f0a27410c512f3a44ccfc6c3151
Gerrit-Change-Number: 3404
Gerrit-PatchSet: 1
Gerrit-Owner: Ali Alsuliman
Change in asterixdb[master]: [ASTERIXDB-2288][COMP] Fix field access with UNION
Ali Alsuliman has submitted this change and it was merged. ( https://asterix-gerrit.ics.uci.edu/3400 ) Change subject: [ASTERIXDB-2288][COMP] Fix field access with UNION .. [ASTERIXDB-2288][COMP] Fix field access with UNION - user model changes: no - storage format changes: no - interface changes: no Details: For UNION, if there is an assign operator above UNION, the optimizer will move the assign operator under UNION into both branches. If the assign operator has field access by index, the index is based on the output record of the union operator which could be different from the field index in left and right branches. The fix is to map the index to the correct index of the input operator where the assign is copied to. Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 Reviewed-on: https://asterix-gerrit.ics.uci.edu/3400 Contrib: Jenkins Tested-by: Jenkins Integration-Tests: Jenkins Reviewed-by: Dmitry Lychagin --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java A asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/AsterixPushAssignBelowUnionAllRule.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.7.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/PushAssignBelowUnionAllRule.java 15 files changed, 397 insertions(+), 21 deletions(-) Approvals: Jenkins: Verified; ; Verified Anon. E. Moose (1000171): Dmitry Lychagin: Looks good to me, approved Objections: Jenkins: Violations found diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java index e4c35e4..f9aecab 100644 --- a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java +++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java @@ -30,6 +30,7 @@ import org.apache.asterix.optimizer.rules.AsterixExtractFunctionsFromJoinConditionRule; import org.apache.asterix.optimizer.rules.AsterixInlineVariablesRule; import org.apache.asterix.optimizer.rules.AsterixIntroduceGroupByCombinerRule; +import org.apache.asterix.optimizer.rules.AsterixPushAssignBelowUnionAllRule; import org.apache.asterix.optimizer.rules.ByNameToByIndexFieldAccessRule; import org.apache.asterix.optimizer.rules.CancelUnnestWithNestedListifyRule; import org.apache.asterix.optimizer.rules.CheckFilterExpressionTypeRule; @@ -117,7 +118,6 @@ import org.apache.hyracks.algebricks.rewriter.rules.IntroduceProjectsRule; import org.apache.hyracks.algebricks.rewriter.rules.IsolateHyracksOperatorsRule; import org.apache.hyracks.algebricks.rewriter.rules.PullSelectOutOfEqJoin; -import org.apache.hyracks.algebricks.rewriter.rules.PushAssignBelowUnionAllRule; import org.apache.hyracks.algebricks.rewriter.rules.PushGroupByIntoSortRule; import org.apache.hyracks.algebricks.rewriter.rules.PushMapOperatorDownThroughProductRule; import org.apache.hyracks.algebricks.rewriter.rules.PushNestedOrderByUnderPreSortedGroupByRule; @@ -330,7 +330,7 @@ public static final List
Change in asterixdb[master]: [ASTERIXDB-2288][COMP] Fix field access with UNION
Hello Anon. E. Moose (1000171), Till Westmann, Jenkins, Dmitry Lychagin, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3400 to look at the new patch set (#10). Change subject: [ASTERIXDB-2288][COMP] Fix field access with UNION .. [ASTERIXDB-2288][COMP] Fix field access with UNION - user model changes: no - storage format changes: no - interface changes: no Details: For UNION, if there is an assign operator above UNION, the optimizer will move the assign operator under UNION into both branches. If the assign operator has field access by index, the index is based on the output record of the union operator which could be different from the field index in left and right branches. The fix is to map the index to the correct index of the input operator where the assign is copied to. Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java A asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/AsterixPushAssignBelowUnionAllRule.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.7.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/PushAssignBelowUnionAllRule.java 15 files changed, 397 insertions(+), 21 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/00/3400/10 -- To view, visit https://asterix-gerrit.ics.uci.edu/3400 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 Gerrit-Change-Number: 3400 Gerrit-PatchSet: 10 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann
Change in asterixdb[master]: [ASTERIXDB-2288][COMP] Fix field access with UNION
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3400 ) Change subject: [ASTERIXDB-2288][COMP] Fix field access with UNION .. Patch Set 9: (2 comments) https://asterix-gerrit.ics.uci.edu/#/c/3400/8/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushAssignBelowUnionAllRule.java File asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushAssignBelowUnionAllRule.java: https://asterix-gerrit.ics.uci.edu/#/c/3400/8/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushAssignBelowUnionAllRule.java@196 PS8, Line 196: > I'd move this 'if' outside of this method into modifyExpression loop. So li Done. a bit different now with the new logic. https://asterix-gerrit.ics.uci.edu/#/c/3400/8/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushAssignBelowUnionAllRule.java@220 PS8, Line 220: > what if it returns -1 (not found)? hmm. i don't think the assign can be pushed in this case (even though it should not happen). i changed the logic now to not push the assign. -- To view, visit https://asterix-gerrit.ics.uci.edu/3400 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 Gerrit-Change-Number: 3400 Gerrit-PatchSet: 9 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Tue, 21 May 2019 01:43:09 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [ASTERIXDB-2288][COMP] Fix field access with UNION
Hello Anon. E. Moose (1000171), Till Westmann, Jenkins, Dmitry Lychagin, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3400 to look at the new patch set (#9). Change subject: [ASTERIXDB-2288][COMP] Fix field access with UNION .. [ASTERIXDB-2288][COMP] Fix field access with UNION - user model changes: no - storage format changes: no - interface changes: no Details: For UNION, if there is an assign operator above UNION, the optimizer will move the assign operator under UNION into both branches. If the assign operator has field access by index, the index is based on the output record of the union operator which could be different from the field index in left and right branches. The fix is to map the index to the correct index of the input operator where the assign is copied to. Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java A asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/AsterixPushAssignBelowUnionAllRule.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.7.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/PushAssignBelowUnionAllRule.java 15 files changed, 396 insertions(+), 21 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/00/3400/9 -- To view, visit https://asterix-gerrit.ics.uci.edu/3400 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 Gerrit-Change-Number: 3400 Gerrit-PatchSet: 9 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann
Change in asterixdb[master]: [ASTERIXDB-2288][COMP] Fix field access with UNION
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3400 ) Change subject: [ASTERIXDB-2288][COMP] Fix field access with UNION .. Patch Set 8: never mind the multiple patches. I was fighting gerrit to convince it that PushAssignBelowUnionAllRule is just renamed and not a new file. -- To view, visit https://asterix-gerrit.ics.uci.edu/3400 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 Gerrit-Change-Number: 3400 Gerrit-PatchSet: 8 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Fri, 17 May 2019 01:33:38 + Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2289][COMP] Fix field access with CASE
Ali Alsuliman has submitted this change and it was merged. (
https://asterix-gerrit.ics.uci.edu/3399 )
Change subject: [ASTERIXDB-2289][COMP] Fix field access with CASE
..
[ASTERIXDB-2289][COMP] Fix field access with CASE
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
This patch fixes field access in the presense CASE and JOIN.
This is a scenario where push-down-field-access rule throws an
exception if the field access has potentially two sources and
it could not push down the field access to left or right branch.
Don't throw an exception and just return false
(i.e. field access was not pushed) instead of throwing an exception.
Change-Id: I911e4e9018c15e8f226e46fa610d222eb2301fcd
Reviewed-on: https://asterix-gerrit.ics.uci.edu/3399
Contrib: Jenkins
Tested-by: Jenkins
Integration-Tests: Jenkins
Reviewed-by: Dmitry Lychagin
---
M
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushFieldAccessRule.java
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access-ASTERIXDB-2289/field_access-ASTERIXDB-2289.1.ddl.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access-ASTERIXDB-2289/field_access-ASTERIXDB-2289.2.update.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access-ASTERIXDB-2289/field_access-ASTERIXDB-2289.3.query.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access-ASTERIXDB-2289/field_access-ASTERIXDB-2289.4.ddl.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access-ASTERIXDB-2289/field_access-ASTERIXDB-2289.3.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/util/OperatorPropertiesUtil.java
8 files changed, 162 insertions(+), 24 deletions(-)
Approvals:
Jenkins: Verified; ; Verified
Dmitry Lychagin: Looks good to me, approved
Objections:
Jenkins: Violations found
Anon. E. Moose (1000171): Violations found
diff --git
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushFieldAccessRule.java
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushFieldAccessRule.java
index 421b00d..c82aa33 100644
---
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushFieldAccessRule.java
+++
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushFieldAccessRule.java
@@ -84,6 +84,9 @@
if (op.getOperatorTag() != LogicalOperatorTag.ASSIGN) {
return false;
}
+if (!OperatorPropertiesUtil.isMovable(op)) {
+return false;
+}
AssignOperator access = (AssignOperator) op;
ILogicalExpression expr = getFirstExpr(access);
String finalAnnot;
@@ -196,17 +199,17 @@
pushDownFieldAccessRec(opRef2, context, finalAnnot);
return true;
}
-List usedInAccess = new LinkedList<>();
+HashSet usedInAccess = new HashSet<>();
VariableUtilities.getUsedVariables(assignOp, usedInAccess);
-List produced2 = new LinkedList<>();
+HashSet produced2 = new HashSet<>();
if (inputOp.getOperatorTag() == LogicalOperatorTag.GROUP) {
VariableUtilities.getLiveVariables(inputOp, produced2);
} else {
VariableUtilities.getProducedVariables(inputOp, produced2);
}
boolean pushItDown = false;
-List inter = new ArrayList<>(usedInAccess);
+HashSet inter = new HashSet<>(usedInAccess);
if (inter.isEmpty()) { // ground value
return false;
}
@@ -234,8 +237,7 @@
LogicalVariable oldVar = assignOp.getVariables().get(0);
VariableReferenceExpression v2Ref = new
VariableReferenceExpression(v2);
v2Ref.setSourceLocation(g.getSourceLocation());
-g.getDecorList().add(new Pair>(oldVar,
-new MutableObject(v2Ref)));
+g.getDecorList().add(new Pair<>(oldVar, new
MutableObject<>(v2Ref)));
changed = true;
assignOp.getVariables().set(0, v2);
VariableUtilities.substituteVariables(assignOp, m.first,
m.second, context);
@@ -281,8 +283,7 @@
}
}
}
-throw new CompilationException(ErrorCode.COMPILATION_ERROR,
assignOp.getSourceLocation(),
-"Field access " +
assignOp.getExpressions().get(0).getValue() + " doesn't correspond to any
input");
+return false;
} else {
// check if the accessed field is one of the partitioning key
fields. If yes, we can equate the 2
Change in asterixdb[master]: [ASTERIXDB-2288][COMP] Fix field access with UNION
Hello Anon. E. Moose (1000171), Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3400 to look at the new patch set (#8). Change subject: [ASTERIXDB-2288][COMP] Fix field access with UNION .. [ASTERIXDB-2288][COMP] Fix field access with UNION - user model changes: no - storage format changes: no - interface changes: no Details: For UNION, if there is an assign operator above UNION, the optimizer will move the assign operator under UNION into both branches. If the assign operator has field access by index, the index is based on the output record of the union operator which could be different from the field index in left and right branches. The fix is to map the index to the correct index of the input operator where the assign is copied to. Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java R asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushAssignBelowUnionAllRule.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.7.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml 14 files changed, 334 insertions(+), 8 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/00/3400/8 -- To view, visit https://asterix-gerrit.ics.uci.edu/3400 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 Gerrit-Change-Number: 3400 Gerrit-PatchSet: 8 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2289][COMP] Fix field access with CASE
Hello Anon. E. Moose (1000171), Till Westmann, Jenkins, Michael Blow, Dmitry Lychagin, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3399 to look at the new patch set (#4). Change subject: [ASTERIXDB-2289][COMP] Fix field access with CASE .. [ASTERIXDB-2289][COMP] Fix field access with CASE - user model changes: no - storage format changes: no - interface changes: no Details: This patch fixes field access in the presense CASE and JOIN. This is a scenario where push-down-field-access rule throws an exception if the field access has potentially two sources and it could not push down the field access to left or right branch. Don't throw an exception and just return false (i.e. field access was not pushed) instead of throwing an exception. Change-Id: I911e4e9018c15e8f226e46fa610d222eb2301fcd --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushFieldAccessRule.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access-ASTERIXDB-2289/field_access-ASTERIXDB-2289.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access-ASTERIXDB-2289/field_access-ASTERIXDB-2289.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access-ASTERIXDB-2289/field_access-ASTERIXDB-2289.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access-ASTERIXDB-2289/field_access-ASTERIXDB-2289.4.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access-ASTERIXDB-2289/field_access-ASTERIXDB-2289.3.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/util/OperatorPropertiesUtil.java 8 files changed, 162 insertions(+), 24 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/99/3399/4 -- To view, visit https://asterix-gerrit.ics.uci.edu/3399 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I911e4e9018c15e8f226e46fa610d222eb2301fcd Gerrit-Change-Number: 3399 Gerrit-PatchSet: 4 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Till Westmann
Change in asterixdb[master]: [ASTERIXDB-2288][COMP] Fix field access with UNION
Hello Anon. E. Moose (1000171), Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3400 to look at the new patch set (#7). Change subject: [ASTERIXDB-2288][COMP] Fix field access with UNION .. [ASTERIXDB-2288][COMP] Fix field access with UNION - user model changes: no - storage format changes: no - interface changes: no Details: For UNION, if there is an assign operator above UNION, the optimizer will move the assign operator under UNION into both branches. If the assign operator has field access by index, the index is based on the output record of the union operator which could be different from the field index in left and right branches. The fix is to convert the index into a named field, and then move the assign operator. The optimizer will fire by-name-to-by-index rule after this and replace the name with the correct index for each record of left and right branch. by-name-to-by-index rule is already a required rule to be fired after push-assign-under-union rule. Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java R asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushAssignBelowUnionAllRule.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.7.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml 14 files changed, 345 insertions(+), 7 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/00/3400/7 -- To view, visit https://asterix-gerrit.ics.uci.edu/3400 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 Gerrit-Change-Number: 3400 Gerrit-PatchSet: 7 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2288][COMP] Fix field access with UNION
Ali Alsuliman has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/3400 Change subject: [ASTERIXDB-2288][COMP] Fix field access with UNION .. [ASTERIXDB-2288][COMP] Fix field access with UNION - user model changes: no - storage format changes: no - interface changes: no Details: For UNION, if there is an assign operator above UNION, the optimizer will move the assign operator under UNION into both branches. If the assign operator has field access by index, the index is based on the output record of the union operator which could be different from the field index in left and right branches. The fix is to convert the index into a named field, and then move the assign operator. The optimizer will fire by-name-to-by-index rule after this and replace the name with the correct index for each record of left and right branch. by-name-to-by-index rule is already a required rule to be fired after push-assign-under-union rule. Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java A asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushAssignBelowUnionAllRule.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.7.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml D hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/PushAssignBelowUnionAllRule.java 15 files changed, 515 insertions(+), 171 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/00/3400/1 diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java index e4c35e4..9d5fc2f 100644 --- a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java +++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java @@ -117,7 +117,7 @@ import org.apache.hyracks.algebricks.rewriter.rules.IntroduceProjectsRule; import org.apache.hyracks.algebricks.rewriter.rules.IsolateHyracksOperatorsRule; import org.apache.hyracks.algebricks.rewriter.rules.PullSelectOutOfEqJoin; -import org.apache.hyracks.algebricks.rewriter.rules.PushAssignBelowUnionAllRule; +import org.apache.asterix.optimizer.rules.PushAssignBelowUnionAllRule; import org.apache.hyracks.algebricks.rewriter.rules.PushGroupByIntoSortRule; import org.apache.hyracks.algebricks.rewriter.rules.PushMapOperatorDownThroughProductRule; import org.apache.hyracks.algebricks.rewriter.rules.PushNestedOrderByUnderPreSortedGroupByRule; diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushAssignBelowUnionAllRule.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushAssignBelowUnionAllRule.java new file mode 100644 index 000..552b691 --- /dev/null +++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushAssignBelowUnionAllRule.java @@ -0,0 +1,245 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with
Change in asterixdb[master]: [ASTERIXDB-2288][COMP] Fix field access with UNION
Hello Anon. E. Moose (1000171), Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3400 to look at the new patch set (#6). Change subject: [ASTERIXDB-2288][COMP] Fix field access with UNION .. [ASTERIXDB-2288][COMP] Fix field access with UNION - user model changes: no - storage format changes: no - interface changes: no Details: For UNION, if there is an assign operator above UNION, the optimizer will move the assign operator under UNION into both branches. If the assign operator has field access by index, the index is based on the output record of the union operator which could be different from the field index in left and right branches. The fix is to convert the index into a named field, and then move the assign operator. The optimizer will fire by-name-to-by-index rule after this and replace the name with the correct index for each record of left and right branch. by-name-to-by-index rule is already a required rule to be fired after push-assign-under-union rule. Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java R asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushAssignBelowUnionAllRule.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.7.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml 14 files changed, 345 insertions(+), 7 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/00/3400/6 -- To view, visit https://asterix-gerrit.ics.uci.edu/3400 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 Gerrit-Change-Number: 3400 Gerrit-PatchSet: 6 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2288][COMP] Fix field access with UNION
Hello Anon. E. Moose (1000171), Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3400 to look at the new patch set (#5). Change subject: [ASTERIXDB-2288][COMP] Fix field access with UNION .. [ASTERIXDB-2288][COMP] Fix field access with UNION - user model changes: no - storage format changes: no - interface changes: no Details: For UNION, if there is an assign operator above UNION, the optimizer will move the assign operator under UNION into both branches. If the assign operator has field access by index, the index is based on the output record of the union operator which could be different from the field index in left and right branches. The fix is to convert the index into a named field, and then move the assign operator. The optimizer will fire by-name-to-by-index rule after this and replace the name with the correct index for each record of left and right branch. by-name-to-by-index rule is already a required rule to be fired after push-assign-under-union rule. Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java R asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushAssignBelowUnionAllRule.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.7.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml 14 files changed, 271 insertions(+), 2 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/00/3400/5 -- To view, visit https://asterix-gerrit.ics.uci.edu/3400 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 Gerrit-Change-Number: 3400 Gerrit-PatchSet: 5 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2288][COMP] Fix field access with UNION
Hello Anon. E. Moose (1000171), Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3400 to look at the new patch set (#3). Change subject: [ASTERIXDB-2288][COMP] Fix field access with UNION .. [ASTERIXDB-2288][COMP] Fix field access with UNION - user model changes: no - storage format changes: no - interface changes: no Details: For UNION, if there is an assign operator above UNION, the optimizer will move the assign operator under UNION into both branches. If the assign operator has field access by index, the index is based on the output record of the union operator which could be different from the field index in left and right branches. The fix is to convert the index into a named field, and then move the assign operator. The optimizer will fire by-name-to-by-index rule after this and replace the name with the correct index for each record of left and right branch. by-name-to-by-index rule is already a required rule to be fired after push-assign-under-union rule. Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 --- A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.7.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml 12 files changed, 269 insertions(+), 0 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/00/3400/3 -- To view, visit https://asterix-gerrit.ics.uci.edu/3400 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 Gerrit-Change-Number: 3400 Gerrit-PatchSet: 3 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2288][COMP] Fix field access with UNION
Hello Anon. E. Moose (1000171), Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3400 to look at the new patch set (#2). Change subject: [ASTERIXDB-2288][COMP] Fix field access with UNION .. [ASTERIXDB-2288][COMP] Fix field access with UNION - user model changes: no - storage format changes: no - interface changes: no Details: For UNION, if there is an assign operator above UNION, the optimizer will move the assign operator under UNION into both branches. If the assign operator has field access by index, the index is based on the output record of the union operator which could be different from the field index in left and right branches. The fix is to convert the index into a named field, and then move the assign operator. The optimizer will fire by-name-to-by-index rule after this and replace the name with the correct index for each record of left and right branch. by-name-to-by-index rule is already a required rule to be fired after push-assign-under-union rule. Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java A asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushAssignBelowUnionAllRule.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.7.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.3.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.4.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.5.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access_union-ASTERIXDB-2288/field_access_union-ASTERIXDB-2288.6.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml D hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/PushAssignBelowUnionAllRule.java 15 files changed, 515 insertions(+), 171 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/00/3400/2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3400 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4d65374aca6959fd5e70279fac8f7f0a6c7c64c5 Gerrit-Change-Number: 3400 Gerrit-PatchSet: 2 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2289][COMP] Fix field access with CASE
Hello Anon. E. Moose (1000171), Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3399 to look at the new patch set (#3). Change subject: [ASTERIXDB-2289][COMP] Fix field access with CASE .. [ASTERIXDB-2289][COMP] Fix field access with CASE - user model changes: no - storage format changes: no - interface changes: no Details: This patch fixes field access in the presense CASE and JOIN. This is a scenario where push-down-field-access rule throws an exception if the field access has potentially two sources and it could not push down the field access to left or right branch. The exception thinks that the field access has no source when in reality it has but there were two potential sources because of CASE. Don't throw an exception if the field access is a valid one and return false (i.e. field access was not push) instead of throwing an exception. Change-Id: I911e4e9018c15e8f226e46fa610d222eb2301fcd --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushFieldAccessRule.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access-ASTERIXDB-2289/field_access-ASTERIXDB-2289.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access-ASTERIXDB-2289/field_access-ASTERIXDB-2289.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access-ASTERIXDB-2289/field_access-ASTERIXDB-2289.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access-ASTERIXDB-2289/field_access-ASTERIXDB-2289.3.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml 6 files changed, 143 insertions(+), 3 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/99/3399/3 -- To view, visit https://asterix-gerrit.ics.uci.edu/3399 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I911e4e9018c15e8f226e46fa610d222eb2301fcd Gerrit-Change-Number: 3399 Gerrit-PatchSet: 3 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2289][COMP] Fix field access with CASE
Hello Anon. E. Moose (1000171), Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3399 to look at the new patch set (#2). Change subject: [ASTERIXDB-2289][COMP] Fix field access with CASE .. [ASTERIXDB-2289][COMP] Fix field access with CASE - user model changes: no - storage format changes: no - interface changes: no Details: This patch fixes field access in the presense CASE and JOIN. This is a scenario where push-down-field-access rule throws an exception if the field access has potentially two sources and it could not push down the field access to left or right branch. The exception thinks that the field access has no source when in reality it has but there were two potential sources because of CASE. Don't throw an exception if the field access is a valid one and return false (i.e. field access was not push) instead of throwing an exception. Change-Id: I911e4e9018c15e8f226e46fa610d222eb2301fcd --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushFieldAccessRule.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access-ASTERIXDB-2289/field_access-ASTERIXDB-2289.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access-ASTERIXDB-2289/field_access-ASTERIXDB-2289.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access-ASTERIXDB-2289/field_access-ASTERIXDB-2289.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access-ASTERIXDB-2289/field_access-ASTERIXDB-2289.3.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml 6 files changed, 143 insertions(+), 3 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/99/3399/2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3399 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I911e4e9018c15e8f226e46fa610d222eb2301fcd Gerrit-Change-Number: 3399 Gerrit-PatchSet: 2 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2289][COMP] Fix field access with UNION and CASE
Ali Alsuliman has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/3399 Change subject: [ASTERIXDB-2289][COMP] Fix field access with UNION and CASE .. [ASTERIXDB-2289][COMP] Fix field access with UNION and CASE - user model changes: no - storage format changes: no - interface changes: no Details: This patch fixes field access in the presense of UNION and CASE. For CASE scenario, push-down-field-access rule throws an exception if the field access has potentially two sources and it could not push down the field access to left or right branch. The exception thinks that the field access has no source when in reality it has but there were two potential sources because of CASE. Don't throw an exception if the field access is a valid one and return false (i.e. field access was not push) instead of throwing an exception. For UNION, if there is an assign operator above UNION, the optimizer will move the assign operator under UNION into both branches. If the assign operator has field access by index, the index is based on the output record of the union operator which could be different from the field index in left and right branches. The fix is to convert the index into a named field, and then move the assign operator. The optimizer will fire by-name-to-by-index rule after this and replace the name with the correct index for each record of left and right branch. by-name-to-by-index rule is already a required rule to be fired after push-assign-under-union rule. Change-Id: I911e4e9018c15e8f226e46fa610d222eb2301fcd --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java A asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushAssignBelowUnionAllRule.java M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/PushFieldAccessRule.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access/field_access.1.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access/field_access.2.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access/field_access.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access/field_access.4.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access/field_access.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access/field_access.6.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/field_access/field_access.7.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access/field_access.3.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access/field_access.4.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access/field_access.5.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access/field_access.6.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/field_access/field_access.7.adm D hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/PushAssignBelowUnionAllRule.java 16 files changed, 537 insertions(+), 174 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/99/3399/1 diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java index e4c35e4..79f565f 100644 --- a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java +++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java @@ -66,6 +66,7 @@ import org.apache.asterix.optimizer.rules.NestGroupByRule; import org.apache.asterix.optimizer.rules.PushAggFuncIntoStandaloneAggregateRule; import org.apache.asterix.optimizer.rules.PushAggregateIntoNestedSubplanRule; +import org.apache.asterix.optimizer.rules.PushAssignBelowUnionAllRule; import org.apache.asterix.optimizer.rules.PushFieldAccessRule; import org.apache.asterix.optimizer.rules.PushGroupByThroughProduct; import org.apache.asterix.optimizer.rules.PushLimitIntoOrderByRule; @@ -117,7 +118,6 @@ import org.apache.hyracks.algebricks.rewriter.rules.IntroduceProjectsRule; import org.apache.hyracks.algebricks.rewriter.rules.IsolateHyracksOperatorsRule; import org.apache.hyracks.algebricks.rewriter.rules.PullSelectOutOfEqJoin; -import org.apache.hyracks.algebricks.rewriter.rules.PushAssignBelowUnionAllRule; import org.apache.hyracks.algebricks.rewriter.rules.PushGroupByIntoSortRule; import org.apache.hyracks.algebricks.rewriter.rules.PushMapOperatorDownThroughProductRule; import
Change in asterixdb[master]: [ASTERIXDB-2567][RT] Fix serializing nullable record fields
Ali Alsuliman has submitted this change and it was merged. ( https://asterix-gerrit.ics.uci.edu/3398 ) Change subject: [ASTERIXDB-2567][RT] Fix serializing nullable record fields .. [ASTERIXDB-2567][RT] Fix serializing nullable record fields - user model changes: no - storage format changes: no - interface changes: no Details: When serializing a record, a field should be checked if it's nullable. The value to be serialized should be checked if it's null or value to serialize it correctly. Exceptions could happen when the record is in an array if record fields are not checked. Change-Id: I42105965e69c60bc4b123337296476b4d75f0ad8 Reviewed-on: https://asterix-gerrit.ics.uci.edu/3398 Contrib: Jenkins Tested-by: Jenkins Reviewed-by: Murtadha Hubail Integration-Tests: Jenkins --- A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.1.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.2.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.4.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.6.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.7.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.8.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.2.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.3.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.4.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.5.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.6.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.7.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.8.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml M asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/serde/ARecordSerializerDeserializer.java 18 files changed, 207 insertions(+), 9 deletions(-) Approvals: Jenkins: Verified; ; Verified Murtadha Hubail: Looks good to me, approved Anon. E. Moose (1000171): Objections: Jenkins: Violations found diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.1.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.1.query.sqlpp new file mode 100644 index 000..8d50408 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.1.query.sqlpp @@ -0,0 +1,21 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the
Change in asterixdb[master]: [ASTERIXDB-2567][RT] Fix serializing nullable record fields
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3398 ) Change subject: [ASTERIXDB-2567][RT] Fix serializing nullable record fields .. Patch Set 4: (1 comment) https://asterix-gerrit.ics.uci.edu/#/c/3398/3/asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/serde/ARecordSerializerDeserializer.java File asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/serde/ARecordSerializerDeserializer.java: https://asterix-gerrit.ics.uci.edu/#/c/3398/3/asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/serde/ARecordSerializerDeserializer.java@185 PS3, Line 185: for (int fieldIndex > not changed in your patch, but why not move this into the for-loop initiali right. done since i had to rebase anyway. -- To view, visit https://asterix-gerrit.ics.uci.edu/3398 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I42105965e69c60bc4b123337296476b4d75f0ad8 Gerrit-Change-Number: 3398 Gerrit-PatchSet: 4 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Wed, 15 May 2019 18:31:05 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [ASTERIXDB-2567][RT] Fix serializing nullable record fields
Hello Anon. E. Moose (1000171), Till Westmann, Jenkins, Michael Blow, Dmitry Lychagin, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3398 to look at the new patch set (#4). Change subject: [ASTERIXDB-2567][RT] Fix serializing nullable record fields .. [ASTERIXDB-2567][RT] Fix serializing nullable record fields - user model changes: no - storage format changes: no - interface changes: no Details: When serializing a record, a field should be checked if it's nullable. The value to be serialized should be checked if it's null or value to serialize it correctly. Exceptions could happen when the record is in an array if record fields are not checked. Change-Id: I42105965e69c60bc4b123337296476b4d75f0ad8 --- A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.1.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.2.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.4.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.6.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.7.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.8.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.2.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.3.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.4.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.5.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.6.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.7.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.8.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml M asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/serde/ARecordSerializerDeserializer.java 18 files changed, 207 insertions(+), 9 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/98/3398/4 -- To view, visit https://asterix-gerrit.ics.uci.edu/3398 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I42105965e69c60bc4b123337296476b4d75f0ad8 Gerrit-Change-Number: 3398 Gerrit-PatchSet: 4 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Till Westmann
Change in asterixdb[master]: [NO ISSUE][COMP] Change plan printing for window operators
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3397 ) Change subject: [NO ISSUE][COMP] Change plan printing for window operators .. Patch Set 2: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3397 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1446238a7c550abf618a72650c2a811fbc4dd710 Gerrit-Change-Number: 3397 Gerrit-PatchSet: 2 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Wed, 15 May 2019 18:16:47 + Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2562][FUN] Add support for bitwise functions
Ali Alsuliman has posted comments on this change. (
https://asterix-gerrit.ics.uci.edu/3377 )
Change subject: [ASTERIXDB-2562][FUN] Add support for bitwise functions
..
Patch Set 12:
(14 comments)
https://asterix-gerrit.ics.uci.edu/#/c/3377/11/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/BitwiseValueCountFlagTypeComputer.java
File
asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/BitwiseValueCountFlagTypeComputer.java:
https://asterix-gerrit.ics.uci.edu/#/c/3377/11/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/BitwiseValueCountFlagTypeComputer.java@45
PS11, Line 45: BitwiseValueCountFlagTypeComputer
this class and BitwiseValuePositionFlagTypeComputer and
BitwiseValuePositionTypeComputer have a lot in common. you could either create
instances with several flags (return type, whether to include ARRAY), or create
instances with an enum that differentiates each one. Based on the enum (or
flags) you check and configure the return type.
https://asterix-gerrit.ics.uci.edu/#/c/3377/11/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/bitwise/AbstractBitMultipleValuesEvaluator.java
File
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/bitwise/AbstractBitMultipleValuesEvaluator.java:
https://asterix-gerrit.ics.uci.edu/#/c/3377/11/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/bitwise/AbstractBitMultipleValuesEvaluator.java@94
PS11, Line 94: resultStorage.reset();
resultStorage is already reset down, right?
https://asterix-gerrit.ics.uci.edu/#/c/3377/11/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/bitwise/AbstractBitMultipleValuesEvaluator.java@102
PS11, Line 102: 0
better to use getStartOffset
https://asterix-gerrit.ics.uci.edu/#/c/3377/11/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/bitwise/AbstractBitMultipleValuesEvaluator.java@122
PS11, Line 122: if (!ATypeHierarchy.canPromote(typeTag,
ATypeTag.DOUBLE)) {
: PointableHelper.setNull(result);
: return;
: }
:
: // Value validity check
: if (!isValidLongValue(bytes, startOffset + 1, typeTag))
{
: PointableHelper.setNull(result);
: return;
: }
we could refactor this code and reuse below (and for other bit functions, too).
The refactored method would return boolean (true to return, false to continue).
It would also set the result if it says true.
https://asterix-gerrit.ics.uci.edu/#/c/3377/11/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/bitwise/AbstractBitMultipleValuesEvaluator.java@154
PS11, Line 154: getLongValue(bytes, startOffset + 1, typeTag, i)
you could plug it directly in the method if you want.
https://asterix-gerrit.ics.uci.edu/#/c/3377/11/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/bitwise/AbstractBitMultipleValuesEvaluator.java@185
PS11, Line 185: isValidLongValue
The returned result is always inverted. we could call it "isInvalidLongValue"
and change accordingly.
https://asterix-gerrit.ics.uci.edu/#/c/3377/11/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/bitwise/AbstractBitMultipleValuesEvaluator.java@189
PS11, Line 189: float value
what is the behaviour if the float is NaN or INF/-INF?
https://asterix-gerrit.ics.uci.edu/#/c/3377/11/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/bitwise/AbstractBitMultipleValuesEvaluator.java@192
PS11, Line 192: value > Long.MAX_VALUE
what if the value is < MIN_VALUE? do these functions accept negative arguments?
https://asterix-gerrit.ics.uci.edu/#/c/3377/11/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/bitwise/AbstractBitMultipleValuesEvaluator.java@219
PS11, Line 219: private long getLongValue(byte[] bytes, int startOffset,
ATypeTag typeTag, int argIndex)
we should refactor this and reuse in other bit functions. you could put it in a
util class.
https://asterix-gerrit.ics.uci.edu/#/c/3377/11/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/bitwise/AbstractBitValueCountFlagEvaluator.java
File
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/bitwise/AbstractBitValueCountFlagEvaluator.java:
https://asterix-gerrit.ics.uci.edu/#/c/3377/11/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/bitwise/AbstractBitValueCountFlagEvaluator.java@174
PS11, Line 174: (int) count)
are we losing precision here?
Change in asterixdb[master]: [ASTERIXDB-2567][RT] Fix serializing nullable record fields
Hello Anon. E. Moose (1000171), Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3398 to look at the new patch set (#2). Change subject: [ASTERIXDB-2567][RT] Fix serializing nullable record fields .. [ASTERIXDB-2567][RT] Fix serializing nullable record fields - user model changes: no - storage format changes: no - interface changes: no Details: When serializing a record, a field should be checked if it's nullable. The value to be serialized should be checked if it's null or value to serialize it correctly. Exceptions could happen when the record is in an array if record fields are not checked. Change-Id: I42105965e69c60bc4b123337296476b4d75f0ad8 --- A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.1.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.2.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.3.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.4.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.5.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.6.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.7.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.8.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.1.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.2.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.3.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.4.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.5.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.6.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.7.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.8.adm M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml M asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/serde/ARecordSerializerDeserializer.java 18 files changed, 206 insertions(+), 7 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/98/3398/2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3398 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I42105965e69c60bc4b123337296476b4d75f0ad8 Gerrit-Change-Number: 3398 Gerrit-PatchSet: 2 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2567][RT] Fix serializing nullable record fields
Ali Alsuliman has uploaded this change for review. (
https://asterix-gerrit.ics.uci.edu/3398
Change subject: [ASTERIXDB-2567][RT] Fix serializing nullable record fields
..
[ASTERIXDB-2567][RT] Fix serializing nullable record fields
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
When serializing a record, a field should be checked if it's nullable.
The value to be serialized should be checked if it's null or value to
serialize it correctly. Exceptions could happen when the record is in
an array if record fields are not checked.
Change-Id: I42105965e69c60bc4b123337296476b4d75f0ad8
---
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.1.query.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.2.query.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.3.query.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.4.query.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.5.query.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.6.query.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.7.query.sqlpp
A
asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.1.adm
A
asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.2.adm
A
asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.3.adm
A
asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.4.adm
A
asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.5.adm
A
asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.6.adm
A
asterixdb/asterix-app/src/test/resources/runtimets/results/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.7.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M
asterixdb/asterix-om/src/main/java/org/apache/asterix/dataflow/data/nontagged/serde/ARecordSerializerDeserializer.java
16 files changed, 181 insertions(+), 7 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb
refs/changes/98/3398/1
diff --git
a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.1.query.sqlpp
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.1.query.sqlpp
new file mode 100644
index 000..8d50408
--- /dev/null
+++
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.1.query.sqlpp
@@ -0,0 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+// testing fix for record serialization with nullable fields where the record
is in an array and will be constant folded
+[{"f1": "fieldStr", "f2": 1 < "text"}];
\ No newline at end of file
diff --git
a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/record-serialization-ASTERIXDB-2567/record-serialization-ASTERIXDB-2567.2.query.sqlpp
Change in asterixdb[master]: [ASTERIXDB-2535][COMP] Fix uuid present in insert/upsert statement
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3370 ) Change subject: [ASTERIXDB-2535][COMP] Fix uuid present in insert/upsert statement .. Patch Set 11: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3370 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I22100d3ff29864b8bfd54b0decb183e5056fdb4a Gerrit-Change-Number: 3370 Gerrit-PatchSet: 11 Gerrit-Owner: Hussain Towaileb Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Hussain Towaileb Gerrit-Reviewer: Hussain Towaileb Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Tue, 14 May 2019 18:04:30 + Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2535][COMP] Fix uuid present in insert/upsert statement
Ali Alsuliman has posted comments on this change. (
https://asterix-gerrit.ics.uci.edu/3370 )
Change subject: [ASTERIXDB-2535][COMP] Fix uuid present in insert/upsert
statement
..
Patch Set 11:
(1 comment)
https://asterix-gerrit.ics.uci.edu/#/c/3370/11/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/records/RecordMergeIgnoreDuplicatesDescriptor.java
File
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/records/RecordMergeIgnoreDuplicatesDescriptor.java:
https://asterix-gerrit.ics.uci.edu/#/c/3370/11/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/records/RecordMergeIgnoreDuplicatesDescriptor.java@89
PS11, Line 89: @Override
: protected void handleDuplicateFields() {
: // Don't throw an exception, just ignore
the duplicate field merging, this will take the
: // leftRecord field and ignore the one
on the rightRecord.
: }
we could just pass a flag to RecordMergeEvaluator that would either throw or
not throw. The inheritance seems simple here.
--
To view, visit https://asterix-gerrit.ics.uci.edu/3370
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22100d3ff29864b8bfd54b0decb183e5056fdb4a
Gerrit-Change-Number: 3370
Gerrit-PatchSet: 11
Gerrit-Owner: Hussain Towaileb
Gerrit-Reviewer: Ali Alsuliman
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Dmitry Lychagin
Gerrit-Reviewer: Hussain Towaileb
Gerrit-Reviewer: Hussain Towaileb
Gerrit-Reviewer: Jenkins
Gerrit-Reviewer: Till Westmann
Gerrit-Comment-Date: Tue, 14 May 2019 18:08:53 +
Gerrit-HasComments: Yes
Change in asterixdb[master]: [NO ISSUE] Additional constructor for TaggedValuePointable
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3242 ) Change subject: [NO ISSUE] Additional constructor for TaggedValuePointable .. Patch Set 5: Code-Review+2 (1 comment) https://asterix-gerrit.ics.uci.edu/#/c/3242/5/hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/src/main/java/org/apache/hyracks/data/std/primitive/TaggedValuePointable.java File hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/src/main/java/org/apache/hyracks/data/std/primitive/TaggedValuePointable.java: https://asterix-gerrit.ics.uci.edu/#/c/3242/5/hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/src/main/java/org/apache/hyracks/data/std/primitive/TaggedValuePointable.java@47 PS5, Line 47: public should we make it private? -- To view, visit https://asterix-gerrit.ics.uci.edu/3242 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3cad369ecb2f7cafd901f11e757272dce7e1e6b5 Gerrit-Change-Number: 3242 Gerrit-PatchSet: 5 Gerrit-Owner: Till Westmann Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Tue, 14 May 2019 00:41:23 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [NO ISSUE] Additional constructor for TaggedValuePointable
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3242 ) Change subject: [NO ISSUE] Additional constructor for TaggedValuePointable .. Patch Set 5: (1 comment) https://asterix-gerrit.ics.uci.edu/#/c/3242/5/hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/src/main/java/org/apache/hyracks/data/std/primitive/TaggedValuePointable.java File hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/src/main/java/org/apache/hyracks/data/std/primitive/TaggedValuePointable.java: https://asterix-gerrit.ics.uci.edu/#/c/3242/5/hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/src/main/java/org/apache/hyracks/data/std/primitive/TaggedValuePointable.java@39 PS5, Line 39: value.set(bytes, start + 1, length - 1); > Yes, we would need to adjust the pointers when setting the pointable. given the assumption that the value passed to "set" should be tagged, we should keep this as is. -- To view, visit https://asterix-gerrit.ics.uci.edu/3242 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3cad369ecb2f7cafd901f11e757272dce7e1e6b5 Gerrit-Change-Number: 3242 Gerrit-PatchSet: 5 Gerrit-Owner: Till Westmann Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Tue, 14 May 2019 00:40:47 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [NO ISSUE][FUN] Add ARRAY_AGG() SQL aggregate function
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3390 ) Change subject: [NO ISSUE][FUN] Add ARRAY_AGG() SQL aggregate function .. Patch Set 4: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3390 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I704e031a1252493e83ad8d45c38b75e0b15c1896 Gerrit-Change-Number: 3390 Gerrit-PatchSet: 4 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Mon, 13 May 2019 19:50:10 + Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2564][RT] Too many objects created in min() and max()
Ali Alsuliman has submitted this change and it was merged. (
https://asterix-gerrit.ics.uci.edu/3391 )
Change subject: [ASTERIXDB-2564][RT] Too many objects created in min() and max()
..
[ASTERIXDB-2564][RT] Too many objects created in min() and max()
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
During min() and max() aggregation, the functions keep track of
the aggregation type in order to handle heterogeneous lists.
It promotes the aggregation type if needed (e.g. encountered double).
Don't switch to new aggregation type and create a new comparator
when the new input value type is the same as the previously
aggregated values. That is because canPromote(agg_type, new_val_type)
will always return true for same types.
Change-Id: I0bb9f0715985ae555de00bbf3173c80371d8968b
Reviewed-on: https://asterix-gerrit.ics.uci.edu/3391
Contrib: Jenkins
Sonar-Qube: Jenkins
Tested-by: Jenkins
Integration-Tests: Jenkins
Reviewed-by: Dmitry Lychagin
---
M
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractMinMaxAggregateFunction.java
1 file changed, 43 insertions(+), 60 deletions(-)
Approvals:
Jenkins: Verified; No violations found; ; Verified
Anon. E. Moose (1000171):
Dmitry Lychagin: Looks good to me, approved
diff --git
a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractMinMaxAggregateFunction.java
b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractMinMaxAggregateFunction.java
index 86ae924..90f006d 100644
---
a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractMinMaxAggregateFunction.java
+++
b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractMinMaxAggregateFunction.java
@@ -30,7 +30,6 @@
import org.apache.hyracks.algebricks.runtime.base.IScalarEvaluatorFactory;
import org.apache.hyracks.api.context.IHyracksTaskContext;
import org.apache.hyracks.api.dataflow.value.IBinaryComparator;
-import org.apache.hyracks.api.dataflow.value.IBinaryComparatorFactory;
import org.apache.hyracks.api.exceptions.HyracksDataException;
import org.apache.hyracks.api.exceptions.SourceLocation;
import org.apache.hyracks.data.std.api.IPointable;
@@ -39,18 +38,16 @@
import org.apache.hyracks.dataflow.common.data.accessors.IFrameTupleReference;
public abstract class AbstractMinMaxAggregateFunction extends
AbstractAggregateFunction {
-private IPointable inputVal = new VoidPointable();
-private ArrayBackedValueStorage outputVal = new ArrayBackedValueStorage();
-private ArrayBackedValueStorage tempValForCasting = new
ArrayBackedValueStorage();
-
-protected ArrayBackedValueStorage resultStorage = new
ArrayBackedValueStorage();
-private IScalarEvaluator eval;
+protected final ArrayBackedValueStorage resultStorage = new
ArrayBackedValueStorage();
+private final IPointable inputVal = new VoidPointable();
+private final ArrayBackedValueStorage outputVal = new
ArrayBackedValueStorage();
+private final ArrayBackedValueStorage tempValForCasting = new
ArrayBackedValueStorage();
+private final IScalarEvaluator eval;
+private final boolean isMin;
protected ATypeTag aggType;
private IBinaryComparator cmp;
-private ITypeConvertComputer tpc;
-private final boolean isMin;
-public AbstractMinMaxAggregateFunction(IScalarEvaluatorFactory[] args,
IHyracksTaskContext context, boolean isMin,
+AbstractMinMaxAggregateFunction(IScalarEvaluatorFactory[] args,
IHyracksTaskContext context, boolean isMin,
SourceLocation sourceLoc) throws HyracksDataException {
super(sourceLoc);
eval = args[0].createScalarEvaluator(context);
@@ -82,9 +79,8 @@
// First value encountered. Set type, comparator, and initial
value.
aggType = typeTag;
// Set comparator.
-IBinaryComparatorFactory cmpFactory =
-
BinaryComparatorFactoryProvider.INSTANCE.getBinaryComparatorFactory(aggType,
isMin);
-cmp = cmpFactory.createBinaryComparator();
+cmp =
BinaryComparatorFactoryProvider.INSTANCE.getBinaryComparatorFactory(aggType,
isMin)
+.createBinaryComparator();
// Initialize min value.
outputVal.assign(inputVal);
} else if (typeTag != ATypeTag.SYSTEM_NULL &&
!ATypeHierarchy.isCompatible(typeTag, aggType)) {
@@ -94,56 +90,27 @@
throw new IncompatibleTypeException(sourceLoc, "min/max",
aggType.serialize(), typeTag.serialize());
}
} else {
-
// If a system_null is encountered locally, it would be an error;
otherwise if it is seen
// by a global aggregator, it is simple ignored.
if (typeTag == ATypeTag.SYSTEM_NULL) {
Change in asterixdb[master]: [ASTERIXDB-2564][RT] Too many objects created in min() and max()
Hello Anon. E. Moose (1000171), Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3391 to look at the new patch set (#2). Change subject: [ASTERIXDB-2564][RT] Too many objects created in min() and max() .. [ASTERIXDB-2564][RT] Too many objects created in min() and max() - user model changes: no - storage format changes: no - interface changes: no Details: During min() and max() aggregation, the functions keep track of the aggregation type in order to handle heterogeneous lists. It promotes the aggregation type if needed (e.g. encountered double). Don't switch to new aggregation type and create a new comparator when the new input value type is the same as the previously aggregated values. That is because canPromote(agg_type, new_val_type) will always return true for same types. Change-Id: I0bb9f0715985ae555de00bbf3173c80371d8968b --- M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractMinMaxAggregateFunction.java 1 file changed, 43 insertions(+), 60 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/91/3391/2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3391 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I0bb9f0715985ae555de00bbf3173c80371d8968b Gerrit-Change-Number: 3391 Gerrit-PatchSet: 2 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2564][RT] Too many objects created in min() and max()
Ali Alsuliman has uploaded this change for review. (
https://asterix-gerrit.ics.uci.edu/3391
Change subject: [ASTERIXDB-2564][RT] Too many objects created in min() and max()
..
[ASTERIXDB-2564][RT] Too many objects created in min() and max()
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
During min() and max() aggregation, the functions keep track of
the aggregation type in order to handle heterogeneous lists.
It promotes the aggregation type if needed (e.g. encountered double).
Don't switch to new aggregation type and create a new comparator
when the new input value type is the same as the previously
aggregated values. That is because canPromote(agg_type, new_val_type)
will always return true for same types.
Change-Id: I0bb9f0715985ae555de00bbf3173c80371d8968b
---
M
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractMinMaxAggregateFunction.java
1 file changed, 40 insertions(+), 54 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb
refs/changes/91/3391/1
diff --git
a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractMinMaxAggregateFunction.java
b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractMinMaxAggregateFunction.java
index 86ae924..616bb5a 100644
---
a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractMinMaxAggregateFunction.java
+++
b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractMinMaxAggregateFunction.java
@@ -30,7 +30,6 @@
import org.apache.hyracks.algebricks.runtime.base.IScalarEvaluatorFactory;
import org.apache.hyracks.api.context.IHyracksTaskContext;
import org.apache.hyracks.api.dataflow.value.IBinaryComparator;
-import org.apache.hyracks.api.dataflow.value.IBinaryComparatorFactory;
import org.apache.hyracks.api.exceptions.HyracksDataException;
import org.apache.hyracks.api.exceptions.SourceLocation;
import org.apache.hyracks.data.std.api.IPointable;
@@ -39,18 +38,16 @@
import org.apache.hyracks.dataflow.common.data.accessors.IFrameTupleReference;
public abstract class AbstractMinMaxAggregateFunction extends
AbstractAggregateFunction {
-private IPointable inputVal = new VoidPointable();
-private ArrayBackedValueStorage outputVal = new ArrayBackedValueStorage();
-private ArrayBackedValueStorage tempValForCasting = new
ArrayBackedValueStorage();
-
-protected ArrayBackedValueStorage resultStorage = new
ArrayBackedValueStorage();
-private IScalarEvaluator eval;
+protected final ArrayBackedValueStorage resultStorage = new
ArrayBackedValueStorage();
+private final IPointable inputVal = new VoidPointable();
+private final ArrayBackedValueStorage outputVal = new
ArrayBackedValueStorage();
+private final ArrayBackedValueStorage tempValForCasting = new
ArrayBackedValueStorage();
+private final IScalarEvaluator eval;
+private final boolean isMin;
protected ATypeTag aggType;
private IBinaryComparator cmp;
-private ITypeConvertComputer tpc;
-private final boolean isMin;
-public AbstractMinMaxAggregateFunction(IScalarEvaluatorFactory[] args,
IHyracksTaskContext context, boolean isMin,
+AbstractMinMaxAggregateFunction(IScalarEvaluatorFactory[] args,
IHyracksTaskContext context, boolean isMin,
SourceLocation sourceLoc) throws HyracksDataException {
super(sourceLoc);
eval = args[0].createScalarEvaluator(context);
@@ -82,9 +79,8 @@
// First value encountered. Set type, comparator, and initial
value.
aggType = typeTag;
// Set comparator.
-IBinaryComparatorFactory cmpFactory =
-
BinaryComparatorFactoryProvider.INSTANCE.getBinaryComparatorFactory(aggType,
isMin);
-cmp = cmpFactory.createBinaryComparator();
+cmp =
BinaryComparatorFactoryProvider.INSTANCE.getBinaryComparatorFactory(aggType,
isMin)
+.createBinaryComparator();
// Initialize min value.
outputVal.assign(inputVal);
} else if (typeTag != ATypeTag.SYSTEM_NULL &&
!ATypeHierarchy.isCompatible(typeTag, aggType)) {
@@ -94,56 +90,27 @@
throw new IncompatibleTypeException(sourceLoc, "min/max",
aggType.serialize(), typeTag.serialize());
}
} else {
-
// If a system_null is encountered locally, it would be an error;
otherwise if it is seen
// by a global aggregator, it is simple ignored.
if (typeTag == ATypeTag.SYSTEM_NULL) {
processSystemNull();
return;
}
-
+if (aggType == typeTag) {
+compareAndUpdate(cmp, inputVal, outputVal);
+return;
+}
if
Change in asterixdb[master]: [ASTERIXDB-2535][COMP] Fix uuid present in insert/upsert statement
Ali Alsuliman has posted comments on this change. (
https://asterix-gerrit.ics.uci.edu/3370 )
Change subject: [ASTERIXDB-2535][COMP] Fix uuid present in insert/upsert
statement
..
Patch Set 8:
(1 comment)
https://asterix-gerrit.ics.uci.edu/#/c/3370/7/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/RecordMergeTypeComputer.java
File
asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/RecordMergeTypeComputer.java:
https://asterix-gerrit.ics.uci.edu/#/c/3370/7/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/RecordMergeTypeComputer.java@115
PS7, Line 115:
: else {
> I agree with you. But my concern is, if a user (say by mistake) tries input
You should think of both merge functions as standalone functions. They could
appear anywhere in any query. There is no guarantee that casting will be
injected. Casting is not specific to insert/upsert. If you are worried about
the error message being confusing for uuid case, there are workarounds. But we
shouldn't change the generic case to fit the special case.
--
To view, visit https://asterix-gerrit.ics.uci.edu/3370
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22100d3ff29864b8bfd54b0decb183e5056fdb4a
Gerrit-Change-Number: 3370
Gerrit-PatchSet: 8
Gerrit-Owner: Hussain Towaileb
Gerrit-Reviewer: Ali Alsuliman
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Dmitry Lychagin
Gerrit-Reviewer: Hussain Towaileb
Gerrit-Reviewer: Hussain Towaileb
Gerrit-Reviewer: Jenkins
Gerrit-Reviewer: Till Westmann
Gerrit-Comment-Date: Sat, 11 May 2019 05:30:27 +
Gerrit-HasComments: Yes
Change in asterixdb[master]: [ASTERIXDB-2535][COMP] Fix uuid present in insert/upsert statement
Ali Alsuliman has posted comments on this change. (
https://asterix-gerrit.ics.uci.edu/3370 )
Change subject: [ASTERIXDB-2535][COMP] Fix uuid present in insert/upsert
statement
..
Patch Set 8:
(5 comments)
https://asterix-gerrit.ics.uci.edu/#/c/3370/8/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/BuiltinFunctions.java
File
asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/BuiltinFunctions.java:
https://asterix-gerrit.ics.uci.edu/#/c/3370/8/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/BuiltinFunctions.java@2186
PS8, Line 2186: addFunction(RECORD_MERGE_IGNORE_DUPLICATES,
RecordMergeTypeComputer.INSTANCE_IGNORE_DUPLICATES, true);
with this now being a new function and not related to UUID, we should add few
test cases to make sure you are covered. You could add them in a later change.
https://asterix-gerrit.ics.uci.edu/#/c/3370/7/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/RecordMergeTypeComputer.java
File
asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/RecordMergeTypeComputer.java:
https://asterix-gerrit.ics.uci.edu/#/c/3370/7/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/RecordMergeTypeComputer.java@115
PS7, Line 115:
: else {
> Short answer:
I think relying on ARecordCaster being injected isn't a good idea. We should
maintain that the duplicate fields must have the same types. That's at
compile-time here in the type computer. At runtime, you get to choose what to
do for duplicates; whether to
1) throw duplicate exception when left and right have different types (normal
current behaviour) (actually it throws exception when they have different
values, i.e. if "id": 2, "id": 3. For "id":2, "id":2, it picks left)
2) ignore and pick the one on left input record (but also making sure left type
is the same as the corresponding field in output record. This will be
consistent with the type computer since it's picking the left type)
In your example, if the "id" is part of the output record schema, then we
should check that the input value type is the same as the one in the schema.
The semantics of the record-merge-ignore-duplicates "runtime" says if there are
duplicate fields, it is going to pick the left value (that's also the semantics
of normal record merge for same-value duplicates that the compiler couldn't
figure out because they are open fields).
https://asterix-gerrit.ics.uci.edu/#/c/3370/8/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/RecordMergeTypeComputer.java
File
asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/RecordMergeTypeComputer.java:
https://asterix-gerrit.ics.uci.edu/#/c/3370/8/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/typecomputer/impl/RecordMergeTypeComputer.java@49
PS8, Line 49: private RecordMergeTypeComputer() {
: this(false);
: }
:
: private RecordMergeTypeComputer(boolean isIgnoreDuplicates) {
: this.isIgnoreDuplicates = isIgnoreDuplicates;
: }
it's better to have one constructor with a boolean parameter)
https://asterix-gerrit.ics.uci.edu/#/c/3370/8/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/records/RecordMergeIgnoreDuplicatesDescriptor.java
File
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/records/RecordMergeIgnoreDuplicatesDescriptor.java:
https://asterix-gerrit.ics.uci.edu/#/c/3370/8/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/records/RecordMergeIgnoreDuplicatesDescriptor.java@36
PS8, Line 36: @MissingNullInOutFunction
just to make sure; is this annotation supposed to be for descriptors or
evaluators?
https://asterix-gerrit.ics.uci.edu/#/c/3370/8/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/records/RecordMergeIgnoreDuplicatesDescriptor.java@68
PS8, Line 68: RecordMergeIgnoreDuplicatesEvaluator
we could remove this class and just extend RecordMergeEvaluator right here.
--
To view, visit https://asterix-gerrit.ics.uci.edu/3370
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I22100d3ff29864b8bfd54b0decb183e5056fdb4a
Gerrit-Change-Number: 3370
Gerrit-PatchSet: 8
Gerrit-Owner: Hussain Towaileb
Gerrit-Reviewer: Ali Alsuliman
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Dmitry Lychagin
Gerrit-Reviewer: Hussain Towaileb
Gerrit-Reviewer: Hussain Towaileb
Gerrit-Reviewer: Jenkins
Gerrit-Reviewer: Till Westmann
Gerrit-Comment-Date: Fri, 10 May 2019 19:55:33 +
Gerrit-HasComments: Yes
Change in asterixdb[master]: [ASTERIXDB-2555][RT][COMP] Make hash join use logical comparison
Ali Alsuliman has posted comments on this change. (
https://asterix-gerrit.ics.uci.edu/3387 )
Change subject: [ASTERIXDB-2555][RT][COMP] Make hash join use logical comparison
..
Patch Set 4:
(6 comments)
https://asterix-gerrit.ics.uci.edu/#/c/3387/2/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/evaluators/TuplePairEvaluatorFactory.java
File
hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/evaluators/TuplePairEvaluatorFactory.java:
https://asterix-gerrit.ics.uci.edu/#/c/3387/2/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/evaluators/TuplePairEvaluatorFactory.java@52
PS2, Line 52: ITuplePairCo
> agreed, I think synchronization is not required here
Done
https://asterix-gerrit.ics.uci.edu/#/c/3387/2/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/evaluators/TuplePairEvaluatorFactory.java@78
PS2, Line 78: reseter.reset(tuplePairRef, leftAccessor, leftIndex,
rightAccessor, rightIndex);
: conditionEvaluator.evaluate(tuplePairRef, res);
: return
booleanInspector.getBooleanValue(res.getByteArray(), res.getStartOffset(),
res.getLength()) ? 0 : 1;
: }
:
> I think I prefer patch set 1 here- but not with a ton of conviction
Done
https://asterix-gerrit.ics.uci.edu/#/c/3387/2/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/evaluators/TuplePairEvaluatorFactory.java@84
PS2, Line 84: int leftTupleIndex, IFrameTupleAccessor
rightAccessor, int rightTupleIndex) {
: ref.reset(leftAccessor, leftTupleIndex,
rightAccessor, rightTupleIndex);
: }
:
: priva
> should we use a ternary op here?
Done
https://asterix-gerrit.ics.uci.edu/#/c/3387/2/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/evaluators/TuplePairEvaluatorFactory.java@116
PS2, Line 116: rride
: public int getFieldCount() {
: return refLeft.getFieldCount() +
refRight.getFieldCount();
: }
:
> ternary?
Done
https://asterix-gerrit.ics.uci.edu/#/c/3387/2/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/dataflow/value/ITuplePairComparator.java
File
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/dataflow/value/ITuplePairComparator.java:
https://asterix-gerrit.ics.uci.edu/#/c/3387/2/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/dataflow/value/ITuplePairComparator.java@27
PS2, Line 27: CriticalPath
> What does this do?
It does nothing. It's just a reminder that this method will be executed a lot,
not only once per tuple, but every time you join a tuple with a bunch of other
tuples.
https://asterix-gerrit.ics.uci.edu/#/c/3387/2/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/util/JoinUtil.java
File
hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/util/JoinUtil.java:
https://asterix-gerrit.ics.uci.edu/#/c/3387/2/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/util/JoinUtil.java@36
PS2, Line 36:
> ok. let's move it to the "join" package and may be rename to HybridHashJoin
Done
--
To view, visit https://asterix-gerrit.ics.uci.edu/3387
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If1834967fdd913fdc76003f09636b2450d07cd5e
Gerrit-Change-Number: 3387
Gerrit-PatchSet: 4
Gerrit-Owner: Ali Alsuliman
Gerrit-Reviewer: Ali Alsuliman
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Dmitry Lychagin
Gerrit-Reviewer: Jenkins
Gerrit-Reviewer: Michael Blow
Gerrit-Reviewer: Till Westmann
Gerrit-Comment-Date: Fri, 10 May 2019 11:30:34 +
Gerrit-HasComments: Yes
Change in asterixdb[master]: [ASTERIXDB-2555][RT][COMP] Make hash join use logical comparison
Hello Anon. E. Moose (1000171), Till Westmann, Jenkins, Michael Blow, Dmitry Lychagin, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3387 to look at the new patch set (#4). Change subject: [ASTERIXDB-2555][RT][COMP] Make hash join use logical comparison .. [ASTERIXDB-2555][RT][COMP] Make hash join use logical comparison - user model changes: no - storage format changes: no - interface changes: no Details: This patch changes the hash join operator to use the join condition to evaluate if tuples are equal when joining. Binary physical comparators have been removed. The join condition evaluator is in TuplePairEvaluator. - extraced TuplePairEvaluatorFactory out of nested loop join class into a separate class so that it is shared among nested loop and hash join. - switched from FrameTuplePairComparator to ITuplePairComparator in in OptimizedHybridHashJoin and InMemoryHashJoin. - moved debugging code from OptimizedHybridHashJoin into a separate class, JoinUtil. - temporarily made the logical comparison of multisets use raw binary comparison instead of returning null until the logic is implemented. - made IBinaryBooleanInspector a functional interface and updated the implementations. - updated record and array test cases to reflect the new behaviour of hash join where logical comparison could produce null. Also, updated sorting, group by and distinct test cases since the input data has been modified. - added two new input files arrays1nulls.adm & arrays2nulls.adm to be used by the open dataset. previous arrays1.adm & arrays2.adm are used by the closed dataset since it cannot accept arrays with null values. Change-Id: If1834967fdd913fdc76003f09636b2450d07cd5e --- A asterixdb/asterix-app/data/complex/arrays1nulls.adm A asterixdb/asterix-app/data/complex/arrays2nulls.adm M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/distinct/array/array.2.update.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/group-by/gby-array/gby-array.2.update.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/join/hash_join_array/hash_join_array.2.update.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/sorting/arrays/arrays.2.update.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/results/distinct/array/array.3.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/group-by/gby-array/gby-array.3.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/group-by/gby-array/gby-array.5.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_array/hash_join_array.7.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_array/hash_join_array.8.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.03.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.04.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.05.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.06.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.07.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.08.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.09.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.10.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.11.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.12.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.13.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.14.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.15.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.16.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.17.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.18.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.19.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.20.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/sorting/arrays/arrays.3.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/sorting/arrays/arrays.5.adm M
Change in asterixdb[master]: [ASTERIXDB-2555][RT][COMP] Make hash join use logical comparison
Hello Anon. E. Moose (1000171), Till Westmann, Jenkins, Michael Blow, Dmitry Lychagin, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3387 to look at the new patch set (#3). Change subject: [ASTERIXDB-2555][RT][COMP] Make hash join use logical comparison .. [ASTERIXDB-2555][RT][COMP] Make hash join use logical comparison - user model changes: no - storage format changes: no - interface changes: no Details: This patch changes the hash join operator to use the join condition to evaluate if tuples are equal when joining. Binary physical comparators have been removed. The join condition evaluator is in TuplePairEvaluator. - extraced TuplePairEvaluatorFactory out of nested loop join class into a separate class so that it is shared among nested loop and hash join. - switched from FrameTuplePairComparator to ITuplePairComparator in in OptimizedHybridHashJoin and InMemoryHashJoin. - moved debugging code from OptimizedHybridHashJoin into a separate class, JoinUtil. - temporarily made the logical comparison of multisets use raw binary comparison instead of returning null until the logic is implemented. - made IBinaryBooleanInspector a functional interface and updated the implementations. - updated record and array test cases to reflect the new behaviour of hash join where logical comparison could produce null. Also, updated sorting, group by and distinct test cases since the input data has been modified. - added two new input files arrays1nulls.adm & arrays2nulls.adm to be used by the open dataset. previous arrays1.adm & arrays2.adm are used by the closed dataset since it cannot accept arrays with null values. Change-Id: If1834967fdd913fdc76003f09636b2450d07cd5e --- A asterixdb/asterix-app/data/complex/arrays1nulls.adm A asterixdb/asterix-app/data/complex/arrays2nulls.adm M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/distinct/array/array.2.update.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/group-by/gby-array/gby-array.2.update.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/join/hash_join_array/hash_join_array.2.update.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/sorting/arrays/arrays.2.update.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/results/distinct/array/array.3.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/group-by/gby-array/gby-array.3.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/group-by/gby-array/gby-array.5.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_array/hash_join_array.7.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_array/hash_join_array.8.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.03.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.04.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.05.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.06.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.07.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.08.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.09.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.10.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.11.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.12.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.13.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.14.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.15.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.16.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.17.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.18.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.19.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.20.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/sorting/arrays/arrays.3.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/sorting/arrays/arrays.5.adm M
Change in asterixdb[master]: [ASTERIXDB-2555][RT][COMP] Make hash join use logical comparison
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3387 ) Change subject: [ASTERIXDB-2555][RT][COMP] Make hash join use logical comparison .. Patch Set 2: (2 comments) https://asterix-gerrit.ics.uci.edu/#/c/3387/2/asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.03.adm File asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.03.adm: https://asterix-gerrit.ics.uci.edu/#/c/3387/2/asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.03.adm@a16 PS2, Line 16: > why these records no longer join? "address" values seem to be the same on b it's because of the "apt" field being null. Logical comparison will render the whole comparison as null which will render the comparison as false. This is inner join. https://asterix-gerrit.ics.uci.edu/#/c/3387/2/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/util/JoinUtil.java File hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/util/JoinUtil.java: https://asterix-gerrit.ics.uci.edu/#/c/3387/2/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/util/JoinUtil.java@36 PS2, Line 36: public String printPartitionInfo(BitSet spilledStatus, OptimizedHybridHashJoin.SIDE whichSide, int numOfPartitions, > This method is specific to OptimizedHybridHashJoin, so why did we move into it's a debugging print which is not used in OptimizedHybridHashJoin. Having it there in production code isn't suitable, I think. But I can move it back if you see otherwise. If it's about the package, I can keep this Util class in the same package as the hybrid join. Let me know. -- To view, visit https://asterix-gerrit.ics.uci.edu/3387 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If1834967fdd913fdc76003f09636b2450d07cd5e Gerrit-Change-Number: 3387 Gerrit-PatchSet: 2 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Thu, 09 May 2019 18:35:31 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [NO ISSUE][COMP] Reduce memory requirements for metadata only queries
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3385 ) Change subject: [NO ISSUE][COMP] Reduce memory requirements for metadata only queries .. Patch Set 2: Code-Review+1 -- To view, visit https://asterix-gerrit.ics.uci.edu/3385 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0d8902f88f61953e839e2fa0b06a77c324edbc57 Gerrit-Change-Number: 3385 Gerrit-PatchSet: 2 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Thu, 09 May 2019 18:27:01 + Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2555][RT][COMP] Make hash join use logical comparison
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3387 ) Change subject: [ASTERIXDB-2555][RT][COMP] Make hash join use logical comparison .. Patch Set 2: (1 comment) https://asterix-gerrit.ics.uci.edu/#/c/3387/2/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/evaluators/TuplePairEvaluatorFactory.java File hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/evaluators/TuplePairEvaluatorFactory.java: https://asterix-gerrit.ics.uci.edu/#/c/3387/2/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/evaluators/TuplePairEvaluatorFactory.java@52 PS2, Line 52: synchronized not sure why the old code needed this to be synchronized. -- To view, visit https://asterix-gerrit.ics.uci.edu/3387 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If1834967fdd913fdc76003f09636b2450d07cd5e Gerrit-Change-Number: 3387 Gerrit-PatchSet: 2 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Comment-Date: Thu, 09 May 2019 15:57:50 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [ASTERIXDB-2555][RT][COMP] Make hash join use logical comparison
Hello Anon. E. Moose (1000171), Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3387 to look at the new patch set (#2). Change subject: [ASTERIXDB-2555][RT][COMP] Make hash join use logical comparison .. [ASTERIXDB-2555][RT][COMP] Make hash join use logical comparison - user model changes: no - storage format changes: no - interface changes: no Details: This patch changes the hash join operator to use the join condition to evaluate if tuples are equal when joining. Binary physical comparators have been removed. The join condition evaluator is in TuplePairEvaluator. - extraced TuplePairEvaluatorFactory out of nested loop join class into a separate class so that it is shared among nested loop and hash join. - switched from FrameTuplePairComparator to ITuplePairComparator in in OptimizedHybridHashJoin and InMemoryHashJoin. - moved debugging code from OptimizedHybridHashJoin into a separate class, JoinUtil. - temporarily made the logical comparison of multisets use raw binary comparison instead of returning null until the logic is implemented. - made IBinaryBooleanInspector a functional interface and updated the implementations. - updated record and array test cases to reflect the new behaviour of hash join where logical comparison could produce null. Also, updated sorting, group by and distinct test cases since the input data has been modified. - added two new input files arrays1nulls.adm & arrays2nulls.adm to be used by the open dataset. previous arrays1.adm & arrays2.adm are used by the closed dataset since it cannot accept arrays with null values. Change-Id: If1834967fdd913fdc76003f09636b2450d07cd5e --- A asterixdb/asterix-app/data/complex/arrays1nulls.adm A asterixdb/asterix-app/data/complex/arrays2nulls.adm M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/distinct/array/array.2.update.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/group-by/gby-array/gby-array.2.update.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/join/hash_join_array/hash_join_array.2.update.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/sorting/arrays/arrays.2.update.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/results/distinct/array/array.3.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/group-by/gby-array/gby-array.3.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/group-by/gby-array/gby-array.5.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_array/hash_join_array.7.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_array/hash_join_array.8.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.03.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.04.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.05.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.06.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.07.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.08.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.09.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.10.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.11.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.12.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.13.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.14.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.15.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.16.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.17.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.18.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.19.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/join/hash_join_record/hash_join_record.20.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/sorting/arrays/arrays.3.adm M asterixdb/asterix-app/src/test/resources/runtimets/results/sorting/arrays/arrays.5.adm M
Change in asterixdb[master]: [NO ISSUE][COMP] Make memory requirements an operator property
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3374 ) Change subject: [NO ISSUE][COMP] Make memory requirements an operator property .. Patch Set 3: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3374 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3481ddfe163c6ce786290c540cbd05db16a7f64f Gerrit-Change-Number: 3374 Gerrit-PatchSet: 3 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Wed, 08 May 2019 17:55:34 + Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2535][COMP] Fix uuid present in insert/upsert statement
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3370 ) Change subject: [ASTERIXDB-2535][COMP] Fix uuid present in insert/upsert statement .. Patch Set 4: (1 comment) https://asterix-gerrit.ics.uci.edu/#/c/3370/4/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/records/RecordMergeUuidIgnoreDuplicateEvaluator.java File asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/records/RecordMergeUuidIgnoreDuplicateEvaluator.java: https://asterix-gerrit.ics.uci.edu/#/c/3370/4/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/evaluators/functions/records/RecordMergeUuidIgnoreDuplicateEvaluator.java@37 PS4, Line 37: isUuidIgnoreDuplicate; do we need this? this function should always ignore uuid duplicates. -- To view, visit https://asterix-gerrit.ics.uci.edu/3370 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I22100d3ff29864b8bfd54b0decb183e5056fdb4a Gerrit-Change-Number: 3370 Gerrit-PatchSet: 4 Gerrit-Owner: Hussain Towaileb Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Hussain Towaileb Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Tue, 07 May 2019 03:35:36 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [NO ISSUE][COMP] Make memory requirements an operator property
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3374 ) Change subject: [NO ISSUE][COMP] Make memory requirements an operator property .. Patch Set 2: Code-Review+1 -- To view, visit https://asterix-gerrit.ics.uci.edu/3374 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3481ddfe163c6ce786290c540cbd05db16a7f64f Gerrit-Change-Number: 3374 Gerrit-PatchSet: 2 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Tue, 07 May 2019 03:24:05 + Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][COMP] Make memory requirements an operator property
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3374 ) Change subject: [NO ISSUE][COMP] Make memory requirements an operator property .. Patch Set 2: (2 comments) https://asterix-gerrit.ics.uci.edu/#/c/3374/2/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/SetAsterixPhysicalOperatorsRule.java File asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/SetAsterixPhysicalOperatorsRule.java: https://asterix-gerrit.ics.uci.edu/#/c/3374/2/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/SetAsterixPhysicalOperatorsRule.java@135 PS2, Line 135: inputFileSize should we remove this too? it seems that the inputFileSize depends on the frame limit which we can calculate later at code gen when we get the frame limit from the local memory requirements object. no? https://asterix-gerrit.ics.uci.edu/#/c/3374/2/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/HybridToInMemoryHashJoinRule.java File hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/HybridToInMemoryHashJoinRule.java: https://asterix-gerrit.ics.uci.edu/#/c/3374/2/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/HybridToInMemoryHashJoinRule.java@37 PS2, Line 37: rewritePre I think with "Pre" we will not descend to inputs if the currently visited operator returns true (i.e. changed from hybrid to in memory). no? -- To view, visit https://asterix-gerrit.ics.uci.edu/3374 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3481ddfe163c6ce786290c540cbd05db16a7f64f Gerrit-Change-Number: 3374 Gerrit-PatchSet: 2 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Tue, 07 May 2019 03:23:58 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [ASTERIXDB-2552][RT] Reuse objects on reseting buffer manager
Ali Alsuliman has submitted this change and it was merged. ( https://asterix-gerrit.ics.uci.edu/3372 ) Change subject: [ASTERIXDB-2552][RT] Reuse objects on reseting buffer manager .. [ASTERIXDB-2552][RT] Reuse objects on reseting buffer manager - user model changes: no - storage format changes: no - interface changes: yes Details: The "VariableFrameMemoryManager" releases the physical frames objects, the logical frames objects and the free slots objects on reset(). This patch is to retain those objects and only release them on close() (as also stated in the interface). - added close() to IFrameFreeSlotPolicy to make the interface similar to the other interfaces (IFrameBufferManager, IFramePool, ... etc) where close() is supposed to release while reset() is supposed to only reset but retain resources. - renamed "InMemorySortRuntimeFactory" to "MicroSortRuntimeFactory". Change-Id: I88bea27e5024c621412ef609475e2a7ba1913afa Reviewed-on: https://asterix-gerrit.ics.uci.edu/3372 Contrib: Jenkins Tested-by: Jenkins Integration-Tests: Jenkins Reviewed-by: Dmitry Lychagin Reviewed-by: Till Westmann --- M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/MicroStableSortPOperator.java R hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/sort/MicroSortRuntimeFactory.java M hyracks-fullstack/algebricks/algebricks-tests/src/test/java/org/apache/hyracks/algebricks/tests/pushruntime/PushRuntimeTest.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/FrameFreeSlotBiggestFirst.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/FrameFreeSlotLastFit.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/FrameFreeSlotSmallestFit.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/IFrameBufferManager.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/IFrameFreeSlotPolicy.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/VariableDeletableTupleMemoryManager.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/VariableFrameMemoryManager.java 10 files changed, 90 insertions(+), 43 deletions(-) Approvals: Jenkins: Verified; ; Verified Anon. E. Moose (1000171): Dmitry Lychagin: Looks good to me, but someone else must approve Till Westmann: Looks good to me, approved Objections: Jenkins: Violations found diff --git a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/MicroStableSortPOperator.java b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/MicroStableSortPOperator.java index c6a7d9d..413c1a4 100644 --- a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/MicroStableSortPOperator.java +++ b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/MicroStableSortPOperator.java @@ -32,7 +32,7 @@ import org.apache.hyracks.algebricks.data.IBinaryComparatorFactoryProvider; import org.apache.hyracks.algebricks.data.INormalizedKeyComputerFactoryProvider; import org.apache.hyracks.algebricks.runtime.base.IPushRuntimeFactory; -import org.apache.hyracks.algebricks.runtime.operators.sort.InMemorySortRuntimeFactory; +import org.apache.hyracks.algebricks.runtime.operators.sort.MicroSortRuntimeFactory; import org.apache.hyracks.api.dataflow.value.IBinaryComparatorFactory; import org.apache.hyracks.api.dataflow.value.INormalizedKeyComputerFactory; import org.apache.hyracks.api.dataflow.value.RecordDescriptor; @@ -80,7 +80,7 @@ i++; } -IPushRuntimeFactory runtime = new InMemorySortRuntimeFactory(sortFields, nkcf, comps, null, maxNumberOfFrames); +IPushRuntimeFactory runtime = new MicroSortRuntimeFactory(sortFields, nkcf, comps, null, maxNumberOfFrames); builder.contributeMicroOperator(op, runtime, recDescriptor); ILogicalOperator src = op.getInputs().get(0).getValue(); builder.contributeGraphEdge(src, 0, op, 0); diff --git a/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/sort/InMemorySortRuntimeFactory.java b/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/sort/MicroSortRuntimeFactory.java similarity index 96%
Change in asterixdb[master]: [ASTERIXDB-2552][RT] Reuse objects on reseting buffer manager
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3372 ) Change subject: [ASTERIXDB-2552][RT] Reuse objects on reseting buffer manager .. Patch Set 2: (1 comment) https://asterix-gerrit.ics.uci.edu/#/c/3372/2/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/FrameFreeSlotBiggestFirst.java File hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/FrameFreeSlotBiggestFirst.java: https://asterix-gerrit.ics.uci.edu/#/c/3372/2/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/FrameFreeSlotBiggestFirst.java@104 PS2, Line 104: // TODO(ali): fix to not release resources > In this change? If you insist. But I would prefer to do it in a later change to keep this one small and manageable. -- To view, visit https://asterix-gerrit.ics.uci.edu/3372 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I88bea27e5024c621412ef609475e2a7ba1913afa Gerrit-Change-Number: 3372 Gerrit-PatchSet: 2 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Dmitry Lychagin Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Fri, 03 May 2019 17:57:10 + Gerrit-HasComments: Yes
Change in asterixdb[master]: [NO ISSUE][HYR][RT] Handle calls to close() properly in hash join
Ali Alsuliman has submitted this change and it was merged. (
https://asterix-gerrit.ics.uci.edu/3375 )
Change subject: [NO ISSUE][HYR][RT] Handle calls to close() properly in hash
join
..
[NO ISSUE][HYR][RT] Handle calls to close() properly in hash join
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
Hash join build activity assumes that upon calling close()
the join state must have been allocated in open() and
therefore, it will directly call "close build" using the join
state. However, if the build activity throws an exception
in open(), then it could happen that the join state has
not been initialized. Calling close() on the build activity
will lead to a NPE since it will use the join state directly.
This patch checks if the join state has been initialized. If
so, it will close the build as usual.
Change-Id: I41fff7deed5e56818bfefb4d36b8018a9f512cbb
Reviewed-on: https://asterix-gerrit.ics.uci.edu/3375
Contrib: Jenkins
Tested-by: Jenkins
Integration-Tests: Jenkins
Reviewed-by: Murtadha Hubail
---
M
hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoinOperatorDescriptor.java
1 file changed, 9 insertions(+), 7 deletions(-)
Approvals:
Jenkins: Verified; ; Verified
Anon. E. Moose (1000171):
Murtadha Hubail: Looks good to me, approved
Objections:
Jenkins: Violations found
diff --git
a/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoinOperatorDescriptor.java
b/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoinOperatorDescriptor.java
index 403c492..81d08b2 100644
---
a/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoinOperatorDescriptor.java
+++
b/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoinOperatorDescriptor.java
@@ -315,13 +315,15 @@
@Override
public void close() throws HyracksDataException {
-state.hybridHJ.closeBuild();
-if (isFailed) {
-state.hybridHJ.clearBuildTempFiles();
-} else {
-ctx.setStateObject(state);
-if (LOGGER.isTraceEnabled()) {
-LOGGER.trace("OptimizedHybridHashJoin closed its
build phase");
+if (state.hybridHJ != null) {
+state.hybridHJ.closeBuild();
+if (isFailed) {
+state.hybridHJ.clearBuildTempFiles();
+} else {
+ctx.setStateObject(state);
+if (LOGGER.isTraceEnabled()) {
+LOGGER.trace("OptimizedHybridHashJoin closed
its build phase");
+}
}
}
}
--
To view, visit https://asterix-gerrit.ics.uci.edu/3375
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I41fff7deed5e56818bfefb4d36b8018a9f512cbb
Gerrit-Change-Number: 3375
Gerrit-PatchSet: 2
Gerrit-Owner: Ali Alsuliman
Gerrit-Reviewer: Ali Alsuliman
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins
Gerrit-Reviewer: Murtadha Hubail
Gerrit-Reviewer: Till Westmann
Change in asterixdb[master]: [NO ISSUE][OTR] GlobalConfig Cleanup
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3376 ) Change subject: [NO ISSUE][OTR] GlobalConfig Cleanup .. Patch Set 1: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3376 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie59a646d7d2eedaf1360c97c355f827d80917adf Gerrit-Change-Number: 3376 Gerrit-PatchSet: 1 Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Thu, 02 May 2019 19:10:17 + Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][FUN] Move negative tests to seperate test unit
Ali Alsuliman has submitted this change and it was merged. ( https://asterix-gerrit.ics.uci.edu/3371 ) Change subject: [NO ISSUE][FUN] Move negative tests to seperate test unit .. [NO ISSUE][FUN] Move negative tests to seperate test unit - user model changes: no - storage format changes: no - interface changes: no Details: Moved array functions negative tests to a seperate test unit. Change-Id: I0a3bfce402e4b7091c40483b5802db7b27afa717 Reviewed-on: https://asterix-gerrit.ics.uci.edu/3371 Contrib: Jenkins Sonar-Qube: Jenkins Tested-by: Jenkins Integration-Tests: Jenkins Reviewed-by: Dmitry Lychagin --- R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_concat/array_concat.4.ddl.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.01.query.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.02.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.03.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.04.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.05.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.06.query.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.07.query.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.08.query.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.09.query.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_intersect/array_intersect.4.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_intersect/array_intersect.5.ddl.sqlpp D asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_intersect/array_intersect.5.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_range/array_range.4.ddl.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_replace/array_replace.5.ddl.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_symdiff/array_symdiff.4.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_symdiff/array_symdiff.5.ddl.sqlpp D asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_symdiff/array_symdiff.5.query.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_symdiffn/array_symdiffn.4.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_symdiffn/array_symdiffn.5.ddl.sqlpp D asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_symdiffn/array_symdiffn.5.query.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_union/array_union.4.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_union/array_union.5.ddl.sqlpp D asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_union/array_union.5.query.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml 25 files changed, 58 insertions(+), 179 deletions(-) Approvals: Jenkins: Verified; No violations found; ; Verified Anon. E. Moose (1000171): Dmitry Lychagin: Looks good to me, approved diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_concat/array_concat.5.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_concat/array_concat.4.ddl.sqlpp similarity index 100% rename from asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_concat/array_concat.5.ddl.sqlpp rename to asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_concat/array_concat.4.ddl.sqlpp diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_concat/array_concat.4.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.01.query.sqlpp similarity index 97% rename from asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_concat/array_concat.4.query.sqlpp rename to asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.01.query.sqlpp index 805dd99..a65a081 100755 ---
Change in asterixdb[master]: [NO ISSUE][HYR][RT] Handle calls to close() properly in hash join
Ali Alsuliman has uploaded this change for review. (
https://asterix-gerrit.ics.uci.edu/3375
Change subject: [NO ISSUE][HYR][RT] Handle calls to close() properly in hash
join
..
[NO ISSUE][HYR][RT] Handle calls to close() properly in hash join
- user model changes: no
- storage format changes: no
- interface changes: no
Details:
Hash join build activity assumes that upon calling close()
the join state must have been allocated in open() and
therefore, it will directly call "close build" using the join
state. However, if the build activity throws an exception
in open(), then it could happen that the join state has
not been initialized. Calling close() on the build activity
will lead to a NPE since it will use the join state directly.
This patch checks if the join state has been initialized. If
so, it will close the build as usual.
Change-Id: I41fff7deed5e56818bfefb4d36b8018a9f512cbb
---
M
hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoinOperatorDescriptor.java
1 file changed, 9 insertions(+), 7 deletions(-)
git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb
refs/changes/75/3375/1
diff --git
a/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoinOperatorDescriptor.java
b/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoinOperatorDescriptor.java
index 403c492..81d08b2 100644
---
a/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoinOperatorDescriptor.java
+++
b/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/OptimizedHybridHashJoinOperatorDescriptor.java
@@ -315,13 +315,15 @@
@Override
public void close() throws HyracksDataException {
-state.hybridHJ.closeBuild();
-if (isFailed) {
-state.hybridHJ.clearBuildTempFiles();
-} else {
-ctx.setStateObject(state);
-if (LOGGER.isTraceEnabled()) {
-LOGGER.trace("OptimizedHybridHashJoin closed its
build phase");
+if (state.hybridHJ != null) {
+state.hybridHJ.closeBuild();
+if (isFailed) {
+state.hybridHJ.clearBuildTempFiles();
+} else {
+ctx.setStateObject(state);
+if (LOGGER.isTraceEnabled()) {
+LOGGER.trace("OptimizedHybridHashJoin closed
its build phase");
+}
}
}
}
--
To view, visit https://asterix-gerrit.ics.uci.edu/3375
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I41fff7deed5e56818bfefb4d36b8018a9f512cbb
Gerrit-Change-Number: 3375
Gerrit-PatchSet: 1
Gerrit-Owner: Ali Alsuliman
Change in asterixdb[master]: [ASTERIXDB-2552][RT] Reuse objects on reseting buffer manager
Hello Anon. E. Moose (1000171), Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3372 to look at the new patch set (#2). Change subject: [ASTERIXDB-2552][RT] Reuse objects on reseting buffer manager .. [ASTERIXDB-2552][RT] Reuse objects on reseting buffer manager - user model changes: no - storage format changes: no - interface changes: yes Details: The "VariableFrameMemoryManager" releases the physical frames objects, the logical frames objects and the free slots objects on reset(). This patch is to retain those objects and only release them on close() (as also stated in the interface). - added close() to IFrameFreeSlotPolicy to make the interface similar to the other interfaces (IFrameBufferManager, IFramePool, ... etc) where close() is supposed to release while reset() is supposed to only reset but retain resources. - renamed "InMemorySortRuntimeFactory" to "MicroSortRuntimeFactory". Change-Id: I88bea27e5024c621412ef609475e2a7ba1913afa --- M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/MicroStableSortPOperator.java R hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/sort/MicroSortRuntimeFactory.java M hyracks-fullstack/algebricks/algebricks-tests/src/test/java/org/apache/hyracks/algebricks/tests/pushruntime/PushRuntimeTest.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/FrameFreeSlotBiggestFirst.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/FrameFreeSlotLastFit.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/FrameFreeSlotSmallestFit.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/IFrameBufferManager.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/IFrameFreeSlotPolicy.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/VariableDeletableTupleMemoryManager.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/VariableFrameMemoryManager.java 10 files changed, 90 insertions(+), 43 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/72/3372/2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3372 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I88bea27e5024c621412ef609475e2a7ba1913afa Gerrit-Change-Number: 3372 Gerrit-PatchSet: 2 Gerrit-Owner: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins
Change in asterixdb[master]: [ASTERIXDB-2552][RT] Reuse objects on reseting buffer manager
Ali Alsuliman has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/3372 Change subject: [ASTERIXDB-2552][RT] Reuse objects on reseting buffer manager .. [ASTERIXDB-2552][RT] Reuse objects on reseting buffer manager - user model changes: no - storage format changes: no - interface changes: yes Details: The "VariableFrameMemoryManager" releases the physical frames objects, the logical frames objects and the free slots objects on reset(). This patch is to retain those objects and only release them on close() (as also stated in the interface). - added close() to IFrameFreeSlotPolicy to make the interface similar to the other interfaces (IFrameBufferManager, IFramePool, ... etc) where close() is supposed to release while reset() is supposed to only reset but retain resources. - renamed "InMemorySortRuntimeFactory" to "MicroSortRuntimeFactory". Change-Id: I88bea27e5024c621412ef609475e2a7ba1913afa --- M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/MicroStableSortPOperator.java R hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/sort/MicroSortRuntimeFactory.java M hyracks-fullstack/algebricks/algebricks-tests/src/test/java/org/apache/hyracks/algebricks/tests/pushruntime/PushRuntimeTest.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/FrameFreeSlotBiggestFirst.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/FrameFreeSlotLastFit.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/FrameFreeSlotSmallestFit.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/IFrameBufferManager.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/IFrameFreeSlotPolicy.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/VariableDeletableTupleMemoryManager.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/buffermanager/VariableFrameMemoryManager.java 10 files changed, 90 insertions(+), 40 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/72/3372/1 diff --git a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/MicroStableSortPOperator.java b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/MicroStableSortPOperator.java index c6a7d9d..413c1a4 100644 --- a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/MicroStableSortPOperator.java +++ b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/MicroStableSortPOperator.java @@ -32,7 +32,7 @@ import org.apache.hyracks.algebricks.data.IBinaryComparatorFactoryProvider; import org.apache.hyracks.algebricks.data.INormalizedKeyComputerFactoryProvider; import org.apache.hyracks.algebricks.runtime.base.IPushRuntimeFactory; -import org.apache.hyracks.algebricks.runtime.operators.sort.InMemorySortRuntimeFactory; +import org.apache.hyracks.algebricks.runtime.operators.sort.MicroSortRuntimeFactory; import org.apache.hyracks.api.dataflow.value.IBinaryComparatorFactory; import org.apache.hyracks.api.dataflow.value.INormalizedKeyComputerFactory; import org.apache.hyracks.api.dataflow.value.RecordDescriptor; @@ -80,7 +80,7 @@ i++; } -IPushRuntimeFactory runtime = new InMemorySortRuntimeFactory(sortFields, nkcf, comps, null, maxNumberOfFrames); +IPushRuntimeFactory runtime = new MicroSortRuntimeFactory(sortFields, nkcf, comps, null, maxNumberOfFrames); builder.contributeMicroOperator(op, runtime, recDescriptor); ILogicalOperator src = op.getInputs().get(0).getValue(); builder.contributeGraphEdge(src, 0, op, 0); diff --git a/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/sort/InMemorySortRuntimeFactory.java b/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/sort/MicroSortRuntimeFactory.java similarity index 96% rename from hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/sort/InMemorySortRuntimeFactory.java rename to hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/sort/MicroSortRuntimeFactory.java index
Change in asterixdb[master]: [NO ISSUE][FUN] Move negative tests to seperate test unit
Ali Alsuliman has uploaded this change for review. ( https://asterix-gerrit.ics.uci.edu/3371 Change subject: [NO ISSUE][FUN] Move negative tests to seperate test unit .. [NO ISSUE][FUN] Move negative tests to seperate test unit - user model changes: no - storage format changes: no - interface changes: no Details: Moved array functions negative tests to a seperate test unit. Change-Id: I0a3bfce402e4b7091c40483b5802db7b27afa717 --- R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_concat/array_concat.4.ddl.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.01.query.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.02.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.03.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.04.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.05.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.06.query.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.07.query.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.08.query.sqlpp C asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.09.query.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_intersect/array_intersect.4.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_intersect/array_intersect.5.ddl.sqlpp D asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_intersect/array_intersect.5.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_range/array_range.4.ddl.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_replace/array_replace.5.ddl.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_symdiff/array_symdiff.4.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_symdiff/array_symdiff.5.ddl.sqlpp D asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_symdiff/array_symdiff.5.query.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_symdiffn/array_symdiffn.4.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_symdiffn/array_symdiffn.5.ddl.sqlpp D asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_symdiffn/array_symdiffn.5.query.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_union/array_union.4.query.sqlpp R asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_union/array_union.5.ddl.sqlpp D asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_union/array_union.5.query.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml 25 files changed, 58 insertions(+), 179 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/71/3371/1 diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_concat/array_concat.5.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_concat/array_concat.4.ddl.sqlpp similarity index 100% rename from asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_concat/array_concat.5.ddl.sqlpp rename to asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_concat/array_concat.4.ddl.sqlpp diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_concat/array_concat.4.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.01.query.sqlpp similarity index 97% rename from asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_concat/array_concat.4.query.sqlpp rename to asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.01.query.sqlpp index 805dd99..a65a081 100755 --- a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_concat/array_concat.4.query.sqlpp +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/array_fun/array_errors/array_errors.01.query.sqlpp @@ -22,6 +22,4 @@ * Expected Res : Error due to processing different list types */ -use TinySocial;
Change in asterixdb[master]: [NO ISSUE][COMP] Refactor physical window operator
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3369 ) Change subject: [NO ISSUE][COMP] Refactor physical window operator .. Patch Set 2: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3369 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3863fa3d298aef53d4098be9fc17b0451eb2c23e Gerrit-Change-Number: 3369 Gerrit-PatchSet: 2 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Wed, 01 May 2019 19:27:03 + Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][COMP] Refactor physical window operator
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3369 ) Change subject: [NO ISSUE][COMP] Refactor physical window operator .. Patch Set 1: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3369 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3863fa3d298aef53d4098be9fc17b0451eb2c23e Gerrit-Change-Number: 3369 Gerrit-PatchSet: 1 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Wed, 01 May 2019 17:20:20 + Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2552][RT] Implement micro external sort for subplans
Ali Alsuliman has submitted this change and it was merged. ( https://asterix-gerrit.ics.uci.edu/3353 ) Change subject: [ASTERIXDB-2552][RT] Implement micro external sort for subplans .. [ASTERIXDB-2552][RT] Implement micro external sort for subplans - user model changes: no - storage format changes: no - interface changes: no Details: This patch is to change the in memory sort used in subplans to be a micro external sort to avoid out of memory error. - added reset() to the runs merger to allow reusing the runs merger. - renamed "InMemoryStableSortPOperator" to "MicroStableSortPOperator". - changed the tag from "IN_MEMORY_STABLE_SORT" to "MICRO_STABLE_SORT". - added test cases. Change-Id: I930849d644c60d461d2869c9773b85e49b46fbdb Reviewed-on: https://asterix-gerrit.ics.uci.edu/3353 Contrib: Jenkins Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Dmitry Lychagin --- A asterixdb/asterix-app/src/test/resources/optimizerts/queries/micro_external_sort/micro_external_sort.sqlpp M asterixdb/asterix-app/src/test/resources/optimizerts/results/ASTERIXDB-2402.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.10.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.11.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.12.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.13.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.14.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.16.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.17.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.9.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/group-by/sugar-06-distinct.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/insert-return-custom-result.plan A asterixdb/asterix-app/src/test/resources/optimizerts/results/micro_external_sort/micro_external_sort.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/nest_aggregate.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/nested_loj3.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/query-ASTERIXDB-1263.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/query-issue785.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/similarity/jaccard-similarity-join-dual-order.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/similarity/jaccard-similarity-join-right-ahead.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/udfs/query-ASTERIXDB-1308-1.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/udfs/query-ASTERIXDB-1308-2.plan A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/sorting/micro_external_sort/micro_external_sort.1.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/sorting/micro_external_sort/micro_external_sort.1.adm M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/base/PhysicalOperatorTag.java M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/AbstractStableSortPOperator.java R hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/MicroStableSortPOperator.java M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/StableSortPOperator.java M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/EnforceStructuralPropertiesRule.java M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/PushNestedOrderByUnderPreSortedGroupByRule.java M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/SetAlgebricksPhysicalOperatorsRule.java M hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/sort/InMemorySortRuntimeFactory.java M hyracks-fullstack/algebricks/algebricks-tests/src/test/java/org/apache/hyracks/algebricks/tests/pushruntime/PushRuntimeTest.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/AbstractExternalSortRunMerger.java 33 files changed, 328 insertions(+), 101 deletions(-) Approvals: Jenkins:
Change in asterixdb[master]: [ASTERIXDB-2552][RT] Implement micro external sort for subplans
Hello Anon. E. Moose (1000171), Till Westmann, Jenkins, Michael Blow, Dmitry Lychagin, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3353 to look at the new patch set (#3). Change subject: [ASTERIXDB-2552][RT] Implement micro external sort for subplans .. [ASTERIXDB-2552][RT] Implement micro external sort for subplans - user model changes: no - storage format changes: no - interface changes: no Details: This patch is to change the in memory sort used in subplans to be a micro external sort to avoid out of memory error. - added reset() to the runs merger to allow reusing the runs merger. - renamed "InMemoryStableSortPOperator" to "MicroStableSortPOperator". - changed the tag from "IN_MEMORY_STABLE_SORT" to "MICRO_STABLE_SORT". - added test cases. Change-Id: I930849d644c60d461d2869c9773b85e49b46fbdb --- A asterixdb/asterix-app/src/test/resources/optimizerts/queries/micro_external_sort/micro_external_sort.sqlpp M asterixdb/asterix-app/src/test/resources/optimizerts/results/ASTERIXDB-2402.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.10.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.11.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.12.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.13.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.14.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.16.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.17.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.9.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/group-by/sugar-06-distinct.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/insert-return-custom-result.plan A asterixdb/asterix-app/src/test/resources/optimizerts/results/micro_external_sort/micro_external_sort.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/nest_aggregate.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/nested_loj3.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/query-ASTERIXDB-1263.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/query-issue785.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/similarity/jaccard-similarity-join-dual-order.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/similarity/jaccard-similarity-join-right-ahead.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/udfs/query-ASTERIXDB-1308-1.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/udfs/query-ASTERIXDB-1308-2.plan A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/sorting/micro_external_sort/micro_external_sort.1.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/sorting/micro_external_sort/micro_external_sort.1.adm M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/base/PhysicalOperatorTag.java M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/AbstractStableSortPOperator.java R hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/MicroStableSortPOperator.java M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/StableSortPOperator.java M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/EnforceStructuralPropertiesRule.java M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/PushNestedOrderByUnderPreSortedGroupByRule.java M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/SetAlgebricksPhysicalOperatorsRule.java M hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/sort/InMemorySortRuntimeFactory.java M hyracks-fullstack/algebricks/algebricks-tests/src/test/java/org/apache/hyracks/algebricks/tests/pushruntime/PushRuntimeTest.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/AbstractExternalSortRunMerger.java 33 files changed, 328 insertions(+), 101 deletions(-) git pull
Change in asterixdb[master]: [NO ISSUE][API] Return Bad Request on Insufficient Cluster Resources
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3368 ) Change subject: [NO ISSUE][API] Return Bad Request on Insufficient Cluster Resources .. Patch Set 1: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3368 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I15500bfa4051a97fe3f5277d318b5a1187da225c Gerrit-Change-Number: 3368 Gerrit-PatchSet: 1 Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Tue, 30 Apr 2019 18:59:09 + Gerrit-HasComments: No
Change in asterixdb[master]: [NO ISSUE][COMP] Refactor physical operator assignment rules
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3367 ) Change subject: [NO ISSUE][COMP] Refactor physical operator assignment rules .. Patch Set 1: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3367 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I502f367464a6fabc595cff804722f793e052570f Gerrit-Change-Number: 3367 Gerrit-PatchSet: 1 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Tue, 30 Apr 2019 18:37:58 + Gerrit-HasComments: No
Change in asterixdb[master]: [ASTERIXDB-2552][RT] Implement micro external sort for subplans
Ali Alsuliman has posted comments on this change. (
https://asterix-gerrit.ics.uci.edu/3353 )
Change subject: [ASTERIXDB-2552][RT] Implement micro external sort for subplans
..
Patch Set 1:
(2 comments)
https://asterix-gerrit.ics.uci.edu/#/c/3353/1/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/sort/MicroSortRuntimeFactory.java
File
hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/sort/MicroSortRuntimeFactory.java:
https://asterix-gerrit.ics.uci.edu/#/c/3353/1/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/sort/MicroSortRuntimeFactory.java@94
PS1, Line 94: public void close() throws HyracksDataException {
> Where do we close the "writer"?
It's changed now with the refactoring. It's closed here explicitly.
https://asterix-gerrit.ics.uci.edu/#/c/3353/1/hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/sort/MicroSortRuntimeFactory.java@127
PS1, Line 127: }
> should we call super.fail() to fail the writer?
It's changed now. The writer is failed here in the wrappingWriter.
--
To view, visit https://asterix-gerrit.ics.uci.edu/3353
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I930849d644c60d461d2869c9773b85e49b46fbdb
Gerrit-Change-Number: 3353
Gerrit-PatchSet: 1
Gerrit-Owner: Ali Alsuliman
Gerrit-Reviewer: Ali Alsuliman
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Dmitry Lychagin
Gerrit-Reviewer: Jenkins
Gerrit-Reviewer: Michael Blow
Gerrit-Reviewer: Till Westmann
Gerrit-Comment-Date: Tue, 30 Apr 2019 06:04:46 +
Gerrit-HasComments: Yes
Change in asterixdb[master]: [ASTERIXDB-2552][RT] Implement micro external sort for subplans
Hello Anon. E. Moose (1000171), Till Westmann, Jenkins, Michael Blow, Dmitry Lychagin, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/3353 to look at the new patch set (#2). Change subject: [ASTERIXDB-2552][RT] Implement micro external sort for subplans .. [ASTERIXDB-2552][RT] Implement micro external sort for subplans - user model changes: no - storage format changes: no - interface changes: no Details: This patch is to change the in memory sort used in subplans to be a micro external sort to avoid out of memory error. - added reset() to the runs merger to allow reusing the runs merger. - renamed "InMemoryStableSortPOperator" to "MicroStableSortPOperator". - changed the tag from "IN_MEMORY_STABLE_SORT" to "MICRO_STABLE_SORT". - added test cases. Change-Id: I930849d644c60d461d2869c9773b85e49b46fbdb --- A asterixdb/asterix-app/src/test/resources/optimizerts/queries/micro_external_sort/micro_external_sort.sqlpp M asterixdb/asterix-app/src/test/resources/optimizerts/results/ASTERIXDB-2402.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.10.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.11.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.12.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.13.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.14.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.16.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.17.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/aggregate-sql-sugar/distinct_mixed/distinct_mixed.9.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/group-by/sugar-06-distinct.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/insert-return-custom-result.plan A asterixdb/asterix-app/src/test/resources/optimizerts/results/micro_external_sort/micro_external_sort.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/nest_aggregate.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/nested_loj3.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/query-ASTERIXDB-1263.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/query-issue785.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/similarity/jaccard-similarity-join-dual-order.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/similarity/jaccard-similarity-join-right-ahead.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/udfs/query-ASTERIXDB-1308-1.plan M asterixdb/asterix-app/src/test/resources/optimizerts/results/udfs/query-ASTERIXDB-1308-2.plan A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/sorting/micro_external_sort/micro_external_sort.1.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/sorting/micro_external_sort/micro_external_sort.1.adm M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/base/PhysicalOperatorTag.java M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/AbstractStableSortPOperator.java R hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/MicroStableSortPOperator.java M hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/StableSortPOperator.java M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/EnforceStructuralPropertiesRule.java M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/PushNestedOrderByUnderPreSortedGroupByRule.java M hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/SetAlgebricksPhysicalOperatorsRule.java M hyracks-fullstack/algebricks/algebricks-runtime/src/main/java/org/apache/hyracks/algebricks/runtime/operators/sort/InMemorySortRuntimeFactory.java M hyracks-fullstack/algebricks/algebricks-tests/src/test/java/org/apache/hyracks/algebricks/tests/pushruntime/PushRuntimeTest.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/AbstractExternalSortRunMerger.java 33 files changed, 329 insertions(+), 101 deletions(-) git pull
Change in asterixdb[master]: [ASTERIXDB-2552][RT] Refactor runs generator and merger
Ali Alsuliman has submitted this change and it was merged. ( https://asterix-gerrit.ics.uci.edu/3363 ) Change subject: [ASTERIXDB-2552][RT] Refactor runs generator and merger .. [ASTERIXDB-2552][RT] Refactor runs generator and merger - user model changes: no - storage format changes: no - interface changes: no Details: Moved the writer and sorter out of the merger to allow micro external sort to use the run generator and merger plus minor clean-ups. Change-Id: Idda31c92cbcddba5ebef8bbbf7855b9c8293dd51 Reviewed-on: https://asterix-gerrit.ics.uci.edu/3363 Contrib: Jenkins Tested-by: Jenkins Integration-Tests: Jenkins Reviewed-by: Dmitry Lychagin --- M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/group/sort/ExternalSortGroupByRunMerger.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/group/sort/SortGroupByOperatorDescriptor.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/AbstractExternalSortRunMerger.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/AbstractFrameSorter.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/AbstractSortRunGenerator.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/AbstractSorterOperatorDescriptor.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/ExternalSortOperatorDescriptor.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/ExternalSortRunMerger.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/FrameSorterMergeSort.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/FrameSorterQuickSort.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/HeapSortRunGenerator.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/HybridTopKSortRunGenerator.java M hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/TopKSorterOperatorDescriptor.java M hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/unit/SortGroupbyTest.java 14 files changed, 209 insertions(+), 234 deletions(-) Approvals: Jenkins: Verified; ; Verified Anon. E. Moose (1000171): Dmitry Lychagin: Looks good to me, approved Objections: Jenkins: Violations found diff --git a/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/group/sort/ExternalSortGroupByRunMerger.java b/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/group/sort/ExternalSortGroupByRunMerger.java index 364f1c7..cb2da4c 100644 --- a/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/group/sort/ExternalSortGroupByRunMerger.java +++ b/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/group/sort/ExternalSortGroupByRunMerger.java @@ -32,7 +32,6 @@ import org.apache.hyracks.dataflow.std.group.IAggregatorDescriptorFactory; import org.apache.hyracks.dataflow.std.group.preclustered.PreclusteredGroupWriter; import org.apache.hyracks.dataflow.std.sort.AbstractExternalSortRunMerger; -import org.apache.hyracks.dataflow.std.sort.ISorter; /** * Group-by aggregation is pushed into multi-pass merge of external sort. @@ -44,28 +43,23 @@ private final RecordDescriptor inputRecordDesc; private final RecordDescriptor partialAggRecordDesc; private final RecordDescriptor outRecordDesc; - private final int[] groupFields; private final IAggregatorDescriptorFactory mergeAggregatorFactory; private final IAggregatorDescriptorFactory partialAggregatorFactory; private final boolean localSide; - private final int[] mergeSortFields; private final int[] mergeGroupFields; private final IBinaryComparator[] groupByComparators; -public ExternalSortGroupByRunMerger(IHyracksTaskContext ctx, ISorter frameSorter, List runs, -int[] sortFields, RecordDescriptor inRecordDesc, RecordDescriptor partialAggRecordDesc, -RecordDescriptor outRecordDesc, int framesLimit, IFrameWriter writer, int[] groupFields, -INormalizedKeyComputer nmk, IBinaryComparator[] comparators, +public ExternalSortGroupByRunMerger(IHyracksTaskContext ctx, List runs, int[] sortFields, +RecordDescriptor inRecordDesc, RecordDescriptor partialAggRecordDesc, RecordDescriptor outRecordDesc, +int framesLimit, int[]
Change in asterixdb[master]: [NO ISSUE][COMP] Refactor physical operators for GroupBy and DistinctBy
Ali Alsuliman has posted comments on this change. ( https://asterix-gerrit.ics.uci.edu/3364 ) Change subject: [NO ISSUE][COMP] Refactor physical operators for GroupBy and DistinctBy .. Patch Set 4: Code-Review+2 -- To view, visit https://asterix-gerrit.ics.uci.edu/3364 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I93f17a6a3e0df6587e518e18eea01ca54a289b4c Gerrit-Change-Number: 3364 Gerrit-PatchSet: 4 Gerrit-Owner: Dmitry Lychagin Gerrit-Reviewer: Ali Alsuliman Gerrit-Reviewer: Anon. E. Moose (1000171) Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Till Westmann Gerrit-Comment-Date: Fri, 26 Apr 2019 22:56:24 + Gerrit-HasComments: No
