Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-28 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 14:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-verify-no-installer-app/4244/
 (11/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 14
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-28 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 14:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-asterix-app-sql-execution/3815/
 (1/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 14
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-28 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 14:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-asterix-app-java10/110/ 
(4/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 14
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-28 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 14:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-source-format/3775/ 
(6/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 14
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-28 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 14:

Build Started https://asterix-jenkins.ics.uci.edu/job/hyracks-gerrit/3725/ 
(10/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 14
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-28 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 14:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-source-assemblies/4028/ 
(9/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 14
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-28 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 14:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-verify-storage-jre10/112/ (7/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 14
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-28 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 14:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-sonar/7745/ (3/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 14
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-28 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 14:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-ensure-ancestor/1821/ 
(2/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 14
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-28 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 14:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/9281/ (5/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 14
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-28 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 14:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-verify-asterix-app/4265/ 
(13/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 14
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-28 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 14:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-cancellation-test/3801/ 
(12/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 14
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-28 Thread Ali Alsuliman (Code Review)
Hello Anon. E. Moose #1000171, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/2393

to look at the new patch set (#14).

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..

[ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

- user model changes: yes
- storage format changes: no
- interface changes: yes

details:
- new plan for sort operation which includes sampling and
replicating the stream of data to be sorted. Sort-merge connector
is removed from the plan. The sorted result now is in multiple partitions.
- new optimization rule to check whether full parallel sort is applicable.
- new Forward operator to read the replicated sort input stream and
to receive the ouput of the sampling.
- new sequential merge connector to merge a globally ordered result residing
in multiple partitions (in addition to the connector's partition computer).
- "asterix-lang-aql/pom.xml" is changed as a result of refactoring
code related to the range map handling.
- new private sampling function to generate the range map object
(local & global functions) & their type computers.

user model changes:
- new hint to disable the parallel sort optimization. Parser is changed
to accommodate this hint.

interface changes:
- "ILogicalOperatorVisitor.java" includes Forward Operator
- "ITuplePartitionComputer.java" includes initialize() to enable partitioner
to do some initialization. FieldRangePartitionComputerFactory uses it to
pick a range map
- "ITuplePartitionComputerFactory.java". createPartitioner() is changed to
createPartitioner(IHyracksTaskContext hyracksTaskContext). Context is needed
for transferring the range map throught the context.

Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java
A 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/CheckFullParallelSortRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/SweepIllegalNonfunctionalFunctions.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineAllNtsInSubplanVisitor.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineLeftNtsInSubplanJoinFlatteningVisitor.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/SubplanSpecialFlatteningCheckVisitor.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/LangExpressionToPlanTranslator.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/resource/OperatorResourcesComputer.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/resource/PlanStagesGenerator.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/resource/RequiredCapacityVisitor.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/dataflow/TestPartitionComputerFactory.java
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index-join/leftouterjoin-probe-pidx-with-join-btree-sidx_02.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index-join/secondary-self-equi-join-index-only.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index-join/secondary-self-equi-join.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index-join/sidx-non-idxonly-to-sidx-idxonly-equi-join_01.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-01-disable-idxonly.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-01.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-02.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-03.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-04.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-05.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-06.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-07.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-08.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/introhashpartitionmerge.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-basic/fulltext-contains-without-option.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-basic/fulltext-contains.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-basic/ngram-contains-panic.plan
M 

Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-28 Thread Ali Alsuliman (Code Review)
Ali Alsuliman has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 13:

(16 comments)

https://asterix-gerrit.ics.uci.edu/#/c/2393/13/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/GlobalSamplingAggregateDescriptor.java
File 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/GlobalSamplingAggregateDescriptor.java:

Line 173: ByteArrayInputStream byteStream = new 
ByteArrayInputStream(localSamples.getByteArray(),
> don't create new instance for each tuple. use ByteArrayAccessibleInputStrea
ByteArrayAccessibleInputStream is not available at asterix-runtime. It's in 
asterix-external-data. Any idea how to go about this?


Line 175: AOrderedList localSamplesList = listSerde.deserialize(new 
DataInputStream(byteStream));
> don't create DateInputStream instance for each tuple. create it once with B
ByteArrayAccessibleInputStream is not available at asterix-runtime. It's in 
asterix-external-data. Any idea how to go about this?


https://asterix-gerrit.ics.uci.edu/#/c/2393/13/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/base/OperatorAnnotations.java
File 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/base/OperatorAnnotations.java:

Line 25: public static final String USE_RANGE_CONNECTOR = 
"USE_RANGE_CONNECTOR"; // -->
> should we rename USE_RANGE_CONECTOR to USE_STATIC_RANGE and rename the valu
Done


https://asterix-gerrit.ics.uci.edu/#/c/2393/13/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/AbstractStableSortPOperator.java
File 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/AbstractStableSortPOperator.java:

Line 82: if 
(sortOp.getAnnotations().get(OperatorAnnotations.USE_DYNAMIC_RANGE) == 
Boolean.TRUE) {
> should the same be required for USE_RANGE_CONNECTOR annotation? We know the
Check my other comment related to this. Probably we should talk about this.


https://asterix-gerrit.ics.uci.edu/#/c/2393/13/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/utils/DotFormatGenerator.java
File 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/utils/DotFormatGenerator.java:

Line 52: private static final LogicalOperatorDotVisitor dotVisitor = new 
LogicalOperatorDotVisitor();
> why was it made static? this could be a problem if we want to call it concu
Well, it's not meant for concurrent use. But I made it non-static now just in 
case.


https://asterix-gerrit.ics.uci.edu/#/c/2393/13/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/EnforceStructuralPropertiesRule.java
File 
hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/EnforceStructuralPropertiesRule.java:

Line 117: private static final FunctionIdentifier LOCAL_SAMPLING_FUN = 
AlgebricksBuiltinFunctions.LOCAL_SAMPLING;
> just a thought. you can pass these two function identifiers through the con
Done


Line 144: debug(" Optimizing operator " + op.getPhysicalOperator() 
+ ".\n");
> the idea behind calling Logger.isDebugEnabled() is to avoid string concaten
Sure. Done.


Line 165: trace(STR_PROP_FOR + op.getPhysicalOperator() + ": " + 
op.getDeliveredPhysicalProperties() + "\n");
> same issue with potentially unnecessary message construction. revert to usi
Done


Line 170: private boolean physOptimizeOp(Mutable opRef, 
IPhysicalPropertiesVector required,
> minor. can you put this method back after getStartChildIndex()? This would 
Done. I reverted all clean-up/renaming/restructuring code to make it even 
easier for reviewing. This rule is really messy.


Line 611: if 
(parentOp.getAnnotations().containsKey(OperatorAnnotations.USE_RANGE_CONNECTOR))
 {
> I think that we once we start requiring OrderedPartitionedProperty for chil
I didn't want to change the old code which handles the static range map since I 
have some concerns about its implementation. In fact, I'm not sure if the 
implementation of static range map is correct. Take a look at 
RangePartitionMergeExchangePOperator. I have no history of that change, so I 
don't know why it was done this way. My thought was to leave 
discussing/changing it for another change since the parallel sort change keeps 
growing. If you prefer, I can include it in the parallel sort change, but we 
have to discuss some of the issues I found.


Line 666: // options for range partitioning: 1. range map computed at 
run time, 2. no range map??
> I think the two options 

Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-27 Thread Dmitry Lychagin (Code Review)
Dmitry Lychagin has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 13:

(18 comments)

still reviewing, but here's the first batch of comments

https://asterix-gerrit.ics.uci.edu/#/c/2393/11/asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj
File asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj:

Line 206: private static final String GEN_FIELDS_HINT = "gen-fields";
> Done. I changed it to "unpartitioned".
I'd set the OrderByClause.isFullParallel to false by default. Then set to 
'true' in SQL++ parser (unless there's 'unpartitioned' hint). Then AQL parser 
can remain the same and will continue to generate the same sort as before.


https://asterix-gerrit.ics.uci.edu/#/c/2393/13/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/GlobalSamplingAggregateDescriptor.java
File 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/GlobalSamplingAggregateDescriptor.java:

Line 173: ByteArrayInputStream byteStream = new 
ByteArrayInputStream(localSamples.getByteArray(),
don't create new instance for each tuple. use ByteArrayAccessibleInputStream


Line 175: AOrderedList localSamplesList = listSerde.deserialize(new 
DataInputStream(byteStream));
don't create DateInputStream instance for each tuple. create it once with 
ByteArrayAccessibleInputStream then reset that stream's content


https://asterix-gerrit.ics.uci.edu/#/c/2393/13/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/base/OperatorAnnotations.java
File 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/base/OperatorAnnotations.java:

Line 25: public static final String USE_RANGE_CONNECTOR = 
"USE_RANGE_CONNECTOR"; // -->
should we rename USE_RANGE_CONECTOR to USE_STATIC_RANGE and rename the value 
accordingly?


https://asterix-gerrit.ics.uci.edu/#/c/2393/13/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/AbstractStableSortPOperator.java
File 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/AbstractStableSortPOperator.java:

Line 82: if 
(sortOp.getAnnotations().get(OperatorAnnotations.USE_DYNAMIC_RANGE) == 
Boolean.TRUE) {
should the same be required for USE_RANGE_CONNECTOR annotation? We know the 
range statically in that case, but the OrderedParititonedProperty requirement 
is the same. Right?


https://asterix-gerrit.ics.uci.edu/#/c/2393/13/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/utils/DotFormatGenerator.java
File 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/utils/DotFormatGenerator.java:

Line 52: private static final LogicalOperatorDotVisitor dotVisitor = new 
LogicalOperatorDotVisitor();
why was it made static? this could be a problem if we want to call it 
concurrently. Not sure if we'll ever do that, but it's not a good pattern to 
follow.


https://asterix-gerrit.ics.uci.edu/#/c/2393/13/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/EnforceStructuralPropertiesRule.java
File 
hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/EnforceStructuralPropertiesRule.java:

Line 117: private static final FunctionIdentifier LOCAL_SAMPLING_FUN = 
AlgebricksBuiltinFunctions.LOCAL_SAMPLING;
just a thought. you can pass these two function identifiers through the 
constructor. that way they won't need to be defined in algebricks


Line 144: debug(" Optimizing operator " + op.getPhysicalOperator() 
+ ".\n");
the idea behind calling Logger.isDebugEnabled() is to avoid string 
concatenation (i.e. create log message only if it will be logged). with your 
change the message is always created which is suboptimal. can we revert this?


Line 165: trace(STR_PROP_FOR + op.getPhysicalOperator() + ": " + 
op.getDeliveredPhysicalProperties() + "\n");
same issue with potentially unnecessary message construction. revert to using 
Logger.isTraceEnabled()


Line 170: private boolean physOptimizeOp(Mutable opRef, 
IPhysicalPropertiesVector required,
minor. can you put this method back after getStartChildIndex()? This would make 
the review easier. Let's reorder methods in this rule in a follow up change if 
you want to do that.


Line 611: if 
(parentOp.getAnnotations().containsKey(OperatorAnnotations.USE_RANGE_CONNECTOR))
 {
I think that we once we start requiring OrderedPartitionedProperty for children 
in AbstractStableSortPOperator for USE_RANGE_CONNECTOR then this branch should 
go away. What do you think? This 

Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-24 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 13: Integration-Tests+1

Integration Tests Successful

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/6653/ 
: SUCCESS

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 13
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-24 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 13: Contrib+1

BAD Compatibility Tests Successful

https://asterix-jenkins.ics.uci.edu/job/asterixbad-compat/3297/ : SUCCESS

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 13
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-24 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 13:

BAD Compatibility Tests Started 
https://asterix-jenkins.ics.uci.edu/job/asterixbad-compat/3297/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 13
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-24 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 13:

WARNING: THIS CHANGE CONTAINS CROSS-PRODUCT CHANGES IN:
* asterixdb
* hyracks-fullstack

PLEASE REVIEW CAREFULLY AND LOOK FOR API CHANGES!

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 13
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-24 Thread Anon. E. Moose (Code Review)
Anon. E. Moose #1000171 has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 13: Contrib-2

Analytics Compatibility Compilation Failed
https://goo.gl/7BxruX : UNSTABLE

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 13
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-24 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 13:

Integration Tests Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/6653/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 13
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-24 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 13:

Build Started https://asterix-jenkins.ics.uci.edu/job/hyracks-gerrit/3700/ 
(3/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 13
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-24 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 13:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-source-assemblies/4003/ 
(13/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 13
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-24 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 13:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-cancellation-test/3776/ 
(2/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 13
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-24 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 13:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-verify-no-installer-app/4219/
 (4/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 13
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-24 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 13:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-verify-storage-jre10/87/ (6/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 13
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-24 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 13:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-sonar/7720/ (9/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 13
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-24 Thread Ali Alsuliman (Code Review)
Hello Anon. E. Moose #1000171, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/2393

to look at the new patch set (#13).

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..

[ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

- user model changes: yes
- storage format changes: no
- interface changes: yes

details:
- new plan for sort operation which includes sampling and
replicating the stream of data to be sorted. Sort-merge connector
is removed from the plan. The sorted result now is in multiple partitions.
- new optimization rule to check whether full parallel sort is applicable.
- new Forward operator to read the replicated sort input stream and
to receive the ouput of the sampling.
- new sequential merge connector to merge a globally ordered result residing
in multiple partitions (in addition to the connector's partition computer).
- "asterix-lang-aql/pom.xml" is changed as a result of refactoring
code related to the range map handling.
- new private sampling function to generate the range map object
(local & global functions) & their type computers.

user model changes:
- new hint to disable the parallel sort optimization. Parser is changed
to accommodate this hint.

interface changes:
- "ILogicalOperatorVisitor.java" includes Forward Operator
- "ITuplePartitionComputer.java" includes initialize() to enable partitioner
to do some initialization. FieldRangePartitionComputerFactory uses it to
pick a range map
- "ITuplePartitionComputerFactory.java". createPartitioner() is changed to
createPartitioner(IHyracksTaskContext hyracksTaskContext). Context is needed
for transferring the range map throught the context.

Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java
A 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/CheckFullParallelSortRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/SweepIllegalNonfunctionalFunctions.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineAllNtsInSubplanVisitor.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineLeftNtsInSubplanJoinFlatteningVisitor.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/SubplanSpecialFlatteningCheckVisitor.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/LangExpressionToPlanTranslator.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/resource/OperatorResourcesComputer.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/resource/PlanStagesGenerator.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/resource/RequiredCapacityVisitor.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/dataflow/TestPartitionComputerFactory.java
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index-join/leftouterjoin-probe-pidx-with-join-btree-sidx_02.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index-join/secondary-self-equi-join-index-only.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index-join/secondary-self-equi-join.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index-join/sidx-non-idxonly-to-sidx-idxonly-equi-join_01.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-01-disable-idxonly.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-01.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-02.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-03.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-04.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-05.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-06.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-07.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-08.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/introhashpartitionmerge.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-basic/fulltext-contains-without-option.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-basic/fulltext-contains.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-basic/ngram-contains-panic.plan
M 

Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-24 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 13:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-verify-storage/4327/ (12/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 13
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-24 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 13:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-asterix-app-sql-execution/3790/
 (1/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 13
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-24 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 13:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-asterix-app-java10/85/ 
(7/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 13
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-24 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 13:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/9256/ (11/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 13
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-24 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 13:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-verify-asterix-app/4240/ 
(5/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 13
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-24 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 13:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-ensure-ancestor/1796/ 
(8/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 13
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-24 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 13:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-source-format/3750/ 
(10/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 13
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-23 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 12:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-source-assemblies/4002/ 
(13/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-23 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 12:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-verify-storage-jre10/86/ (8/10)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-23 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 12:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-verify-asterix-app/4239/ 
(10/11)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-23 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 12:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-asterix-app-java10/84/ 
(6/9)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-23 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 12:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-asterix-app-sql-execution/3789/
 (1/4)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-23 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 12:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-sonar/7719/ (5/8)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-23 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 12:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-ensure-ancestor/1795/ 
(7/9)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-23 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 12:

Build Started https://asterix-jenkins.ics.uci.edu/job/hyracks-gerrit/3699/ 
(9/11)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-23 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 12:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-cancellation-test/3775/ 
(11/12)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-23 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 12:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-verify-no-installer-app/4218/
 (12/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-23 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 12:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-source-format/3749/ (2/5)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-23 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 12:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/9255/ (3/7)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-23 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 12:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-verify-storage/4326/ (4/8)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-23 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 12: Integration-Tests-1

Integration Tests Failed

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/6652/ 
: FAILURE

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-23 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 12: -Integration-Tests

Integration Tests Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/6652/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-23 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 12: Integration-Tests-1

Integration Tests Failed

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/6651/ 
: FAILURE

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-23 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 12:

Integration Tests Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/6651/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-23 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 12:

WARNING: THIS CHANGE CONTAINS CROSS-PRODUCT CHANGES IN:
* asterixdb
* hyracks-fullstack

PLEASE REVIEW CAREFULLY AND LOOK FOR API CHANGES!

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-23 Thread Anon. E. Moose (Code Review)
Anon. E. Moose #1000171 has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 12: Contrib-2

Analytics Compatibility Compilation Failed
https://goo.gl/EBQEev : UNSTABLE

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-23 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 12:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-source-assemblies/4001/ 
(10/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-23 Thread Ali Alsuliman (Code Review)
Hello Anon. E. Moose #1000171, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/2393

to look at the new patch set (#12).

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..

[ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

- user model changes: yes
- storage format changes: no
- interface changes: yes

details:
- new plan for sort operation which includes sampling and
replicating the stream of data to be sorted. Sort-merge connector
is removed from the plan. The sorted result now is in multiple partitions.
- new optimization rule to check whether full parallel sort is applicable.
- new Forward operator to read the replicated sort input stream and
to receive the ouput of the sampling.
- new sequential merge connector to merge a globally ordered result residing
in multiple partitions (in addition to the connector's partition computer).
- "asterix-lang-aql/pom.xml" is changed as a result of refactoring
code related to the range map handling.
- new private sampling function to generate the range map object
(local & global functions) & their type computers.

user model changes:
- new hint to disable the parallel sort optimization. Parser is changed
to accommodate this hint.

interface changes:
- "ILogicalOperatorVisitor.java" includes Forward Operator
- "ITuplePartitionComputer.java" includes initialize() to enable partitioner
to do some initialization. FieldRangePartitionComputerFactory uses it to
pick a range map
- "ITuplePartitionComputerFactory.java". createPartitioner() is changed to
createPartitioner(IHyracksTaskContext hyracksTaskContext). Context is needed
for transferring the range map throught the context.

Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java
A 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/CheckFullParallelSortRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/SweepIllegalNonfunctionalFunctions.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineAllNtsInSubplanVisitor.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineLeftNtsInSubplanJoinFlatteningVisitor.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/SubplanSpecialFlatteningCheckVisitor.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/LangExpressionToPlanTranslator.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/resource/OperatorResourcesComputer.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/resource/PlanStagesGenerator.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/resource/RequiredCapacityVisitor.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/dataflow/TestPartitionComputerFactory.java
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index-join/leftouterjoin-probe-pidx-with-join-btree-sidx_02.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index-join/secondary-self-equi-join-index-only.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index-join/secondary-self-equi-join.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index-join/sidx-non-idxonly-to-sidx-idxonly-equi-join_01.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-01-disable-idxonly.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-01.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-02.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-03.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-04.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-05.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-06.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-07.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-08.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/introhashpartitionmerge.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-basic/fulltext-contains-without-option.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-basic/fulltext-contains.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/inverted-index-basic/ngram-contains-panic.plan
M 

Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-23 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 12:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-sonar/7718/ (3/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-23 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 12:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/9254/ (6/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-23 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 12:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-asterix-app-java10/83/ 
(4/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-23 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 12:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-verify-storage/4325/ (8/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-23 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 12:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-source-format/3748/ 
(7/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-23 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 12:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-asterix-app-sql-execution/3788/
 (1/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-23 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 12:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-verify-storage-jre10/85/ (5/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-23 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 12:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-verify-asterix-app/4238/ 
(12/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-23 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 12:

Build Started https://asterix-jenkins.ics.uci.edu/job/hyracks-gerrit/3698/ 
(11/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-23 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 12:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-ensure-ancestor/1794/ 
(2/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-23 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 12:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-cancellation-test/3774/ 
(13/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-23 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 12:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-verify-no-installer-app/4217/
 (9/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-23 Thread Ali Alsuliman (Code Review)
Ali Alsuliman has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 11:

(17 comments)

https://asterix-gerrit.ics.uci.edu/#/c/2393/11/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineLeftNtsInSubplanJoinFlatteningVisitor.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineLeftNtsInSubplanJoinFlatteningVisitor.java:

Line 385: public ILogicalOperator visitForwardOperator(ForwardOperator op, 
Void arg) throws AlgebricksException {
> union/intersect operators are not supported by this visitor.
Done. Check if it's correct.


https://asterix-gerrit.ics.uci.edu/#/c/2393/11/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/clause/OrderbyClause.java
File 
asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/clause/OrderbyClause.java:

Line 103: return isRPSSort;
> need to copy this new property in CloneAndSubstituteVariablesVisitor.visit(
Done


Line 106: public void setRPSSort(boolean isRPSSort) {
> need to give a better name to this property "sort" is redundant. "FullParal
Done


Line 112: return Objects.hash(modifierList, numFrames, numTuples, 
orderbyList);
> rangeMap and isRPSSort should be used in hashCode()
Done


Line 116: public boolean equals(Object object) {
> rangeMap and isRPSSort should be used in equals(). we don't really need an 
Done


https://asterix-gerrit.ics.uci.edu/#/c/2393/11/asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj
File asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj:

Line 206: private static final String NO_PARALLEL_SORT_HINT = "no-rps-sort";
> can we give this hint a better name? it's on the orderby clause, so "-sort"
Done. I changed it to "unpartitioned".

BTW, I didn't add this hint to AQL since I assumed we are deprecating it. 
Should I add it to AQL as well? or?


Line 3048:   
oc.setRangeMap(RangeMapBuilder.parseHint(hint.substring(RANGE_HINT.length()), 
parserFactory));
> why do we need to create and pass SqllppParserFactory when we're already in
Done


Line 3050:   throw new ParseException(e.getMessage());
> need to add source location: throw SqlppParseException(getSourceLocation(ge
Done


https://asterix-gerrit.ics.uci.edu/#/c/2393/11/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/expressions/AbstractLogicalExpression.java
File 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/expressions/AbstractLogicalExpression.java:

Line 41: public void setParameters(Object[] expressionParameters) {
> why can't we use AbstractFunctionCallExpression.setOpaqueParameters() for t
I don't think that's the right way to go about this. The behaviour of 
TypeInferer is gradually diverging from its semantics. In my case, it is quite 
confusing to use TypeInferer to pass information like "number of partitions" (I 
know it does the job). We should fix this some time soon as it is becoming a 
growing need to pass information from expressions to function descriptors. For 
now, I changed the code to use TypeInferer.


https://asterix-gerrit.ics.uci.edu/#/c/2393/11/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/IsomorphismOperatorVisitor.java
File 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/IsomorphismOperatorVisitor.java:

Line 604: return true;
> need to implement this. look at other operators in this class.
Done


https://asterix-gerrit.ics.uci.edu/#/c/2393/11/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/properties/IPartitioningProperty.java
File 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/properties/IPartitioningProperty.java:

Line 72: /**
> it seems that it should return a new object if there need to be any changes
I remembered why I had this note. In "OrderedPartitionedProperty", even though 
it returns a new object, it also modified the calling object. I think that was 
inadvertent, and I faced some issues with it while testing. So, I changed it so 
that it does not change the current (calling) object. Check 
PropertiesUtil.applyFDsToOrderColumns. That's where I changed it. 
"OrderedPartitionedProperty" is using this method.

I had this note just in case someone had some thoughts about my fix. Note 
removed.


https://asterix-gerrit.ics.uci.edu/#/c/2393/11/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/EnforceStructuralPropertiesRule.java
File 

Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-19 Thread Dmitry Lychagin (Code Review)
Dmitry Lychagin has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 11:

Ali, I replied to you by email. Please take a look.

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 11
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-14 Thread Dmitry Lychagin (Code Review)
Dmitry Lychagin has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 11:

(18 comments)

still reviewing, but here's my first batch of comments.

https://asterix-gerrit.ics.uci.edu/#/c/2393/11/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineLeftNtsInSubplanJoinFlatteningVisitor.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineLeftNtsInSubplanJoinFlatteningVisitor.java:

Line 385: public ILogicalOperator visitForwardOperator(ForwardOperator op, 
Void arg) throws AlgebricksException {
union/intersect operators are not supported by this visitor.
they throw throw new UnsupportedOperationException(
"Nested subplans with a union operator should have been 
disqualified for this rewriting!")
should do the same for forward operator? and make sure that this visitor does 
not run on a subplan with forward operator.


https://asterix-gerrit.ics.uci.edu/#/c/2393/11/asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/clause/OrderbyClause.java
File 
asterixdb/asterix-lang-common/src/main/java/org/apache/asterix/lang/common/clause/OrderbyClause.java:

Line 103: return isRPSSort;
need to copy this new property in 
CloneAndSubstituteVariablesVisitor.visit(OrderbyClause)


Line 106: public void setRPSSort(boolean isRPSSort) {
need to give a better name to this property "sort" is redundant. "FullParallel" 
or "RangeParallel" may be?


Line 112: return Objects.hash(modifierList, numFrames, numTuples, 
orderbyList);
rangeMap and isRPSSort should be used in hashCode()


Line 116: public boolean equals(Object object) {
rangeMap and isRPSSort should be used in equals(). we don't really need an 
interface IRangeMap because we have 1 implementation. let's use that class 
instead and remove the interface. so equals() doesn't have to worry about 
different interface implementations


https://asterix-gerrit.ics.uci.edu/#/c/2393/11/asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj
File asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj:

Line 206: private static final String NO_PARALLEL_SORT_HINT = "no-rps-sort";
can we give this hint a better name? it's on the orderby clause, so "-sort" is 
redundant. may be "no-full-parallel" or "no-range-parallel"?


Line 3048:   
oc.setRangeMap(RangeMapBuilder.parseHint(hint.substring(RANGE_HINT.length()), 
parserFactory));
why do we need to create and pass SqllppParserFactory when we're already inside 
the parser? let's change RangeMapBuilder.parseHint() to take IParser instance 
as an argument. then we create the parser here: new 
SQLPPParser(hint.substring(...))


Line 3050:   throw new ParseException(e.getMessage());
need to add source location: throw 
SqlppParseException(getSourceLocation(getHintToken(token)), e.getMessage())


https://asterix-gerrit.ics.uci.edu/#/c/2393/11/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/expressions/AbstractLogicalExpression.java
File 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/expressions/AbstractLogicalExpression.java:

Line 41: public void setParameters(Object[] expressionParameters) {
why can't we use AbstractFunctionCallExpression.setOpaqueParameters() for this 
instead? and propagate them to FunctionDescriptor by implementing  
IFunctionTypeInferer for that function (BuiltinFunctions.
GLOBAL_SAMPLING)?


https://asterix-gerrit.ics.uci.edu/#/c/2393/11/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/IsomorphismOperatorVisitor.java
File 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/IsomorphismOperatorVisitor.java:

Line 604: return true;
need to implement this. look at other operators in this class.


https://asterix-gerrit.ics.uci.edu/#/c/2393/11/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/properties/IPartitioningProperty.java
File 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/properties/IPartitioningProperty.java:

Line 72: /**
it seems that it should return a new object if there need to be any changes. 
i.e. it should not modify the existing object. let's remove this comment.


https://asterix-gerrit.ics.uci.edu/#/c/2393/6/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/EnforceStructuralPropertiesRule.java
File 
hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/EnforceStructuralPropertiesRule.java:

Line 

Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-11 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 11: Integration-Tests+1

Integration Tests Successful

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/6587/ 
: SUCCESS

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 11
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-11 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 11: Contrib+1

BAD Compatibility Tests Successful

https://asterix-jenkins.ics.uci.edu/job/asterixbad-compat/3261/ : SUCCESS

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 11
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-11 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 11:

BAD Compatibility Tests Started 
https://asterix-jenkins.ics.uci.edu/job/asterixbad-compat/3261/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 11
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-11 Thread Anon. E. Moose (Code Review)
Anon. E. Moose #1000171 has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 11: Contrib-2

Analytics Compatibility Compilation Failed
https://goo.gl/inZuYn : UNSTABLE

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 11
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-11 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 11:

WARNING: THIS CHANGE CONTAINS CROSS-PRODUCT CHANGES IN:
* asterixdb
* hyracks-fullstack

PLEASE REVIEW CAREFULLY AND LOOK FOR API CHANGES!

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 11
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-11 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 11:

Integration Tests Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/6587/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 11
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-11 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 11:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-verify-asterix-app/4177/ 
(6/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 11
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-11 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 11:

Build Started https://asterix-jenkins.ics.uci.edu/job/hyracks-gerrit/3637/ 
(5/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 11
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-11 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 11:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-ensure-ancestor/1732/ 
(2/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 11
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-11 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 11:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-verify-storage-jre10/30/ (8/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 11
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-11 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 11:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-verify-storage/4263/ (10/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 11
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-11 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 11:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-asterix-app-sql-execution/3726/
 (3/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 11
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-11 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 11:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-source-format/3684/ 
(12/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 11
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-11 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 11:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-source-assemblies/3940/ 
(9/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 11
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-11 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 11:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-sonar/7656/ (13/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 11
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-11 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 11:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-verify-no-installer-app/4155/
 (7/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 11
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-11 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 11:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-cancellation-test/3713/ 
(1/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 11
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-11 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 11:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/9193/ (4/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 11
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-11 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 11:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-asterix-app-java10/28/ 
(11/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 11
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-11 Thread Ali Alsuliman (Code Review)
Hello Anon. E. Moose #1000171, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/2393

to look at the new patch set (#11).

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..

[ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

- user model changes: yes
- storage format changes: no
- interface changes: yes

details:
- new plan for sort operation which includes sampling and
replicating the stream of data to be sorted. Sort-merge connector
is removed. The sorted result now is in multiple partitions.
- new optimization rule to do the plan modifications.
- new Forward operator to read the replicated sort input stream and
to receive the ouput of the sampling.
- new sequential merge connector to merge a globally ordered result residing
in multiple partitions (in addition to the connector's partition computer).
- "asterix-lang-aql/pom.xml" is changed as a result of refactoring
code related to the range map handling.
- new private sampling function to generate the range map object
(local & global functions) & their type computers.
- the "IFunctionDescriptor" is changed to include a new method "setup" to
receive users parameters.
- the "ITuplePartitionComputer" is changed to include a new method "initialize"
to allow partitioners to perform any required initializations.
For range-based partitioner, it initializes itself with a range map.

user model changes:
- new hint to disable the parallel sort optimization. Parser is changed
to accommodate this hint.

interface changes:
- the "lookupFunction" is changed to accept setup parameters to allow
users to setup functions with parameters.
- the "createPartitioner" is changed to accept IHyracksTaskContext to enable
the range-based partitioner to receive the range map object.
- the "RangePartitionExchangePOperator" constructor is changed to allow users
to either pass the actual range map object at compile time or pass a key to
look up the range map generated at runtime.
- the "FieldRangePartitionComputerFactory" is changed accordingly to now handle
the two cases, whether a range map is supplied at compile time or a key to it
is supplied

Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
---
A 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/ForwardPOperator.java
A 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/aqlplus/parser/AQLPlusParserFactory.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/jobgen/QueryLogicalExpressionJobGen.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java
A 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceRPSRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/SetAsterixPhysicalOperatorsRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/SweepIllegalNonfunctionalFunctions.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineAllNtsInSubplanVisitor.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineLeftNtsInSubplanJoinFlatteningVisitor.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/SubplanSpecialFlatteningCheckVisitor.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/LangExpressionToPlanTranslator.java
M asterixdb/asterix-algebra/src/main/javacc/AQLPlusExtension.jj
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/resource/OperatorResourcesComputer.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/resource/PlanStagesGenerator.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/resource/RequiredCapacityVisitor.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/dataflow/TestPartitionComputerFactory.java
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index-join/leftouterjoin-probe-pidx-with-join-btree-sidx_02.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index-join/secondary-self-equi-join-index-only.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index-join/secondary-self-equi-join.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index-join/sidx-non-idxonly-to-sidx-idxonly-equi-join_01.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-01-disable-idxonly.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-01.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-02.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/btree-index/btree-sidx-idxonly-03.plan
M 

Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-11 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 10: Integration-Tests+1

Integration Tests Successful

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/6585/ 
: SUCCESS

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 10
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-11 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 10:

Integration Tests Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/6585/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 10
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-11 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 10:

WARNING: THIS CHANGE CONTAINS CROSS-PRODUCT CHANGES IN:
* asterixdb
* hyracks-fullstack

PLEASE REVIEW CAREFULLY AND LOOK FOR API CHANGES!

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 10
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-11 Thread Anon. E. Moose (Code Review)
Anon. E. Moose #1000171 has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 10: Contrib-2

Analytics Compatibility Compilation Failed
https://goo.gl/BWuGGU : UNSTABLE

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 10
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-11 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 10:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-verify-no-installer-app/4153/
 (11/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 10
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-11 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 10:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-asterix-app-java10/26/ 
(13/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 10
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-11 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 10:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/9191/ (9/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 10
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-11 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 10:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-asterix-app-sql-execution/3724/
 (8/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 10
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No


Change in asterixdb[master]: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization

2018-06-11 Thread Jenkins (Code Review)
Jenkins has posted comments on this change.

Change subject: [ASTERIXDB-2286][COMP][FUN][HYR] Parallel Sort Optimization
..


Patch Set 10:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-verify-storage-jre10/28/ (10/13)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2393
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I73e128029a46f45e6b68c23dfb9310d5de10582f
Gerrit-PatchSet: 10
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose #1000171
Gerrit-Reviewer: Ildar Absalyamov 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No


<    1   2   3   4   5   6   7   >