from:"prestonc"

[2/2] asterixdb git commit: Sonar fixes and other clean up items.

2016-07-16 Thread prestonc

Sonar fixes and other clean up items.

Change-Id: Ieb72a0fef3c181282cc4c71b881a7cb52da86818
Reviewed-on: https://asterix-gerrit.ics.uci.edu/999
Tested-by: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Till Westmann 


Project: http://git-wip-us.apache.org/repos/asf/asterixdb/repo
Commit: http://git-wip-us.apache.org/repos/asf/asterixdb/commit/bce1888e
Tree: http://git-wip-us.apache.org/repos/asf/asterixdb/tree/bce1888e
Diff: http://git-wip-us.apache.org/repos/asf/asterixdb/diff/bce1888e

Branch: refs/heads/master
Commit: bce1888eebdda3e306203d68c0c769a71dae739e
Parents: 196db5d
Author: Preston Carman 
Authored: Sat Jul 16 09:22:09 2016 -0700
Committer: Preston Carman 
Committed: Sat Jul 16 22:17:02 2016 -0700

--
 ...econdaryIndexSearchExpressionAnnotation.java |  19 +--
 .../nonvisitor/AIntervalPointable.java  |  24 ++--
 .../impl/AIntervalTypeComputer.java |   2 +-
 .../AbstractComparisonEvaluator.java|   2 +-
 .../comparisons/ComparisonHelper.java   |  45 +++---
 .../AIntervalConstructorDescriptor.java |   2 -
 .../AbstractIntervalLogicFuncDescriptor.java|   4 +-
 .../GetOverlappingIntervalDescriptor.java   |   2 +-
 .../temporal/IntervalAfterDescriptor.java   |   8 +-
 .../temporal/IntervalBeforeDescriptor.java  |   4 +-
 .../temporal/IntervalCoveredByDescriptor.java   |   8 +-
 .../temporal/IntervalCoversDescriptor.java  |   8 +-
 .../temporal/IntervalEndedByDescriptor.java |   8 +-
 .../temporal/IntervalEndsDescriptor.java|   4 +-
 .../functions/temporal/IntervalLogic.java   |  56 
 .../temporal/IntervalMeetsDescriptor.java   |   8 +-
 .../temporal/IntervalMetByDescriptor.java   |   8 +-
 .../IntervalOverlappedByDescriptor.java |   8 +-
 .../temporal/IntervalOverlapsDescriptor.java|   8 +-
 .../temporal/IntervalStartedByDescriptor.java   |   8 +-
 .../temporal/IntervalStartsDescriptor.java  |   8 +-
 .../functions/temporal/OverlapDescriptor.java   |  11 +-
 .../AbstractExpressionAnnotation.java   |  35 +
 .../ExpressionAnnotationNoCopyImpl.java |  14 +-
 .../IndexedNLJoinExpressionAnnotation.java  |  18 +--
 .../logical/AbstractLogicalOperator.java|  37 +++--
 .../operators/logical/ReplicateOperator.java|  12 +-
 .../physical/AbstractExchangePOperator.java |   1 +
 .../physical/BroadcastExchangePOperator.java|   6 +-
 .../physical/OneToOneExchangePOperator.java |   5 +-
 .../physical/RandomMergeExchangePOperator.java  |   4 +-
 .../operators/physical/ReplicatePOperator.java  |   2 +-
 .../operators/physical/UnionAllPOperator.java   |   3 -
 .../properties/IPartitioningProperty.java   |  21 ++-
 .../typing/PropagatingTypeEnvironment.java  |   4 +-
 .../algebricks/core/jobgen/impl/JobBuilder.java |  49 +++
 .../PointableBinaryComparatorFactory.java   |   2 +-
 .../hadoop/HadoopMapperOperatorDescriptor.java  |   4 +
 .../hadoop/HadoopReadOperatorDescriptor.java|   2 +-
 .../mapreduce/MapperOperatorDescriptor.java |   8 +-
 .../dataflow/std/base/AbstractStateObject.java  |  12 ++
 .../AbstractTuplePointerAccessor.java   |   2 +-
 .../IPartitionedTupleBufferManager.java |   2 +-
 .../std/buffermanager/ITupleBufferManager.java  |   2 +-
 .../VPartitionTupleBufferManager.java   |   6 +-
 .../VariableDeletableTupleMemoryManager.java|   8 +-
 .../MToNBroadcastConnectorDescriptor.java   | 136 +++
 .../MToNReplicatingConnectorDescriptor.java | 136 ---
 .../std/group/HashSpillableTableFactory.java|   8 +-
 .../dataflow/std/join/InMemoryHashJoin.java |  19 ++-
 .../std/misc/SplitOperatorDescriptor.java   |   7 -
 .../dataflow/std/sort/FrameSorterMergeSort.java |   8 +-
 .../dataflow/std/sort/TupleSorterHeapSort.java  |   4 +-
 .../std/structures/SerializableHashTable.java   |  34 ++---
 .../dataflow/std/structures/TuplePointer.java   |  35 +++--
 .../AbstractTupleMemoryManagerTest.java |   4 +-
 .../VPartitionTupleBufferManagerTest.java   |   4 +-
 .../VariableTupleMemoryManagerTest.java |   6 +-
 .../structures/SerializableHashTableTest.java   |   2 +-
 .../tests/integration/CountOfCountsTest.java|   8 +-
 .../TPCHCustomerOrderHashJoinTest.java  |  12 +-
 .../TPCHCustomerOrderNestedLoopJoinTest.java|  16 +--
 .../common/util/TreeIndexBufferCacheWarmup.java |   3 +-
 .../linearize/HilbertDoubleComparator.java  |   1 -
 .../rtree/linearize/ZCurveDoubleComparator.java |   1 -
 .../common/arraylist/DoubleArrayList.java   |   3 +-
 .../storage/common/arraylist/IntArrayList.java  |   3 +-
 .../storage/common/arraylist/LongArrayList.java |   4 +-
 68 files changed, 494 insertions(+), 474 deletions(-)

[1/2] asterixdb git commit: Interval join test questions for AQL.

2016-10-04 Thread prestonc

Repository: asterixdb
Updated Branches:
  refs/heads/master 717e9415c -> 2e3d16739


http://git-wip-us.apache.org/repos/asf/asterixdb/blob/2e3d1673/asterixdb/asterix-app/src/test/resources/runtimets/results/temporal/interval_joins/interval_before/interval_before.4.adm
--
diff --git 
a/asterixdb/asterix-app/src/test/resources/runtimets/results/temporal/interval_joins/interval_before/interval_before.4.adm
 
b/asterixdb/asterix-app/src/test/resources/runtimets/results/temporal/interval_joins/interval_before/interval_before.4.adm
new file mode 100644
index 000..f38e40b
--- /dev/null
+++ 
b/asterixdb/asterix-app/src/test/resources/runtimets/results/temporal/interval_joins/interval_before/interval_before.4.adm
@@ -0,0 +1,5 @@
+{ "staff": "Franklin", "student": "Olga" }
+{ "staff": "Maryann", "student": "Charles" }
+{ "staff": "Maryann", "student": "Frank" }
+{ "staff": "Maryann", "student": "Mary" }
+{ "staff": "Maryann", "student": "Olga" }

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/2e3d1673/asterixdb/asterix-app/src/test/resources/runtimets/results/temporal/interval_joins/interval_covers/interval_covers.3.adm
--
diff --git 
a/asterixdb/asterix-app/src/test/resources/runtimets/results/temporal/interval_joins/interval_covers/interval_covers.3.adm
 
b/asterixdb/asterix-app/src/test/resources/runtimets/results/temporal/interval_joins/interval_covers/interval_covers.3.adm
new file mode 100644
index 000..4e22101
--- /dev/null
+++ 
b/asterixdb/asterix-app/src/test/resources/runtimets/results/temporal/interval_joins/interval_covers/interval_covers.3.adm
@@ -0,0 +1,15 @@
+{ "staff": "Elisabeth", "student": "Karen" }
+{ "staff": "Elisabeth", "student": "Mary" }
+{ "staff": "Elisabeth", "student": "Steve" }
+{ "staff": "Elisabeth", "student": "Tess" }
+{ "staff": "Franklin", "student": "Karen" }
+{ "staff": "Maryann", "student": "Karen" }
+{ "staff": "Maryann", "student": "Steve" }
+{ "staff": "Maryann", "student": "Tess" }
+{ "staff": "Vicky", "student": "Charles" }
+{ "staff": "Vicky", "student": "Frank" }
+{ "staff": "Vicky", "student": "Karen" }
+{ "staff": "Vicky", "student": "Mary" }
+{ "staff": "Vicky", "student": "Olga" }
+{ "staff": "Vicky", "student": "Steve" }
+{ "staff": "Vicky", "student": "Tess" }

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/2e3d1673/asterixdb/asterix-app/src/test/resources/runtimets/results/temporal/interval_joins/interval_covers/interval_covers.4.adm
--
diff --git 
a/asterixdb/asterix-app/src/test/resources/runtimets/results/temporal/interval_joins/interval_covers/interval_covers.4.adm
 
b/asterixdb/asterix-app/src/test/resources/runtimets/results/temporal/interval_joins/interval_covers/interval_covers.4.adm
new file mode 100644
index 000..1a67e10
--- /dev/null
+++ 
b/asterixdb/asterix-app/src/test/resources/runtimets/results/temporal/interval_joins/interval_covers/interval_covers.4.adm
@@ -0,0 +1,4 @@
+{ "staff": "Zack", "student": "Charles" }
+{ "staff": "Zack", "student": "Frank" }
+{ "staff": "Zack", "student": "Mary" }
+{ "staff": "Zack", "student": "Olga" }

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/2e3d1673/asterixdb/asterix-app/src/test/resources/runtimets/results/temporal/interval_joins/interval_ends/interval_ends.3.adm
--
diff --git 
a/asterixdb/asterix-app/src/test/resources/runtimets/results/temporal/interval_joins/interval_ends/interval_ends.3.adm
 
b/asterixdb/asterix-app/src/test/resources/runtimets/results/temporal/interval_joins/interval_ends/interval_ends.3.adm
new file mode 100644
index 000..5af9101
--- /dev/null
+++ 
b/asterixdb/asterix-app/src/test/resources/runtimets/results/temporal/interval_joins/interval_ends/interval_ends.3.adm
@@ -0,0 +1,7 @@
+{ "staff": "Elisabeth", "student": "Steve" }
+{ "staff": "Elisabeth", "student": "Tess" }
+{ "staff": "Franklin", "student": "Karen" }
+{ "staff": "Maryann", "student": "Steve" }
+{ "staff": "Maryann", "student": "Tess" }
+{ "staff": "Vicky", "student": "Steve" }
+{ "staff": "Vicky", "student": "Tess" }

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/2e3d1673/asterixdb/asterix-app/src/test/resources/runtimets/results/temporal/interval_joins/interval_ends/interval_ends.4.adm
--
diff --git 
a/asterixdb/asterix-app/src/test/resources/runtimets/results/temporal/interval_joins/interval_ends/interval_ends.4.adm
 
b/asterixdb/asterix-app/src/test/resources/runtimets/results/temporal/interval_joins/interval_ends/interval_ends.4.adm
new file mode 100644
index 000..5857139
--- /dev/null
+++ 
b/asterixdb/asterix-app/src/test/resources/runtimets/results/temporal/interval_joins/interval_ends/interval_ends.4.adm
@@ -0,0 +1 @@
+{ "staff":

[26/50] [abbrv] asterixdb git commit: Add stats tracking for interval joins.

2016-10-17 Thread prestonc

Add stats tracking for interval joins.


Project: http://git-wip-us.apache.org/repos/asf/asterixdb/repo
Commit: http://git-wip-us.apache.org/repos/asf/asterixdb/commit/a9729514
Tree: http://git-wip-us.apache.org/repos/asf/asterixdb/tree/a9729514
Diff: http://git-wip-us.apache.org/repos/asf/asterixdb/diff/a9729514

Branch: refs/heads/ecarm002/interval_join_merge
Commit: a9729514d7d517a24df2d290d15e854af6d65a51
Parents: 1487f2b
Author: Preston Carman 
Authored: Thu Aug 25 11:43:57 2016 -0700
Committer: Preston Carman 
Committed: Thu Aug 25 11:43:57 2016 -0700

--
 .../intervalindex/IntervalIndexJoiner.java  | 54 
 .../InMemoryIntervalPartitionJoin.java  | 18 ++-
 .../IntervalPartitionJoiner.java| 44 
 .../IPartitionedTupleBufferManager.java |  2 +
 .../VPartitionTupleBufferManager.java   |  5 ++
 .../dataflow/std/join/AbstractMergeJoiner.java  | 10 ++--
 .../hyracks/dataflow/std/join/MergeJoiner.java  | 34 +++-
 .../dataflow/std/join/NestedLoopJoin.java   | 23 -
 .../join/NestedLoopJoinOperatorDescriptor.java  | 28 ++
 .../dataflow/std/join/RunFileStream.java| 27 --
 10 files changed, 160 insertions(+), 85 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/asterixdb/blob/a9729514/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalindex/IntervalIndexJoiner.java
--
diff --git 
a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalindex/IntervalIndexJoiner.java
 
b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalindex/IntervalIndexJoiner.java
index 6f04cad..e4c4cbe 100644
--- 
a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalindex/IntervalIndexJoiner.java
+++ 
b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalindex/IntervalIndexJoiner.java
@@ -29,11 +29,9 @@ import 
org.apache.asterix.runtime.operators.joins.IIntervalMergeJoinCheckerFacto
 import org.apache.asterix.runtime.operators.joins.IntervalJoinUtil;
 import org.apache.hyracks.api.comm.IFrameTupleAccessor;
 import org.apache.hyracks.api.comm.IFrameWriter;
-import org.apache.hyracks.api.comm.VSizeFrame;
 import org.apache.hyracks.api.context.IHyracksTaskContext;
 import org.apache.hyracks.api.dataflow.value.RecordDescriptor;
 import org.apache.hyracks.api.exceptions.HyracksDataException;
-import org.apache.hyracks.dataflow.common.comm.io.FrameTupleAppender;
 import org.apache.hyracks.dataflow.common.comm.util.FrameUtils;
 import 
org.apache.hyracks.dataflow.std.buffermanager.IPartitionedDeletableTupleBufferManager;
 import org.apache.hyracks.dataflow.std.buffermanager.ITupleAccessor;
@@ -57,23 +55,27 @@ public class IntervalIndexJoiner extends 
AbstractMergeJoiner {
 
 private static final Logger LOGGER = 
Logger.getLogger(IntervalIndexJoiner.class.getName());
 
-private IPartitionedDeletableTupleBufferManager bufferManager;
+private final IPartitionedDeletableTupleBufferManager bufferManager;
 
-private ActiveSweepManager[] activeManager;
-private ITuplePointerAccessor[] memoryAccessor;
-private int[] streamIndex;
-private RunFileStream[] runFileStream;
+private final ActiveSweepManager[] activeManager;
+private final ITuplePointerAccessor[] memoryAccessor;
+private final int[] streamIndex;
+private final RunFileStream[] runFileStream;
 
-private LinkedList buffer = new LinkedList<>();
+private final LinkedList buffer = new LinkedList<>();
 
-private IIntervalMergeJoinChecker imjc;
+private final IIntervalMergeJoinChecker imjc;
 
-protected byte point;
+private final byte point;
 
-private MergeStatus status;
+private final int leftKey;
+private final int rightKey;
 
-private int leftKey;
-private int rightKey;
+private long joinComparisonCount = 0;
+private long joinResultCount = 0;
+private long spillCount = 0;
+private long spillReadCount = 0;
+private long spillWriteCount = 0;
 
 public IntervalIndexJoiner(IHyracksTaskContext ctx, int memorySize, int 
partition, MergeStatus status,
 MergeJoinLocks locks, Comparator 
endPointComparator,
@@ -87,8 +89,6 @@ public class IntervalIndexJoiner extends AbstractMergeJoiner {
 this.leftKey = leftKeys[0];
 this.rightKey = rightKeys[0];
 
-this.status = status;
-
 RecordDescriptor[] recordDescriptors = new 
RecordDescriptor[JOIN_PARTITIONS];
 recordDescriptors[LEFT_PARTITION] = leftRd;
 recordDescriptors[RIGHT_PARTITION] = rightRd;
@@ -119,8 +119,6 @@ public class

[03/50] [abbrv] asterixdb git commit: snapshot super interval dag.

2016-10-17 Thread prestonc

snapshot super interval dag.


Project: http://git-wip-us.apache.org/repos/asf/asterixdb/repo
Commit: http://git-wip-us.apache.org/repos/asf/asterixdb/commit/aea7fe87
Tree: http://git-wip-us.apache.org/repos/asf/asterixdb/tree/aea7fe87
Diff: http://git-wip-us.apache.org/repos/asf/asterixdb/diff/aea7fe87

Branch: refs/heads/ecarm002/interval_join_merge
Commit: aea7fe87d3c2d03902a9e207e1afa457733e5171
Parents: 199bddd
Author: Preston Carman 
Authored: Thu Jun 30 16:03:28 2016 -0700
Committer: Preston Carman 
Committed: Thu Jun 30 16:03:28 2016 -0700

--
 .../IntervalLocalRangeSplitterOperator.java |  69 +++
 .../physical/AbstractIntervalJoinPOperator.java |  17 +
 .../physical/IntervalIndexJoinPOperator.java|   1 -
 .../physical/IntervalJoinPOperator.java |  63 +++
 .../IntervalLocalRangeOperatorDescriptor.java   | 197 
 .../IntervalLocalRangeSplitterPOperator.java| 101 +
 .../IntervalPartitionJoinPOperator.java |  20 +
 .../asterix/optimizer/base/RuleCollections.java |   2 +
 .../rules/IntervalSplitPartitioningRule.java| 445 +++
 ...IntervalPartitionJoinOperatorDescriptor.java |   3 -
 .../logical/AbstractLogicalOperator.java|  16 +-
 .../operators/logical/ReplicateOperator.java|  12 +-
 .../operators/physical/MergeJoinPOperator.java  |  16 +
 .../operators/physical/NLJoinPOperator.java | 295 
 .../physical/NestedLoopJoinPOperator.java   | 295 
 .../RangePartitionExchangePOperator.java|   8 +
 .../RangePartitionMergeExchangePOperator.java   |   8 +
 .../operators/physical/ReplicatePOperator.java  |   2 +-
 .../operators/physical/UnionAllPOperator.java   |   3 -
 .../typing/PropagatingTypeEnvironment.java  |   4 +-
 .../algebricks/rewriter/util/JoinUtils.java |   6 +-
 .../std/misc/SplitOperatorDescriptor.java   |   6 +-
 22 files changed, 1259 insertions(+), 330 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/asterixdb/blob/aea7fe87/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/IntervalLocalRangeSplitterOperator.java
--
diff --git 
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/IntervalLocalRangeSplitterOperator.java
 
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/IntervalLocalRangeSplitterOperator.java
new file mode 100644
index 000..9ae9f7d
--- /dev/null
+++ 
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/IntervalLocalRangeSplitterOperator.java
@@ -0,0 +1,69 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.asterix.algebra.operators;
+
+import java.util.Collection;
+import java.util.List;
+
+import org.apache.hyracks.algebricks.common.exceptions.AlgebricksException;
+import org.apache.hyracks.algebricks.core.algebra.base.LogicalVariable;
+import 
org.apache.hyracks.algebricks.core.algebra.operators.logical.AbstractExtensibleLogicalOperator;
+import 
org.apache.hyracks.algebricks.core.algebra.operators.logical.IOperatorExtension;
+import 
org.apache.hyracks.algebricks.core.algebra.visitors.ILogicalExpressionReferenceTransform;
+
+public class IntervalLocalRangeSplitterOperator extends 
AbstractExtensibleLogicalOperator {
+
+private final List joinKeyLogicalVars;
+
+public IntervalLocalRangeSplitterOperator(List 
joinKeyLogicalVars) {
+this.joinKeyLogicalVars = joinKeyLogicalVars;
+}
+
+@Override
+public boolean isMap() {
+return false;
+}
+
+@Override
+public IOperatorExtension newInstance() {
+return new IntervalLocalRangeSplitterOperator(joinKeyLogicalVars);
+}
+
+@Override
+public boolean 
acceptExpressionTransform(ILogicalExpressionReferenceTransform transform)
+throws AlgebricksException {
+return false;
+}
+
+@Override
+public String toString() {
+return "IntervalLocalRangeSplitterOperator";
+}
+
+

[10/50] [abbrv] asterixdb git commit: switch merge memory tracking method.

2016-10-17 Thread prestonc

switch merge memory tracking method.


Project: http://git-wip-us.apache.org/repos/asf/asterixdb/repo
Commit: http://git-wip-us.apache.org/repos/asf/asterixdb/commit/4f9e6a82
Tree: http://git-wip-us.apache.org/repos/asf/asterixdb/tree/4f9e6a82
Diff: http://git-wip-us.apache.org/repos/asf/asterixdb/diff/4f9e6a82

Branch: refs/heads/ecarm002/interval_join_merge
Commit: 4f9e6a82eaff790a50f86848e770e4b1dedf4069
Parents: 1f7ac98
Author: Preston Carman 
Authored: Thu Jul 14 09:11:57 2016 -0700
Committer: Preston Carman 
Committed: Thu Jul 14 09:11:57 2016 -0700

--
 .../joins/AbstractIntervalMergeJoinChecker.java | 14 +++
 .../IntervalPartitionJoiner.java|  2 +-
 .../dataflow/std/join/IMergeJoinChecker.java|  3 ++
 .../hyracks/dataflow/std/join/MergeJoiner.java  | 41 +++-
 4 files changed, 41 insertions(+), 19 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/asterixdb/blob/4f9e6a82/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/AbstractIntervalMergeJoinChecker.java
--
diff --git 
a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/AbstractIntervalMergeJoinChecker.java
 
b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/AbstractIntervalMergeJoinChecker.java
index 0a25c25..cf0bf6a 100644
--- 
a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/AbstractIntervalMergeJoinChecker.java
+++ 
b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/AbstractIntervalMergeJoinChecker.java
@@ -105,6 +105,20 @@ public abstract class AbstractIntervalMergeJoinChecker 
implements IIntervalMerge
 }
 
 @Override
+public boolean checkToRemoveInMemory(IFrameTupleAccessor accessorLeft, int 
leftTupleIndex,
+IFrameTupleAccessor accessorRight, int rightTupleIndex) throws 
HyracksDataException {
+try {
+IntervalJoinUtil.getIntervalPointable(accessorLeft, 
leftTupleIndex, idLeft, tvp, ipLeft);
+IntervalJoinUtil.getIntervalPointable(accessorRight, 
rightTupleIndex, idRight, tvp, ipRight);
+ipLeft.getStart(startLeft);
+ipRight.getEnd(endRight);
+return !(ch.compare(ipLeft.getTypeTag(), ipRight.getTypeTag(), 
startLeft, endRight) <= 0);
+} catch (AsterixException e) {
+throw new HyracksDataException(e);
+}
+}
+
+@Override
 public boolean checkToSaveInResult(IFrameTupleAccessor accessorLeft, int 
leftTupleIndex,
 IFrameTupleAccessor accessorRight, int rightTupleIndex, boolean 
reversed) throws HyracksDataException {
 try {

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/4f9e6a82/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalpartition/IntervalPartitionJoiner.java
--
diff --git 
a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalpartition/IntervalPartitionJoiner.java
 
b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalpartition/IntervalPartitionJoiner.java
index 5df7b0a..fe49d2f 100644
--- 
a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalpartition/IntervalPartitionJoiner.java
+++ 
b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalpartition/IntervalPartitionJoiner.java
@@ -320,7 +320,7 @@ public class IntervalPartitionJoiner {
 
 private int selectPartitionsToReload(int freeSpace, int pid) {
 for (int id = ipjd.buildNextSpilled(0); id >= 0; id = 
ipjd.buildNextSpilled(id + 1)) {
-assert buildRFWriters[id].getFileSize() > 0 : "How comes a spilled 
partition have size 0?";
+assert buildRFWriters[id].getFileSize() > 0 : "How come a spilled 
partition have size 0?";
 if (freeSpace >= buildRFWriters[id].getFileSize()) {
 return id;
 }

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/4f9e6a82/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/IMergeJoinChecker.java
--
diff --git 
a/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/IMergeJoinChecker.java
 
b/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/IMergeJoinChecker.java
index ddf04f3..49a3763 100644
---

[19/50] [abbrv] asterixdb git commit: clean up.

2016-10-17 Thread prestonc

clean up.


Project: http://git-wip-us.apache.org/repos/asf/asterixdb/repo
Commit: http://git-wip-us.apache.org/repos/asf/asterixdb/commit/5d029489
Tree: http://git-wip-us.apache.org/repos/asf/asterixdb/tree/5d029489
Diff: http://git-wip-us.apache.org/repos/asf/asterixdb/diff/5d029489

Branch: refs/heads/ecarm002/interval_join_merge
Commit: 5d02948925a3af098459ab3efba882e1d7154bfb
Parents: 933c130
Author: Preston Carman 
Authored: Wed Jul 27 12:14:50 2016 -0700
Committer: Preston Carman 
Committed: Wed Jul 27 12:14:50 2016 -0700

--
 .../sort/AbstractSorterOperatorDescriptor.java  | 10 
 .../dataflow/std/structures/TuplePointer.java   | 26 
 2 files changed, 26 insertions(+), 10 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/asterixdb/blob/5d029489/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/AbstractSorterOperatorDescriptor.java
--
diff --git 
a/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/AbstractSorterOperatorDescriptor.java
 
b/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/AbstractSorterOperatorDescriptor.java
index 665a90b..c3adc20 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/AbstractSorterOperatorDescriptor.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/sort/AbstractSorterOperatorDescriptor.java
@@ -99,16 +99,6 @@ public abstract class AbstractSorterOperatorDescriptor 
extends AbstractOperatorD
 public SortTaskState(JobId jobId, TaskId taskId) {
 super(jobId, taskId);
 }
-
-@Override
-public void toBytes(DataOutput out) throws IOException {
-
-}
-
-@Override
-public void fromBytes(DataInput in) throws IOException {
-
-}
 }
 
 protected abstract class SortActivity extends AbstractActivityNode {

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/5d029489/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/structures/TuplePointer.java
--
diff --git 
a/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/structures/TuplePointer.java
 
b/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/structures/TuplePointer.java
index ae8eb24..42351a7 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/structures/TuplePointer.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/structures/TuplePointer.java
@@ -18,11 +18,37 @@
  */
 package org.apache.hyracks.dataflow.std.structures;
 
+import java.util.Comparator;
+
 public final class TuplePointer implements IResetable {
 public static final int INVALID_ID = -1;
 private int frameIndex;
 private int tupleIndex;
 
+public static final Comparator ASC = new 
Comparator() {
+@Override
+public int compare(TuplePointer tp1, TuplePointer tp2) {
+int c = tp1.getFrameIndex() - tp2.getFrameIndex();
+if (c == 0) {
+c = tp1.getTupleIndex() - tp2.getTupleIndex();
+}
+return c;
+}
+
+};
+
+public static final Comparator DESC = new 
Comparator() {
+@Override
+public int compare(TuplePointer tp1, TuplePointer tp2) {
+int c = tp2.getFrameIndex() - tp1.getFrameIndex();
+if (c == 0) {
+c = tp2.getTupleIndex() - tp1.getTupleIndex();
+}
+return c;
+}
+
+};
+
 public TuplePointer() {
 this(INVALID_ID, INVALID_ID);
 }

[07/50] [abbrv] asterixdb git commit: new annontations.

2016-10-17 Thread prestonc

new annontations.


Project: http://git-wip-us.apache.org/repos/asf/asterixdb/repo
Commit: http://git-wip-us.apache.org/repos/asf/asterixdb/commit/ff7904de
Tree: http://git-wip-us.apache.org/repos/asf/asterixdb/tree/ff7904de
Diff: http://git-wip-us.apache.org/repos/asf/asterixdb/diff/ff7904de

Branch: refs/heads/ecarm002/interval_join_merge
Commit: ff7904def80ec75e0979e25e4f901dded8e7d7f1
Parents: 22d1dbc
Author: Preston Carman 
Authored: Fri Jul 8 16:31:05 2016 -0700
Committer: Preston Carman 
Committed: Fri Jul 8 16:31:05 2016 -0700

--
 .../IntervalJoinExpressionAnnotation.java   | 21 +++-
 .../asterix-lang-aql/src/main/javacc/AQL.jj |  2 +-
 2 files changed, 13 insertions(+), 10 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/asterixdb/blob/ff7904de/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/annotations/IntervalJoinExpressionAnnotation.java
--
diff --git 
a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/annotations/IntervalJoinExpressionAnnotation.java
 
b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/annotations/IntervalJoinExpressionAnnotation.java
index 7d4f02a..93f5990 100644
--- 
a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/annotations/IntervalJoinExpressionAnnotation.java
+++ 
b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/annotations/IntervalJoinExpressionAnnotation.java
@@ -54,15 +54,18 @@ public class IntervalJoinExpressionAnnotation extends 
AbstractExpressionAnnotati
 }
 
 private void parseHint() {
-String[] args = ((String) object).split(" ");
-setJoinType(args[0]);
-
-if (joinType.equals(PARTITION_HINT_STRING) && args.length == 7) {
-leftRecordCount = Long.valueOf(args[2]);
-rightRecordCount = Long.valueOf(args[3]);
-leftMaxDuration = Long.valueOf(args[4]);
-rightMaxDuration = Long.valueOf(args[5]);
-tuplesPerFrame = Integer.valueOf(args[6]);
+String hint = (String) object;
+setJoinType(hint);
+
+hint = hint.substring(hint.indexOf(']', 0) + 2);
+String[] args = hint.split(" ");
+
+if (joinType.equals(PARTITION_HINT_STRING) && args.length == 5) {
+leftRecordCount = Long.valueOf(args[0]);
+rightRecordCount = Long.valueOf(args[1]);
+leftMaxDuration = Long.valueOf(args[2]);
+rightMaxDuration = Long.valueOf(args[3]);
+tuplesPerFrame = Integer.valueOf(args[4]);
 }
 }
 

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/ff7904de/asterixdb/asterix-lang-aql/src/main/javacc/AQL.jj
--
diff --git a/asterixdb/asterix-lang-aql/src/main/javacc/AQL.jj 
b/asterixdb/asterix-lang-aql/src/main/javacc/AQL.jj
index bbba41d..9eaccf7 100644
--- a/asterixdb/asterix-lang-aql/src/main/javacc/AQL.jj
+++ b/asterixdb/asterix-lang-aql/src/main/javacc/AQL.jj
@@ -2313,7 +2313,7 @@ Clause OrderbyClause()throws ParseException :
   }
   if (hint.startsWith(RANGE_HINT)) {
 try{
-  
oc.setRangeMap(RangeMapBuilder.parseHint(hint.substring(RANGE_HINT.length(;
+  
oc.setRangeMap(RangeMapBuilder.parseHint(hint.substring(RANGE_HINT.length(), 
hint.indexOf(']', 0) + 1)));
 } catch (AsterixException e) {
   throw new ParseException(e.getMessage());
 }

[44/50] [abbrv] asterixdb git commit: new partition join algorithm

2016-10-17 Thread prestonc

new partition join algorithm


Project: http://git-wip-us.apache.org/repos/asf/asterixdb/repo
Commit: http://git-wip-us.apache.org/repos/asf/asterixdb/commit/0b900514
Tree: http://git-wip-us.apache.org/repos/asf/asterixdb/tree/0b900514
Diff: http://git-wip-us.apache.org/repos/asf/asterixdb/diff/0b900514

Branch: refs/heads/ecarm002/interval_join_merge
Commit: 0b90051413c96ae52cc3136d41b5589275f11385
Parents: b34a426
Author: Preston Carman 
Authored: Fri Sep 30 13:56:13 2016 -0700
Committer: Preston Carman 
Committed: Fri Sep 30 13:56:13 2016 -0700

--
 .../IntervalPartitionJoinPOperator.java |   2 +-
 .../InMemoryIntervalPartitionJoin.java  |  98 ---
 ...IntervalPartitionJoinOperatorDescriptor.java | 355 +
 .../IntervalPartitionJoinTaskState.java |  33 +
 .../IntervalPartitionJoiner.java| 772 ---
 .../IntervalPartitionUtil.java  |  70 --
 ...IntervalPartitionJoinOperatorDescriptor.java | 319 
 .../IntervalPartitionJoinTaskState.java |  33 -
 .../IntervalPartitionJoiner.java| 288 ---
 9 files changed, 412 insertions(+), 1558 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/asterixdb/blob/0b900514/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/IntervalPartitionJoinPOperator.java
--
diff --git 
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/IntervalPartitionJoinPOperator.java
 
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/IntervalPartitionJoinPOperator.java
index af77a92..73d159e 100644
--- 
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/IntervalPartitionJoinPOperator.java
+++ 
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/IntervalPartitionJoinPOperator.java
@@ -23,7 +23,7 @@ import java.util.List;
 import java.util.logging.Logger;
 
 import 
org.apache.asterix.runtime.operators.joins.IIntervalMergeJoinCheckerFactory;
-import 
org.apache.asterix.runtime.operators.joins.intervalpartition2.IntervalPartitionJoinOperatorDescriptor;
+import 
org.apache.asterix.runtime.operators.joins.intervalpartition.IntervalPartitionJoinOperatorDescriptor;
 import org.apache.hyracks.algebricks.core.algebra.base.LogicalVariable;
 import 
org.apache.hyracks.algebricks.core.algebra.operators.logical.AbstractBinaryJoinOperator.JoinKind;
 import 
org.apache.hyracks.algebricks.core.algebra.operators.logical.OrderOperator.IOrder.OrderKind;

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/0b900514/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalpartition/InMemoryIntervalPartitionJoin.java
--
diff --git 
a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalpartition/InMemoryIntervalPartitionJoin.java
 
b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalpartition/InMemoryIntervalPartitionJoin.java
deleted file mode 100644
index aeea209..000
--- 
a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalpartition/InMemoryIntervalPartitionJoin.java
+++ /dev/null
@@ -1,98 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements.  See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership.  The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License.  You may obtain a copy of the License at
- *
- *   http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied.  See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-package org.apache.asterix.runtime.operators.joins.intervalpartition;
-
-import java.util.logging.Logger;
-
-import org.apache.asterix.runtime.operators.joins.IIntervalMergeJoinChecker;
-import org.apache.hyracks.api.comm.IFrameTupleAccessor;
-import org.apache.hyracks.api.comm.IFrameWriter;
-import org.apache.hyracks.api.comm.VSizeFrame;
-import org.apache.hyracks.api.context.IHyracksTaskContext;
-import org.apache.hyracks.api.dataflow.value.RecordDescriptor;
-import org.apache.hyracks.api.exceptions.HyracksDataException;
-import

[43/50] [abbrv] asterixdb git commit: new partition join algorithm

2016-10-17 Thread prestonc

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/0b900514/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalpartition2/IntervalPartitionJoinOperatorDescriptor.java
--
diff --git 
a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalpartition2/IntervalPartitionJoinOperatorDescriptor.java
 
b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalpartition2/IntervalPartitionJoinOperatorDescriptor.java
deleted file mode 100644
index a985eee..000
--- 
a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalpartition2/IntervalPartitionJoinOperatorDescriptor.java
+++ /dev/null
@@ -1,319 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements.  See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership.  The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License.  You may obtain a copy of the License at
- *
- *   http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied.  See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-
-package org.apache.asterix.runtime.operators.joins.intervalpartition2;
-
-import java.nio.ByteBuffer;
-import java.util.logging.Logger;
-
-import org.apache.asterix.runtime.operators.joins.IIntervalMergeJoinChecker;
-import 
org.apache.asterix.runtime.operators.joins.IIntervalMergeJoinCheckerFactory;
-import 
org.apache.asterix.runtime.operators.joins.intervalpartition.IntervalPartitionComputerFactory;
-import 
org.apache.asterix.runtime.operators.joins.intervalpartition.IntervalPartitionUtil;
-import org.apache.hyracks.api.context.IHyracksTaskContext;
-import org.apache.hyracks.api.dataflow.ActivityId;
-import org.apache.hyracks.api.dataflow.IActivity;
-import org.apache.hyracks.api.dataflow.IActivityGraphBuilder;
-import org.apache.hyracks.api.dataflow.IOperatorNodePushable;
-import org.apache.hyracks.api.dataflow.TaskId;
-import org.apache.hyracks.api.dataflow.value.IRecordDescriptorProvider;
-import org.apache.hyracks.api.dataflow.value.ITuplePartitionComputer;
-import org.apache.hyracks.api.dataflow.value.RecordDescriptor;
-import org.apache.hyracks.api.exceptions.HyracksDataException;
-import org.apache.hyracks.api.job.IOperatorDescriptorRegistry;
-import org.apache.hyracks.dataflow.std.base.AbstractActivityNode;
-import org.apache.hyracks.dataflow.std.base.AbstractOperatorDescriptor;
-import 
org.apache.hyracks.dataflow.std.base.AbstractUnaryInputSinkOperatorNodePushable;
-import 
org.apache.hyracks.dataflow.std.base.AbstractUnaryInputUnaryOutputOperatorNodePushable;
-import org.apache.hyracks.dataflow.std.base.RangeId;
-import org.apache.hyracks.dataflow.std.join.MergeBranchStatus.Stage;
-import org.apache.hyracks.dataflow.std.join.MergeJoinLocks;
-import 
org.apache.hyracks.dataflow.std.misc.RangeForwardOperatorDescriptor.RangeForwardTaskState;
-
-public class IntervalPartitionJoinOperatorDescriptor extends 
AbstractOperatorDescriptor {
-private static final long serialVersionUID = 1L;
-
-private static final int LEFT_ACTIVITY_ID = 0;
-private static final int RIGHT_ACTIVITY_ID = 1;
-private final int[] leftKeys;
-private final int[] rightKeys;
-private final int memoryForJoin;
-private final IIntervalMergeJoinCheckerFactory imjcf;
-private final RangeId rangeId;
-private final int k;
-
-private final int probeKey;
-private final int buildKey;
-
-private static final Logger LOGGER = 
Logger.getLogger(IntervalPartitionJoinOperatorDescriptor.class.getName());
-
-public IntervalPartitionJoinOperatorDescriptor(IOperatorDescriptorRegistry 
spec, int memoryForJoin, int k,
-int[] leftKeys, int[] rightKeys, RecordDescriptor 
recordDescriptor, IIntervalMergeJoinCheckerFactory imjcf,
-RangeId rangeId) {
-super(spec, 2, 1);
-recordDescriptors[0] = recordDescriptor;
-this.buildKey = leftKeys[0];
-this.probeKey = rightKeys[0];
-this.k = k;
-this.leftKeys = leftKeys;
-this.rightKeys = rightKeys;
-this.memoryForJoin = memoryForJoin;
-this.imjcf = imjcf;
-this.rangeId = rangeId;
-}
-
-@Override
-public void contributeActivities(IActivityGraphBuilder builder) {
-MergeJoinLocks locks = new MergeJoinLocks();
-
-ActivityId leftAid = new ActivityId(odId, LEFT_ACTIVITY_ID);
-ActivityId rightAid = new

[47/50] [abbrv] asterixdb git commit: merge master with interval tests

2016-10-17 Thread prestonc

merge master with interval tests


Project: http://git-wip-us.apache.org/repos/asf/asterixdb/repo
Commit: http://git-wip-us.apache.org/repos/asf/asterixdb/commit/e256e637
Tree: http://git-wip-us.apache.org/repos/asf/asterixdb/tree/e256e637
Diff: http://git-wip-us.apache.org/repos/asf/asterixdb/diff/e256e637

Branch: refs/heads/ecarm002/interval_join_merge
Commit: e256e6378fa949070455dc88b25fe9542359e1b7
Parents: ed86c88 2e3d167
Author: Preston Carman 
Authored: Mon Oct 3 23:42:46 2016 -0700
Committer: Preston Carman 
Committed: Mon Oct 3 23:42:46 2016 -0700

--
 .gitignore  |4 +
 README.md   |   31 +-
 .../TranslateIntervalExpressionRule.java|   23 +-
 .../interval_overlapping.3.adm  |   35 +
 .../interval_overlapping.4.adm  |   35 +
 .../api/http/servlet/QueryServiceServlet.java   |   18 +
 .../queries/aggregate/count-dataset.sqlpp   |2 +-
 .../aggregate/query-ASTERIXDB-1626-2.sqlpp  |2 +-
 .../aggregate/query-ASTERIXDB-1626-3.sqlpp  |2 +-
 .../aggregate/query-ASTERIXDB-1626.sqlpp|2 +-
 .../queries/query-ASTERIXDB-159-3.sqlpp |2 +-
 .../inlined_q18_large_volume_customer.sqlpp |4 +-
 .../queries_sqlpp/nest_aggregate.sqlpp  |2 +-
 .../queries_sqlpp/orders-aggreg.sqlpp   |4 +-
 .../q01_pricing_summary_report_nt.sqlpp |   16 +-
 .../queries_sqlpp/q03_shipping_priority.sqlpp   |2 +-
 .../q05_local_supplier_volume.sqlpp |2 +-
 .../queries_sqlpp/query-issue562.sqlpp  |6 +-
 .../queries_sqlpp/query-issue697.sqlpp  |2 +-
 .../queries_sqlpp/query-issue785.sqlpp  |2 +-
 .../queries_sqlpp/query-issue810-2.sqlpp|2 +-
 .../queries_sqlpp/query-issue810.sqlpp  |2 +-
 .../queries_sqlpp/query-issue827-2.sqlpp|   16 +-
 .../queries_sqlpp/split-materialization.sqlpp   |4 +-
 .../interval_before/interval_before.3.query.aql |1 -
 .../interval_before/interval_before.4.query.aql |1 -
 .../interval_covers/interval_covers.3.query.aql |1 -
 .../interval_covers/interval_covers.4.query.aql |1 -
 .../interval_ends/interval_ends.3.query.aql |1 -
 .../interval_ends/interval_ends.4.query.aql |1 -
 .../interval_meets/interval_meets.3.query.aql   |1 -
 .../interval_meets/interval_meets.4.query.aql   |1 -
 .../interval_overlapping.3.query.aql|1 -
 .../interval_overlapping.4.query.aql|1 -
 .../interval_overlaps.3.query.aql   |1 -
 .../interval_overlaps.4.query.aql   |1 -
 .../interval_starts/interval_starts.3.query.aql |1 -
 .../interval_starts/interval_starts.4.query.aql |1 -
 .../agg_null/agg_null.3.query.sqlpp |2 +-
 .../agg_null_rec/agg_null_rec.3.query.sqlpp |   10 +-
 .../agg_null_rec_1/agg_null_rec_1.3.query.sqlpp |   10 +-
 .../agg_number/agg_number.3.query.sqlpp |2 +-
 .../agg_number_rec/agg_number_rec.3.query.sqlpp |   10 +-
 .../avg_double/avg_double.3.query.sqlpp |2 +-
 .../avg_double_null.3.query.sqlpp   |2 +-
 .../avg_empty_01/avg_empty_01.1.ddl.sqlpp   |2 +-
 .../avg_empty_01/avg_empty_01.2.update.sqlpp|2 +-
 .../avg_empty_01/avg_empty_01.3.query.sqlpp |4 +-
 .../avg_empty_02/avg_empty_02.1.ddl.sqlpp   |2 +-
 .../avg_empty_02/avg_empty_02.2.update.sqlpp|2 +-
 .../avg_empty_02/avg_empty_02.3.query.sqlpp |4 +-
 .../avg_float/avg_float.3.query.sqlpp   |2 +-
 .../avg_float_null/avg_float_nu.3.query.sqlpp   |2 +-
 .../avg_int16/avg_int16.3.query.sqlpp   |2 +-
 .../avg_int16_null/avg_int16_null.3.query.sqlpp |2 +-
 .../avg_int32/avg_int32.3.query.sqlpp   |2 +-
 .../avg_int32_null/avg_int32_null.3.query.sqlpp |2 +-
 .../avg_int64/avg_int64.3.query.sqlpp   |2 +-
 .../avg_int64_null/avg_int64_null.3.query.sqlpp |2 +-
 .../avg_int8/avg_int8.3.query.sqlpp |2 +-
 .../avg_int8_null/avg_int8_null.3.query.sqlpp   |2 +-
 .../avg_mixed/avg_mixed.1.ddl.sqlpp |2 +-
 .../avg_mixed/avg_mixed.3.query.sqlpp   |4 +-
 .../count_01/count_01.3.query.sqlpp |2 +-
 .../count_empty_01/count_empty_01.1.ddl.sqlpp   |2 +-
 .../count_empty_01.2.update.sqlpp   |2 +-
 .../count_empty_01/count_empty_01.3.query.sqlpp |4 +-
 .../count_empty_02/count_empty_02.1.ddl.sqlpp   |2 +-
 .../count_empty_02.2.update.sqlpp   |2 +-
 .../count_empty_02/count_empty_02.3.query.sqlpp |4 +-
 .../count_null/count_null.3.query.sqlpp |2 +-
 .../issue395/issue395.3.query.sqlpp |2 +-
 .../issue412_0/issue412_0.3.query.sqlpp |2 +-

[12/50] [abbrv] asterixdb git commit: updated from api change.

2016-10-17 Thread prestonc

updated from api change.


Project: http://git-wip-us.apache.org/repos/asf/asterixdb/repo
Commit: http://git-wip-us.apache.org/repos/asf/asterixdb/commit/1f3fb827
Tree: http://git-wip-us.apache.org/repos/asf/asterixdb/tree/1f3fb827
Diff: http://git-wip-us.apache.org/repos/asf/asterixdb/diff/1f3fb827

Branch: refs/heads/ecarm002/interval_join_merge
Commit: 1f3fb8274284c8f7924c689104f593cfd3300171
Parents: 0725160
Author: Preston Carman 
Authored: Thu Jul 14 13:55:19 2016 -0700
Committer: Preston Carman 
Committed: Thu Jul 14 13:55:19 2016 -0700

--
 .../hyracks/dataflow/std/join/NaturalMergeJoinChecker.java   | 8 +++-
 1 file changed, 7 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/asterixdb/blob/1f3fb827/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/NaturalMergeJoinChecker.java
--
diff --git 
a/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/NaturalMergeJoinChecker.java
 
b/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/NaturalMergeJoinChecker.java
index 030e25b..9a50d0c 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/NaturalMergeJoinChecker.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-dataflow-std/src/main/java/org/apache/hyracks/dataflow/std/join/NaturalMergeJoinChecker.java
@@ -41,7 +41,13 @@ public class NaturalMergeJoinChecker implements 
IMergeJoinChecker {
 @Override
 public boolean checkToRemoveInMemory(ITupleAccessor accessorLeft, 
ITupleAccessor accessorRight)
 throws HyracksDataException {
-int c = comparator.compare(accessorLeft, accessorLeft.getTupleId(), 
accessorRight, accessorRight.getTupleId());
+return checkToRemoveInMemory(accessorLeft, accessorLeft.getTupleId(), 
accessorRight, accessorRight.getTupleId());
+}
+
+@Override
+public boolean checkToRemoveInMemory(IFrameTupleAccessor accessorLeft, int 
leftTupleIndex,
+IFrameTupleAccessor accessorRight, int rightTupleIndex) throws 
HyracksDataException {
+int c = comparator.compare(accessorLeft, leftTupleIndex, 
accessorRight, rightTupleIndex);
 return c < 0;
 }

[25/50] [abbrv] asterixdb git commit: Working version of the range connector and interval join partition.

2016-10-17 Thread prestonc

Working version of the range connector and interval join partition.


Project: http://git-wip-us.apache.org/repos/asf/asterixdb/repo
Commit: http://git-wip-us.apache.org/repos/asf/asterixdb/commit/1487f2be
Tree: http://git-wip-us.apache.org/repos/asf/asterixdb/tree/1487f2be
Diff: http://git-wip-us.apache.org/repos/asf/asterixdb/diff/1487f2be

Branch: refs/heads/ecarm002/interval_join_merge
Commit: 1487f2be3a364e32e3fee7a10246b6f548f75476
Parents: 13af53a
Author: Preston Carman 
Authored: Wed Aug 24 14:35:57 2016 -0700
Committer: Preston Carman 
Committed: Wed Aug 24 14:35:57 2016 -0700

--
 .../physical/AbstractIntervalJoinPOperator.java |  56 +---
 .../IntervalLocalRangeOperatorDescriptor.java   |   2 +-
 .../IntervalPartitionJoinPOperator.java |  86 +++-
 .../asterix/optimizer/base/RuleCollections.java |   2 +-
 .../rules/IntervalSplitPartitioningRule.java|  61 
 .../asterix/optimizer/rules/util/JoinUtils.java | 128 ++---
 .../translator/util/FunctionCollection.java |   6 +
 .../config/AsterixPropertiesAccessor.java   |   6 +-
 .../om/functions/AsterixBuiltinFunctions.java   |   6 +
 .../evaluators/common/FunctionManagerImpl.java  |   6 +-
 .../CalendarDurationFromDateTimeDescriptor.java |   6 +-
 .../IntervalPartitionJoinEndDescriptor.java |  58 
 .../temporal/IntervalPartitionJoinFunction.java | 138 +++
 .../IntervalPartitionJoinStartDescriptor.java   |  58 
 ...rlappingIntervalMergeJoinCheckerFactory.java |   3 +-
 .../IntervalPartitionComputerFactory.java   |  24 +---
 ...IntervalPartitionJoinOperatorDescriptor.java |  28 +---
 .../IntervalPartitionUtil.java  |  22 +++
 .../algebra/functions/FunctionIdentifier.java   |   3 +-
 .../logical/visitors/SchemaVariableVisitor.java |   1 +
 .../rules/EnforceStructuralPropertiesRule.java  |   3 +-
 .../rewriter/rules/IntroduceProjectsRule.java   |  18 +--
 .../SetAlgebricksPhysicalOperatorsRule.java |   7 +
 .../hyracks/dataflow/std/base/RangeId.java  |   3 +-
 .../connectors/PartitionRangeDataWriter.java|   2 +-
 .../misc/RangeForwardOperatorDescriptor.java|  10 ++
 26 files changed, 573 insertions(+), 170 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/asterixdb/blob/1487f2be/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/AbstractIntervalJoinPOperator.java
--
diff --git 
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/AbstractIntervalJoinPOperator.java
 
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/AbstractIntervalJoinPOperator.java
index 3be9e80..cc2a022 100644
--- 
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/AbstractIntervalJoinPOperator.java
+++ 
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/AbstractIntervalJoinPOperator.java
@@ -55,7 +55,7 @@ public abstract class AbstractIntervalJoinPOperator extends 
AbstractJoinPOperato
 
 private final List keysLeftBranch;
 private final List keysRightBranch;
-private final IIntervalMergeJoinCheckerFactory mjcf;
+protected final IIntervalMergeJoinCheckerFactory mjcf;
 private final RangeId leftRangeId;
 private final RangeId rightRangeId;
 private final IRangeMap rangeMapHint;
@@ -115,14 +115,11 @@ public abstract class AbstractIntervalJoinPOperator 
extends AbstractJoinPOperato
 
 @Override
 public void computeDeliveredProperties(ILogicalOperator iop, 
IOptimizationContext context) {
-ArrayList order = new ArrayList<>();
-for (LogicalVariable v : keysLeftBranch) {
-order.add(new OrderColumn(v, mjcf.isOrderAsc() ? OrderKind.ASC : 
OrderKind.DESC));
-}
+ArrayList order = getLeftRangeOrderColumn();
 IPartitioningProperty pp = new OrderedPartitionedProperty(order, null, 
leftRangeId,
 RangePartitioningType.PROJECT, rangeMapHint);
 List propsLocal = new ArrayList<>();
-propsLocal.add(new LocalOrderProperty(order));
+propsLocal.add(new LocalOrderProperty(getLeftLocalSortOrderColumn()));
 deliveredProperties = new StructuralPropertiesVector(pp, propsLocal);
 }
 
@@ -134,26 +131,17 @@ public abstract class AbstractIntervalJoinPOperator 
extends AbstractJoinPOperato
 
 IPartitioningProperty ppLeft = null;
 List ispLeft = new ArrayList<>();
+ispLeft.add(new LocalOrderProperty(getLeftLocalSortOrderColumn()));
+
 IPartitioningProperty ppRight = null;
 List ispRight = new ArrayList<>();
-
-ArrayList orderLeft = new ArrayList<>();
-for (LogicalVariable v :

[09/50] [abbrv] asterixdb git commit: merge with master

2016-10-17 Thread prestonc

merge with master


Project: http://git-wip-us.apache.org/repos/asf/asterixdb/repo
Commit: http://git-wip-us.apache.org/repos/asf/asterixdb/commit/1f7ac985
Tree: http://git-wip-us.apache.org/repos/asf/asterixdb/tree/1f7ac985
Diff: http://git-wip-us.apache.org/repos/asf/asterixdb/diff/1f7ac985

Branch: refs/heads/ecarm002/interval_join_merge
Commit: 1f7ac9850459eb2f4cfe003661ceaa5d5fabb1d7
Parents: ff7904d 81b1f21
Author: Preston Carman 
Authored: Mon Jul 11 15:22:49 2016 -0700
Committer: Preston Carman 
Committed: Mon Jul 11 15:22:49 2016 -0700

--
 README.md   | 6 +-
 asterixdb/asterix-algebra/pom.xml   |29 +-
 .../physical/ExternalDataLookupPOperator.java   | 4 +-
 .../asterix/optimizer/base/RuleCollections.java | 4 +-
 ...IntroduceSecondaryIndexInsertDeleteRule.java |84 +-
 .../optimizer/rules/ResolveVariableRule.java|   286 +
 .../rules/SetClosedRecordConstructorsRule.java  |35 +-
 .../optimizer/rules/UnnestToDataScanRule.java   | 7 +-
 .../am/AbstractIntroduceAccessMethodRule.java   |   186 +-
 .../LangExpressionToPlanTranslator.java |   415 +-
 .../SqlppExpressionToPlanTranslator.java|   163 +-
 .../asterix-algebra/src/main/javacc/AQLPlus.jj  | 4 +-
 asterixdb/asterix-app/pom.xml   |16 +-
 asterixdb/asterix-app/scripts/idefix/startcc.sh | 4 +-
 .../api/common/AsterixAppRuntimeContext.java|17 +-
 .../asterix/api/common/SessionConfig.java   | 7 +-
 .../asterix/api/http/servlet/APIServlet.java|38 +-
 .../asterix/api/http/servlet/JSONUtil.java  |   136 -
 .../api/http/servlet/QueryResultAPIServlet.java | 2 +-
 .../api/http/servlet/QueryServiceServlet.java   |   359 +-
 .../external/ExternalIndexingOperations.java|58 +-
 .../external/ExternalLibraryClassLoader.java|56 +
 .../app/external/ExternalLibraryUtils.java  |35 +-
 .../app/external/FeedWorkCollection.java| 2 +-
 .../asterix/aql/translator/QueryTranslator.java |78 +-
 .../apache/asterix/file/DatasetOperations.java  | 4 +-
 .../apache/asterix/file/IndexOperations.java|42 +-
 .../file/SecondaryBTreeOperationsHelper.java|33 +-
 .../file/SecondaryIndexOperationsHelper.java|49 +-
 .../SecondaryInvertedIndexOperationsHelper.java |30 +-
 .../file/SecondaryRTreeOperationsHelper.java| 4 +-
 .../bootstrap/CCApplicationEntryPoint.java  |10 +-
 .../bootstrap/NCApplicationEntryPoint.java  | 8 +-
 .../org/apache/asterix/result/ResultUtils.java  |80 +-
 asterixdb/asterix-app/src/main/scripts/run.sh   | 2 +-
 .../http/servlet/QueryServiceServletTest.java   |45 +
 .../asterix/app/external/TestLibrarian.java |35 +-
 .../asterix/test/runtime/ExecutionTest.java |12 +-
 .../asterix/test/runtime/ExecutionTestUtil.java |20 +-
 .../asterix/test/runtime/RepeatedTest.java  | 4 -
 .../asterix/test/sqlpp/ParserTestExecutor.java  | 3 +
 .../asterix-app/src/test/resources/dapd/q2.aql  |12 +-
 .../src/test/resources/logging.properties   | 4 +-
 .../dataset_with_meta-1.1.adm   | 2 +-
 .../dataset_with_meta-2.1.adm   | 2 +-
 .../dataset_with_meta-2.3.adm   | 2 +-
 .../dataset_with_meta-4.3.adm   | 2 +-
 .../dataset_with_meta-5.3.adm   | 2 +-
 .../dataset_with_meta-6.3.adm   | 4 +-
 .../dataset_with_meta-7.3.adm   | 4 +-
 .../issue_251_dataset_hint_2.1.adm  | 2 +-
 .../issue_251_dataset_hint_3.1.adm  | 2 +-
 .../issue_251_dataset_hint_4.1.adm  | 2 +-
 .../metadata/results/basic/meta01/meta01.1.adm  | 4 +-
 .../metadata/results/basic/meta02/meta02.1.adm  | 2 +-
 .../metadata/results/basic/meta05/meta05.1.adm  | 4 +-
 .../metadata/results/basic/meta09/meta09.1.adm  | 2 +-
 .../metadata/results/basic/meta12/meta12.1.adm  | 2 +-
 .../metadata/results/basic/meta16/meta16.1.adm  | 4 +-
 .../metadata/results/basic/meta17/meta17.1.adm  | 4 +-
 .../metadata_dataset/metadata_dataset.1.adm |26 +-
 .../metadata_dataverse/metadata_dataverse.1.adm | 6 +-
 .../basic/metadata_index/metadata_index.1.adm   |26 +-
 ...verify_failure_subsequent_no_execution.1.adm | 2 +-
 .../nontagged/custord/local/cust-load.aql   |10 +-
 ...rjoin-probe-pidx-with-join-btree-sidx_01.aql |30 +-
 ...rjoin-probe-pidx-with-join-btree-sidx_02.aql |30 +-
 .../queries/btree-index/btree-datetime-01.aql   | 6 +-
 .../queries/btree-index/btree-datetime-02.aql   | 8 +-
 .../queries/btree-index/btree-datetime-03.aql   | 8 +-
 .../queries/btree-index/btree-datetime-04.aql   | 6 +-
 .../optimizerts/queries/distinct_aggregate.aql  |52 +-

[49/50] [abbrv] asterixdb git commit: merge master

2016-10-17 Thread prestonc

merge master


Project: http://git-wip-us.apache.org/repos/asf/asterixdb/repo
Commit: http://git-wip-us.apache.org/repos/asf/asterixdb/commit/9baab026
Tree: http://git-wip-us.apache.org/repos/asf/asterixdb/tree/9baab026
Diff: http://git-wip-us.apache.org/repos/asf/asterixdb/diff/9baab026

Branch: refs/heads/ecarm002/interval_join_merge
Commit: 9baab026f330a790be49f1dffeb77ae1dd3fc0ef
Parents: b4a3fd5 4819ea4
Author: Preston Carman 
Authored: Mon Oct 17 12:18:06 2016 -0700
Committer: Preston Carman 
Committed: Mon Oct 17 12:18:06 2016 -0700

--
 .../active/message/ActiveManagerMessage.java|   2 +-
 .../active/message/ActivePartitionMessage.java  |   2 +-
 asterixdb/asterix-algebra/pom.xml   |   4 +
 .../optimizer/rules/LoadRecordFieldsRule.java   |  46 +-
 .../rules/PushLimitIntoOrderByRule.java |  16 +-
 .../SweepIllegalNonfunctionalFunctions.java |  11 +-
 .../subplan/InlineAllNtsInSubplanVisitor.java   |  11 +-
 ...neLeftNtsInSubplanJoinFlatteningVisitor.java |   7 +-
 .../SubplanSpecialFlatteningCheckVisitor.java   |  10 +-
 .../translator/AbstractLangTranslator.java  |   9 +-
 .../translator/util/FunctionCollection.java |  12 +
 .../asterix-app/data/tpcds/call_center.csv  |   5 +
 .../asterix-app/data/tpcds/catalog_page.csv |   5 +
 .../asterix-app/data/tpcds/catalog_returns.csv  |  32 +-
 .../asterix-app/data/tpcds/catalog_sales.csv|  38 +-
 asterixdb/asterix-app/data/tpcds/customer.csv   |  20 +-
 .../asterix-app/data/tpcds/customer_address.csv |  20 +-
 .../data/tpcds/customer_demographics.csv|   6 +
 asterixdb/asterix-app/data/tpcds/date_dim.csv   |  27 +-
 .../data/tpcds/household_demographics.csv   |   7 +
 .../asterix-app/data/tpcds/income_band.csv  |   5 +
 asterixdb/asterix-app/data/tpcds/inventory.csv  |  15 +
 asterixdb/asterix-app/data/tpcds/item.csv   |  19 +-
 asterixdb/asterix-app/data/tpcds/promotion.csv  |  10 +
 asterixdb/asterix-app/data/tpcds/reason.csv |  10 +
 asterixdb/asterix-app/data/tpcds/ship_mode.csv  |   5 +
 asterixdb/asterix-app/data/tpcds/store.csv  |   5 +
 .../asterix-app/data/tpcds/store_returns.csv|  17 +
 .../asterix-app/data/tpcds/store_sales.csv  |  48 +-
 asterixdb/asterix-app/data/tpcds/time_dim.csv   |   8 +
 asterixdb/asterix-app/data/tpcds/warehouse.csv  |   5 +
 asterixdb/asterix-app/data/tpcds/web_page.csv   |  10 +
 .../asterix-app/data/tpcds/web_returns.csv  |  13 +
 asterixdb/asterix-app/data/tpcds/web_sales.csv  |  16 +
 asterixdb/asterix-app/data/tpcds/web_site.csv   |   5 +
 asterixdb/asterix-app/pom.xml   |  14 +
 .../interval_overlapping.3.adm  |  35 -
 .../interval_overlapping.4.adm  |  35 -
 .../common/AsterixHyracksIntegrationUtil.java   |   2 +-
 .../asterix/api/http/servlet/APIServlet.java|  12 +-
 .../api/http/servlet/QueryServiceServlet.java   |  19 +-
 .../app/nc/AsterixNCAppRuntimeContext.java  |  24 +-
 .../asterix/app/translator/QueryTranslator.java |   6 +-
 .../bootstrap/CCApplicationEntryPoint.java  |  17 +-
 .../bootstrap/ClusterLifecycleListener.java |  14 +-
 .../bootstrap/NCApplicationEntryPoint.java  |  17 +-
 .../resources/asterix-build-configuration.xml   |   5 +-
 ...ataset_with_default_dataverse_type.1.ddl.aql |  25 +
 ...aset_with_default_dataverse_type.2.query.aql |  23 +
 ...ataset_with_default_dataverse_type.3.ddl.aql |  20 +
 .../dataset_with_default_dataverse_type.2.adm   |   1 +
 .../src/test/resources/metadata/testsuite.xml   |   5 +
 .../queries/multi-indexes/skip-one-index.aql|  49 +
 .../queries/multi-indexes/skip-two-index.aql|  53 ++
 .../queries/query-ASTERIXDB-1671.sqlpp  |  40 +
 .../results/multi-indexes/skip-one-index.plan   |  26 +
 .../results/multi-indexes/skip-two-index.plan   |  15 +
 .../results/query-ASTERIXDB-1671.plan   |  17 +
 .../polygon-from-open-list_issue1627.1.ddl.aql  |  34 +
 ...olygon-from-open-list_issue1627.2.update.aql |  30 +
 ...polygon-from-open-list_issue1627.3.query.aql |  28 +
 .../feed-with-external-function.4.sleep.aql |  19 -
 .../feed-with-external-function.5.pollquery.aql |  33 +
 .../feed-with-external-function.5.query.aql |  32 -
 .../query-ASTERIXDB-1671-2.1.ddl.sqlpp  |  47 +
 .../query-ASTERIXDB-1671-2.2.update.sqlpp   |  24 +
 .../query-ASTERIXDB-1671-2.3.query.sqlpp|  27 +
 .../query-ASTERIXDB-1671.1.ddl.sqlpp|  47 +
 .../query-ASTERIXDB-1671.2.update.sqlpp |  24 +
 .../query-ASTERIXDB-1671.3.query.sqlpp  |  27 +
 .../query-ASTERIXDB-1689.1.query.sqlpp  |  22 +
 .../everysat_04/everysat_04.3.query.sqlpp   |   2 +-
 .../query-ASTERIXDB-1674.1.ddl.sqlpp|  38 +
 .../query-ASTERIXDB-1674.2.update.sqlpp |  24 +
 .../query-ASTERIXDB-1674.3.query.sqlpp  |  39 +

[18/50] [abbrv] asterixdb git commit: Range forward operators.

2016-10-17 Thread prestonc

Range forward operators.


Project: http://git-wip-us.apache.org/repos/asf/asterixdb/repo
Commit: http://git-wip-us.apache.org/repos/asf/asterixdb/commit/933c1307
Tree: http://git-wip-us.apache.org/repos/asf/asterixdb/tree/933c1307
Diff: http://git-wip-us.apache.org/repos/asf/asterixdb/diff/933c1307

Branch: refs/heads/ecarm002/interval_join_merge
Commit: 933c13078397f07e7528908187f06d7c5c018acd
Parents: 5b102cc
Author: Preston Carman 
Authored: Wed Jul 27 12:14:21 2016 -0700
Committer: Preston Carman 
Committed: Wed Jul 27 12:14:21 2016 -0700

--
 .../core/algebra/base/LogicalOperatorTag.java   |   1 +
 .../core/algebra/base/PhysicalOperatorTag.java  |   1 +
 .../operators/logical/RangeForwardOperator.java |  80 +
 .../visitors/CardinalityInferenceVisitor.java   |   6 +
 .../visitors/FDsAndEquivClassesVisitor.java |   7 ++
 .../visitors/IsomorphismOperatorVisitor.java|  10 ++
 .../IsomorphismVariableMappingVisitor.java  |   7 ++
 ...OperatorDeepCopyWithNewVariablesVisitor.java |  10 ++
 .../visitors/LogicalPropertiesVisitor.java  |   7 ++
 .../visitors/OperatorDeepCopyVisitor.java   |  25 ++--
 .../visitors/PrimaryKeyVariablesVisitor.java|   7 ++
 .../visitors/ProducedVariableVisitor.java   |   6 +
 .../logical/visitors/SchemaVariableVisitor.java |   6 +
 .../visitors/SubstituteVariableVisitor.java |   7 ++
 .../logical/visitors/UsedVariableVisitor.java   |   6 +
 .../physical/RangeForwardPOperator.java |  85 +
 .../LogicalOperatorPrettyPrintVisitor.java  |   9 ++
 .../visitors/ILogicalOperatorVisitor.java   |   3 +
 .../misc/RangeForwardOperatorDescriptor.java| 119 +++
 19 files changed, 393 insertions(+), 9 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/asterixdb/blob/933c1307/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/base/LogicalOperatorTag.java
--
diff --git 
a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/base/LogicalOperatorTag.java
 
b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/base/LogicalOperatorTag.java
index d39a8c8..07c35a3 100644
--- 
a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/base/LogicalOperatorTag.java
+++ 
b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/base/LogicalOperatorTag.java
@@ -40,6 +40,7 @@ public enum LogicalOperatorTag {
 ORDER,
 PARTITIONINGSPLIT,
 PROJECT,
+RANGE_FORWARD,
 REPLICATE,
 RUNNINGAGGREGATE,
 SCRIPT,

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/933c1307/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/base/PhysicalOperatorTag.java
--
diff --git 
a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/base/PhysicalOperatorTag.java
 
b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/base/PhysicalOperatorTag.java
index 7f753a5..8ae6602 100644
--- 
a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/base/PhysicalOperatorTag.java
+++ 
b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/base/PhysicalOperatorTag.java
@@ -52,6 +52,7 @@ public enum PhysicalOperatorTag {
 PRE_SORTED_DISTINCT_BY,
 RANDOM_PARTITION_EXCHANGE,
 RANDOM_MERGE_EXCHANGE,
+RANGE_FORWARD,
 RANGE_PARTITION_EXCHANGE,
 RANGE_PARTITION_MERGE_EXCHANGE,
 RTREE_SEARCH,

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/933c1307/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/RangeForwardOperator.java
--
diff --git 
a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/RangeForwardOperator.java
 
b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/RangeForwardOperator.java
new file mode 100644
index 000..5a3bc98
--- /dev/null
+++ 
b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/RangeForwardOperator.java
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the

[34/50] [abbrv] asterixdb git commit: merge master

2016-10-17 Thread prestonc

merge master


Project: http://git-wip-us.apache.org/repos/asf/asterixdb/repo
Commit: http://git-wip-us.apache.org/repos/asf/asterixdb/commit/f186da9e
Tree: http://git-wip-us.apache.org/repos/asf/asterixdb/tree/f186da9e
Diff: http://git-wip-us.apache.org/repos/asf/asterixdb/diff/f186da9e

Branch: refs/heads/ecarm002/interval_join_merge
Commit: f186da9ede14f4beeb3e24c2b14f25407a456bd4
Parents: dc6842d 2a73f41
Author: Preston Carman 
Authored: Fri Sep 9 10:04:44 2016 -0700
Committer: Preston Carman 
Committed: Fri Sep 9 10:04:44 2016 -0700

--
 .../api/http/servlet/ClusterAPIServlet.java |  50 +-
 .../servlet/ClusterNodeDetailsAPIServlet.java   | 163 
 .../app/nc/AsterixNCAppRuntimeContext.java  |   3 +-
 .../bootstrap/CCApplicationEntryPoint.java  |  20 +-
 .../cluster_state_1.1.cstate.aql|  24 +
 .../cluster_state_2.1.cstate.aql|  25 +
 .../cluster_state_3.1.cstate.aql|  25 +
 .../cluster_state_4.1.cstate.aql|  25 +
 .../datetime_functions.3.query.aql  |   4 +
 .../datetime_functions.3.query.sqlpp|   2 +
 .../api/cluster_state_1/cluster_state_1.1.adm   |  74 ++
 .../api/cluster_state_3/cluster_state_3.1.adm   |  20 +
 .../api/cluster_state_4/cluster_state_4.1.adm   |   9 +
 .../datetime_functions/datetime_functions.1.adm |   2 +-
 .../datetime_functions/datetime_functions.3.ast |  26 +
 .../src/test/resources/runtimets/testsuite.xml  |  23 +
 .../config/AbstractAsterixProperties.java   |  39 +
 .../config/AsterixCompilerProperties.java   |  17 +-
 .../config/AsterixExternalProperties.java   |  28 +-
 .../common/config/AsterixFeedProperties.java|  22 +-
 .../config/AsterixReplicationProperties.java|  29 +-
 .../common/config/AsterixStorageProperties.java |  34 +-
 .../config/AsterixTransactionProperties.java|  35 +-
 .../asterix/common/utils/ServletUtil.java   |   3 +-
 .../apache/asterix/test/aql/TestExecutor.java   |  69 +-
 asterixdb/asterix-doc/pom.xml   |  22 +
 .../src/main/markdown/sqlpp/0_toc.md|  69 ++
 .../src/main/markdown/sqlpp/1_intro.md  |   6 +
 .../src/main/markdown/sqlpp/2_expr.md   | 294 +++
 .../src/main/markdown/sqlpp/3_query.md  | 882 +++
 .../src/main/markdown/sqlpp/4_ddl.md| 452 ++
 .../installer/test/AbstractExecutionIT.java |   8 +
 .../runtime/util/AsterixAppContextInfo.java |   3 +-
 .../runtime/util/AsterixClusterProperties.java  |  15 +-
 asterixdb/asterix-server/pom.xml|   2 +-
 .../server/test/NCServiceExecutionIT.java   |   6 +-
 .../client/HyracksClientInterfaceFunctions.java |  33 +-
 .../HyracksClientInterfaceRemoteProxy.java  |   7 +
 .../hyracks/api/client/HyracksConnection.java   |   6 +
 .../api/client/IHyracksClientConnection.java|   9 +
 .../api/client/IHyracksClientInterface.java |   1 +
 .../control/cc/ClusterControllerService.java|  23 +-
 .../hyracks/control/cc/NodeControllerState.java |  98 ++-
 .../control/cc/web/NodesRESTAPIFunction.java|   2 +-
 .../control/cc/work/GetNodeDetailsJSONWork.java |  25 +-
 .../common/controllers/NodeRegistration.java|  13 +-
 .../hyracks/control/common/utils/PidHelper.java |  52 ++
 .../control/nc/NodeControllerService.java   |   4 +-
 .../std/group/HashSpillableTableFactory.java|  31 +-
 49 files changed, 2665 insertions(+), 169 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/asterixdb/blob/f186da9e/asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
--

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/f186da9e/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixCompilerProperties.java
--
diff --cc 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixCompilerProperties.java
index b79c2cb,af5124b..232a541
--- 
a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixCompilerProperties.java
+++ 
b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixCompilerProperties.java
@@@ -18,27 -18,24 +18,32 @@@
   */
  package org.apache.asterix.common.config;
  
+ import org.apache.hyracks.util.StorageUtil;
+ 
+ import static org.apache.hyracks.util.StorageUtil.StorageUnit.KILOBYTE;
+ import static org.apache.hyracks.util.StorageUtil.StorageUnit.MEGABYTE;
+ 
  public class AsterixCompilerProperties extends AbstractAsterixProperties {
 +private static final int MB = 1048576;
 +
  private static final String COMPILER_SORTMEMORY_KEY = 
"compiler.sortmemory";
- private static final long COMPILER_SORTMEMORY_DEFAULT =

[04/50] [abbrv] asterixdb git commit: working mega overlapping plan.

2016-10-17 Thread prestonc

working mega overlapping plan.


Project: http://git-wip-us.apache.org/repos/asf/asterixdb/repo
Commit: http://git-wip-us.apache.org/repos/asf/asterixdb/commit/0f533e8b
Tree: http://git-wip-us.apache.org/repos/asf/asterixdb/tree/0f533e8b
Diff: http://git-wip-us.apache.org/repos/asf/asterixdb/diff/0f533e8b

Branch: refs/heads/ecarm002/interval_join_merge
Commit: 0f533e8bbbac7cf633d87013d71370114970c959
Parents: aea7fe8
Author: Preston Carman 
Authored: Tue Jul 5 17:56:26 2016 -0700
Committer: Preston Carman 
Committed: Tue Jul 5 17:56:26 2016 -0700

--
 .../IntervalLocalRangeSplitterOperator.java |   2 +-
 .../IntervalLocalRangeOperatorDescriptor.java   |  13 +-
 .../IntervalLocalRangeSplitterPOperator.java|  38 ++--
 .../asterix/optimizer/base/RuleCollections.java |   1 +
 .../rules/IntervalSplitPartitioningRule.java| 201 ---
 .../physical/AbstractExchangePOperator.java |   1 +
 .../physical/OneToOneExchangePOperator.java |   5 +-
 .../algebricks/core/jobgen/impl/JobBuilder.java |  28 +--
 8 files changed, 173 insertions(+), 116 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/asterixdb/blob/0f533e8b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/IntervalLocalRangeSplitterOperator.java
--
diff --git 
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/IntervalLocalRangeSplitterOperator.java
 
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/IntervalLocalRangeSplitterOperator.java
index 9ae9f7d..181e0fa 100644
--- 
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/IntervalLocalRangeSplitterOperator.java
+++ 
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/IntervalLocalRangeSplitterOperator.java
@@ -54,7 +54,7 @@ public class IntervalLocalRangeSplitterOperator extends 
AbstractExtensibleLogica
 
 @Override
 public String toString() {
-return "IntervalLocalRangeSplitterOperator";
+return "IntervalLocalRangeSplitterOperator " + joinKeyLogicalVars;
 }
 
 @Override

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/0f533e8b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/IntervalLocalRangeOperatorDescriptor.java
--
diff --git 
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/IntervalLocalRangeOperatorDescriptor.java
 
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/IntervalLocalRangeOperatorDescriptor.java
index 392bf43..ca44f78 100644
--- 
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/IntervalLocalRangeOperatorDescriptor.java
+++ 
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/IntervalLocalRangeOperatorDescriptor.java
@@ -45,20 +45,21 @@ public class IntervalLocalRangeOperatorDescriptor extends 
AbstractOperatorDescri
 private static final long serialVersionUID = 1L;
 private static final int PARTITION_ACTIVITY_ID = 0;
 
+private static final int OUTPUT_ARITY = 3;
+
 private static final int INPUT_STARTS = 0;
 private static final int INPUT_COVERS = 2;
 private static final int INPUT_ENDS = 1;
 
-//private static final int INPUT_STARTS = 0;
-//private static final int INPUT_COVERS = 0;
-//private static final int INPUT_ENDS = 0;
-
 private final int key;
 private final IRangeMap rangeMap;
 
 public IntervalLocalRangeOperatorDescriptor(IOperatorDescriptorRegistry 
spec, int[] keys,
 RecordDescriptor recordDescriptor, IRangeMap rangeMap) {
-super(spec, 1, 3);
+super(spec, 1, OUTPUT_ARITY);
+for (int i = 0; i < outputArity; i++) {
+recordDescriptors[i] = recordDescriptor;
+}
 key = keys[0];
 this.rangeMap = rangeMap;
 }
@@ -105,7 +106,7 @@ public class IntervalLocalRangeOperatorDescriptor extends 
AbstractOperatorDescri
 @Override
 public void flush() throws HyracksDataException {
 for (int i = 0; i < getOutputArity(); i++) {
-resultAppender[i].flush(writers[i]);
+FrameUtils.flushFrame(resultAppender[i].getBuffer(), 
writers[i]);
 }
 }
 

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/0f533e8b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/IntervalLocalRangeSplitterPOperator.java
--
diff --git

[08/50] [abbrv] asterixdb git commit: merge with master

2016-10-17 Thread prestonc

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/1f7ac985/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java
--

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/1f7ac985/asterixdb/asterix-app/src/test/resources/logging.properties
--
diff --cc asterixdb/asterix-app/src/test/resources/logging.properties
index d07819d,5b5e3b6..baf225c
--- a/asterixdb/asterix-app/src/test/resources/logging.properties
+++ b/asterixdb/asterix-app/src/test/resources/logging.properties
@@@ -16,18 -16,18 +16,18 @@@
  # specific language governing permissions and limitations
  # under the License.
  
- # Default Logging Configuration File
+ #  Default Logging Configuration File
  #
  # You can use a different file by specifying a filename
 -# with the java.util.logging.config.file system property.  
 +# with the java.util.logging.config.file system property.
  # For example java -Djava.util.logging.config.file=myfile
  
  
  
- # Global properties
+ #  Global properties
  
  
 -# "handlers" specifies a comma separated list of log Handler 
 +# "handlers" specifies a comma separated list of log Handler
  # classes.  These handlers will be installed during VM startup.
  # Note that these classes must be on the system classpath.
  # By default we only configure a ConsoleHandler, which will only

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/1f7ac985/asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
--

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/1f7ac985/asterixdb/asterix-lang-aql/src/main/javacc/AQL.jj
--

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/1f7ac985/asterixdb/asterix-om/pom.xml
--

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/1f7ac985/asterixdb/pom.xml
--

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/1f7ac985/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/properties/IPartitioningProperty.java
--

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/1f7ac985/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/properties/OrderedPartitionedProperty.java
--
diff --cc 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/properties/OrderedPartitionedProperty.java
index c90aff9,23c8273..c1dcd97
--- 
a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/properties/OrderedPartitionedProperty.java
+++ 
b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/properties/OrderedPartitionedProperty.java
@@@ -77,9 -59,11 +77,11 @@@ public class OrderedPartitionedPropert
  }
  
  @Override
- public void normalize(Map 
equivalenceClasses, List fds) {
- orderColumns = 
PropertiesUtil.replaceOrderColumnsByEqClasses(orderColumns, equivalenceClasses);
- orderColumns = PropertiesUtil.applyFDsToOrderColumns(orderColumns, 
fds);
+ public IPartitioningProperty normalize(Map equivalenceClasses,
+ List fds) {
+ List columns = 
PropertiesUtil.replaceOrderColumnsByEqClasses(orderColumns, equivalenceClasses);
+ columns = PropertiesUtil.applyFDsToOrderColumns(columns, fds);
 -return new OrderedPartitionedProperty(columns, domain);
++return new OrderedPartitionedProperty(columns, domain, rangeMap);
  }
  
  @Override

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/1f7ac985/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/rewriter/base/PhysicalOptimizationConfig.java
--

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/1f7ac985/hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/util/JoinUtils.java
--
diff --cc

[17/50] [abbrv] asterixdb git commit: More clean up form interval prep.

2016-10-17 Thread prestonc

More clean up form interval prep.


Project: http://git-wip-us.apache.org/repos/asf/asterixdb/repo
Commit: http://git-wip-us.apache.org/repos/asf/asterixdb/commit/5b102cca
Tree: http://git-wip-us.apache.org/repos/asf/asterixdb/tree/5b102cca
Diff: http://git-wip-us.apache.org/repos/asf/asterixdb/diff/5b102cca

Branch: refs/heads/ecarm002/interval_join_merge
Commit: 5b102cca23ad96a91df22d5490ea371ba2a14ac4
Parents: 9986726
Author: Preston Carman 
Authored: Mon Jul 18 16:52:02 2016 -0700
Committer: Preston Carman 
Committed: Mon Jul 18 16:52:02 2016 -0700

--
 ...AbstractIntervalInverseMergeJoinChecker.java | 15 ++---
 .../joins/AbstractIntervalMergeJoinChecker.java | 63 +++-
 .../joins/AfterIntervalMergeJoinChecker.java| 17 ++
 .../joins/BeforeIntervalMergeJoinChecker.java   | 17 ++
 .../CoveredByIntervalMergeJoinChecker.java  |  4 +-
 .../joins/CoversIntervalMergeJoinChecker.java   |  4 +-
 .../joins/EndedByIntervalMergeJoinChecker.java  |  4 +-
 .../joins/EndsIntervalMergeJoinChecker.java |  7 +--
 .../joins/EqualsIntervalMergeJoinChecker.java   |  6 +-
 .../joins/IIntervalMergeJoinChecker.java|  4 +-
 .../joins/MeetsIntervalMergeJoinChecker.java|  4 +-
 .../joins/MetByIntervalMergeJoinChecker.java| 17 ++
 .../OverlappedByIntervalMergeJoinChecker.java   |  4 +-
 .../OverlappingIntervalMergeJoinChecker.java|  3 +-
 .../joins/OverlapsIntervalMergeJoinChecker.java |  4 +-
 .../StartedByIntervalMergeJoinChecker.java  |  4 +-
 .../joins/StartsIntervalMergeJoinChecker.java   | 34 +--
 .../intervalindex/IntervalIndexJoiner.java  |  8 +--
 .../IntervalPartitionUtil.java  |  5 +-
 .../VPartitionDeletableTupleBufferManager.java  | 10 ++--
 20 files changed, 93 insertions(+), 141 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/asterixdb/blob/5b102cca/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/AbstractIntervalInverseMergeJoinChecker.java
--
diff --git 
a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/AbstractIntervalInverseMergeJoinChecker.java
 
b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/AbstractIntervalInverseMergeJoinChecker.java
index 9f9f75d..4343386 100644
--- 
a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/AbstractIntervalInverseMergeJoinChecker.java
+++ 
b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/AbstractIntervalInverseMergeJoinChecker.java
@@ -18,7 +18,6 @@
  */
 package org.apache.asterix.runtime.operators.joins;
 
-import org.apache.asterix.common.exceptions.AsterixException;
 import org.apache.hyracks.api.exceptions.HyracksDataException;
 import org.apache.hyracks.dataflow.std.buffermanager.ITupleAccessor;
 
@@ -33,15 +32,11 @@ public abstract class 
AbstractIntervalInverseMergeJoinChecker extends AbstractIn
 @Override
 public boolean checkToSaveInMemory(ITupleAccessor accessorLeft, 
ITupleAccessor accessorRight)
 throws HyracksDataException {
-try {
-IntervalJoinUtil.getIntervalPointable(accessorLeft, idLeft, tvp, 
ipLeft);
-IntervalJoinUtil.getIntervalPointable(accessorRight, idRight, tvp, 
ipRight);
-ipLeft.getStart(startLeft);
-ipRight.getEnd(endRight);
-return ch.compare(ipLeft.getTypeTag(), ipRight.getTypeTag(), 
startLeft, endRight) <= 0;
-} catch (AsterixException e) {
-throw new HyracksDataException(e);
-}
+IntervalJoinUtil.getIntervalPointable(accessorLeft, idLeft, tvp, 
ipLeft);
+IntervalJoinUtil.getIntervalPointable(accessorRight, idRight, tvp, 
ipRight);
+ipLeft.getStart(startLeft);
+ipRight.getEnd(endRight);
+return ch.compare(ipLeft.getTypeTag(), ipRight.getTypeTag(), 
startLeft, endRight) <= 0;
 }
 
 @Override

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/5b102cca/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/AbstractIntervalMergeJoinChecker.java
--
diff --git 
a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/AbstractIntervalMergeJoinChecker.java
 
b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/AbstractIntervalMergeJoinChecker.java
index cf0bf6a..b461799 100644
--- 
a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/AbstractIntervalMergeJoinChecker.java
+++

[35/50] [abbrv] asterixdb git commit: snapshot - reuse frames in deletable partition manager.

2016-10-17 Thread prestonc

snapshot - reuse frames in deletable partition manager.


Project: http://git-wip-us.apache.org/repos/asf/asterixdb/repo
Commit: http://git-wip-us.apache.org/repos/asf/asterixdb/commit/6c312141
Tree: http://git-wip-us.apache.org/repos/asf/asterixdb/tree/6c312141
Diff: http://git-wip-us.apache.org/repos/asf/asterixdb/diff/6c312141

Branch: refs/heads/ecarm002/interval_join_merge
Commit: 6c3121413762f3556d884e16642cefa654e42868
Parents: f186da9
Author: Preston Carman 
Authored: Tue Sep 13 19:37:32 2016 -0700
Committer: Preston Carman 
Committed: Tue Sep 13 19:37:32 2016 -0700

--
 .../operators/joins/IntervalJoinUtil.java   |  1 +
 .../intervalindex/IntervalIndexJoiner.java  | 70 
 .../InMemoryIntervalPartitionJoin.java  |  5 +-
 .../std/buffermanager/IFrameBufferManager.java  | 24 ++-
 ...IPartitionedDeletableTupleBufferManager.java |  2 +
 .../VPartitionDeletableTupleBufferManager.java  | 50 --
 .../VPartitionTupleBufferManager.java   | 57 ++--
 .../VariableFrameMemoryManager.java | 23 +++
 .../hyracks/dataflow/std/join/MergeJoiner.java  | 53 ---
 .../dataflow/std/join/RunFileStream.java|  1 +
 .../sort/util/DeletableFrameTupleAppender.java  | 26 +++-
 .../IAppendDeletableFrameTupleAccessor.java |  3 +
 12 files changed, 249 insertions(+), 66 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/asterixdb/blob/6c312141/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/IntervalJoinUtil.java
--
diff --git 
a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/IntervalJoinUtil.java
 
b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/IntervalJoinUtil.java
index 3e380c7..7c37f08 100644
--- 
a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/IntervalJoinUtil.java
+++ 
b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/IntervalJoinUtil.java
@@ -48,6 +48,7 @@ public class IntervalJoinUtil {
 }
 
 public static long getIntervalStart(IFrameTupleAccessor accessor, int 
tupleId, int fieldId) {
+int length = accessor.getTupleLength(tupleId);
 int start = accessor.getTupleStartOffset(tupleId) + 
accessor.getFieldSlotsLength()
 + accessor.getFieldStartOffset(tupleId, fieldId) + 1;
 long intervalStart = 
AIntervalSerializerDeserializer.getIntervalStart(accessor.getBuffer().array(), 
start);

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/6c312141/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalindex/IntervalIndexJoiner.java
--
diff --git 
a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalindex/IntervalIndexJoiner.java
 
b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalindex/IntervalIndexJoiner.java
index 965411b..d3aaa65 100644
--- 
a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalindex/IntervalIndexJoiner.java
+++ 
b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalindex/IntervalIndexJoiner.java
@@ -73,7 +73,6 @@ public class IntervalIndexJoiner extends AbstractMergeJoiner {
 
 private long joinComparisonCount = 0;
 private long joinResultCount = 0;
-private long spillCount = 0;
 private long leftSpillCount = 0;
 private long rightSpillCount = 0;
 private long[] spillFileCount = { 0, 0 };
@@ -104,16 +103,21 @@ public class IntervalIndexJoiner extends 
AbstractMergeJoiner {
 throw new HyracksDataException(
 "IntervalIndexJoiner does not have enough memory (needs > 
4, got " + memorySize + ").");
 }
+//bufferManager = new 
VPartitionDeletableTupleBufferManager(ctx,
+//VPartitionDeletableTupleBufferManager.NO_CONSTRAIN, 
JOIN_PARTITIONS,
+//(memorySize - 4) * ctx.getInitialFrameSize(), 
recordDescriptors);
 bufferManager = new VPartitionDeletableTupleBufferManager(ctx,
 VPartitionDeletableTupleBufferManager.NO_CONSTRAIN, 
JOIN_PARTITIONS,
-(memorySize - 4) * ctx.getInitialFrameSize(), 
recordDescriptors);
+memorySize * ctx.getInitialFrameSize(), recordDescriptors);
 memoryAccessor = new ITuplePointerAccessor[JOIN_PARTITIONS];
 memoryAccessor[LEFT_PARTITION] = 
bufferManager.getTuplePointerAccessor(leftRd);
 memoryAccessor[RIGHT_PARTITION] =

[15/50] [abbrv] asterixdb git commit: merge master with all interval prep clean up items.

2016-10-17 Thread prestonc

merge master with all interval prep clean up items.


Project: http://git-wip-us.apache.org/repos/asf/asterixdb/repo
Commit: http://git-wip-us.apache.org/repos/asf/asterixdb/commit/c44a7d9b
Tree: http://git-wip-us.apache.org/repos/asf/asterixdb/tree/c44a7d9b
Diff: http://git-wip-us.apache.org/repos/asf/asterixdb/diff/c44a7d9b

Branch: refs/heads/ecarm002/interval_join_merge
Commit: c44a7d9b67e58b9cae4ab38634a944e0940bb1b6
Parents: 88cbdc4 bce1888
Author: Preston Carman 
Authored: Sat Jul 16 22:40:39 2016 -0700
Committer: Preston Carman 
Committed: Sat Jul 16 22:40:39 2016 -0700

--
 .../AqlPlusExpressionToPlanTranslator.java  |  77 ++-
 .../LangExpressionToPlanTranslator.java | 132 ++
 .../asterix-algebra/src/main/javacc/AQLPlus.jj  |  47 +--
 .../optimizerts/queries/subquery/exists.sqlpp   |  73 ++
 .../queries/subquery/not_exists.sqlpp   |  73 ++
 .../optimizerts/results/subquery/exists.plan|  74 ++
 .../results/subquery/not_exists.plan|  74 ++
 .../parserts/results_parser_sqlpp/constant.ast  |   2 +-
 .../list/exists/exists.3.query.sqlpp|  20 +++
 .../subquery/exists/exists.1.ddl.sqlpp  |  55 
 .../subquery/exists/exists.2.update.sqlpp   |  18 +++
 .../subquery/exists/exists.3.query.sqlpp|  40 ++
 .../subquery/not_exists/not_exists.1.ddl.sqlpp  |  55 
 .../not_exists/not_exists.2.update.sqlpp|  18 +++
 .../not_exists/not_exists.3.query.sqlpp |  40 ++
 .../relational_division.1.ddl.sqlpp |  51 +++
 .../relational_division.2.update.sqlpp  |  24 
 .../relational_division.3.query.sqlpp   |  35 +
 .../relational_division2.1.ddl.sqlpp|  51 +++
 .../relational_division2.2.update.sqlpp |  24 
 .../relational_division2.3.query.sqlpp  |  41 ++
 .../relational_division3.1.ddl.sqlpp|  51 +++
 .../relational_division3.2.update.sqlpp |  24 
 .../relational_division3.3.query.sqlpp  |  38 ++
 .../runtimets/results/list/exists/exists.1.adm  |   1 +
 .../results/subquery/exists/exists.1.adm|  21 +++
 .../subquery/not_exists/not_exists.1.adm|  18 +++
 .../relational_division.1.adm   |  74 ++
 .../binary/subbinary/subbinary_01.3.ast |   6 +-
 .../custord/customer_q_04/customer_q_04.3.ast   |   2 +-
 .../custord/customer_q_05/customer_q_05.3.ast   |   2 +-
 .../list/listify_03/listify_03.3.ast|   4 +-
 .../numeric/abs4/abs4.3.ast |   2 +-
 .../numeric/add_double/add_double.1.ast |   2 +-
 .../numeric/ceiling4/ceiling4.3.ast |   2 +-
 .../numeric/floor4/floor4.3.ast |   2 +-
 .../round-half-to-even24.3.ast  |   2 +-
 .../round-half-to-even5.3.ast   |   2 +-
 .../numeric/round4/round4.3.ast |   2 +-
 .../unary-minus_double_02.3.ast |   8 +-
 .../unary-minus_float_02.3.ast  |   8 +-
 .../unary-minus_int_02/unary-minus_int_02.3.ast |   8 +-
 .../unary-minus_null/unary-minus_null.1.ast |   4 +-
 .../query-issue134/query-issue134.3.ast |   2 +-
 .../query-issue442/query-issue442.1.ast |   2 +-
 .../query-issue55/query-issue55.3.ast   |   4 +-
 .../quantifiers/everysat_01/everysat_01.3.ast   |   6 +-
 .../quantifiers/somesat_01/somesat_01.3.ast |   8 +-
 .../cell-aggregation-with-filtering.3.ast   |   2 +-
 .../spatial/line_accessor/line_accessor.3.ast   |   4 +-
 .../rectangle-intersect-rectangle.3.ast |   2 +-
 .../string/substr01/substr01.3.ast  |   2 +-
 .../string/substring2-4/substring2-4.3.ast  |   2 +-
 .../query-issue218-2/query-issue218-2.3.ast |   4 +-
 .../resources/runtimets/testsuite_sqlpp.xml |  30 
 .../aql/visitor/AQLToSQLPPPrintVisitor.java |   6 +-
 .../asterix-lang-aql/src/main/javacc/AQL.jj |  44 --
 .../lang/common/expression/OperatorExpr.java|   4 +-
 .../lang/common/expression/UnaryExpr.java   |  31 +++--
 .../lang/common/struct/UnaryExprType.java   |  44 ++
 .../CloneAndSubstituteVariablesVisitor.java |   2 +-
 .../lang/common/visitor/FormatPrintVisitor.java |   4 +-
 .../lang/common/visitor/QueryPrintVisitor.java  |   4 +-
 .../visitor/OperatorExpressionVisitor.java  |   2 +-
 .../lang/sqlpp/visitor/DeepCopyVisitor.java |   2 +-
 ...SqlppCloneAndSubstituteVariablesVisitor.java |   3 +
 .../src/main/javacc/SQLPP.html  | 131 +-
 .../asterix-lang-sqlpp/src/main/javacc/SQLPP.jj |  98 -
 .../nonvisitor/AIntervalPointable.java  |  24 ++--
 .../AbstractComparisonEvaluator.java|   2 +-
 .../comparisons/ComparisonHelper.java   |  45 +++---

[20/50] [abbrv] asterixdb git commit: snapshot for range state transition

2016-10-17 Thread prestonc

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/2061a388/hyracks-fullstack/hyracks/hyracks-dataflow-common/src/main/java/org/apache/hyracks/dataflow/common/data/partition/range/IRangeMap.java
--
diff --git 
a/hyracks-fullstack/hyracks/hyracks-dataflow-common/src/main/java/org/apache/hyracks/dataflow/common/data/partition/range/IRangeMap.java
 
b/hyracks-fullstack/hyracks/hyracks-dataflow-common/src/main/java/org/apache/hyracks/dataflow/common/data/partition/range/IRangeMap.java
deleted file mode 100644
index ff2e40b..000
--- 
a/hyracks-fullstack/hyracks/hyracks-dataflow-common/src/main/java/org/apache/hyracks/dataflow/common/data/partition/range/IRangeMap.java
+++ /dev/null
@@ -1,50 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements.  See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership.  The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License.  You may obtain a copy of the License at
- *
- *   http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied.  See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-package org.apache.hyracks.dataflow.common.data.partition.range;
-
-public interface IRangeMap {
-
-public int getSplitCount();
-
-public byte[] getByteArray(int columnIndex, int splitIndex);
-
-public int getStartOffset(int columnIndex, int splitIndex);
-
-public int getLength(int columnIndex, int splitIndex);
-
-public int getTag(int columnIndex, int splitIndex);
-
-// Min value functions
-public byte[] getMinByteArray(int columnIndex);
-
-public int getMinStartOffset(int columnIndex);
-
-public int getMinLength(int columnIndex);
-
-public int getMinTag(int columnIndex);
-
-// Max value functions
-public byte[] getMaxByteArray(int columnIndex);
-
-public int getMaxStartOffset(int columnIndex);
-
-public int getMaxLength(int columnIndex);
-
-public int getMaxTag(int columnIndex);
-}

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/2061a388/hyracks-fullstack/hyracks/hyracks-dataflow-common/src/main/java/org/apache/hyracks/dataflow/common/data/partition/range/IRangePartitionType.java
--
diff --git 
a/hyracks-fullstack/hyracks/hyracks-dataflow-common/src/main/java/org/apache/hyracks/dataflow/common/data/partition/range/IRangePartitionType.java
 
b/hyracks-fullstack/hyracks/hyracks-dataflow-common/src/main/java/org/apache/hyracks/dataflow/common/data/partition/range/IRangePartitionType.java
deleted file mode 100644
index dcde70b..000
--- 
a/hyracks-fullstack/hyracks/hyracks-dataflow-common/src/main/java/org/apache/hyracks/dataflow/common/data/partition/range/IRangePartitionType.java
+++ /dev/null
@@ -1,41 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements.  See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership.  The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License.  You may obtain a copy of the License at
- *
- *   http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied.  See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-package org.apache.hyracks.dataflow.common.data.partition.range;
-
-public interface IRangePartitionType {
-public enum RangePartitioningType {
-/**
- * Partitioning is determined by finding the range partition where the 
first data point lies.
- */
-PROJECT,
-/**
- * Partitioning is determined by finding the range partition where the 
last data point lies.
- */
-PROJECT_END,
-/**
- * Partitioning is determined by finding all the range partitions 
where the data has a point.
- */
-SPLIT,
-/**
- * Partitioning is determined by finding all the range partitions 
where the data has a point
- * or comes after the data point.
- */
-REPLICATE
-}
-}

[11/50] [abbrv] asterixdb git commit: merge master

2016-10-17 Thread prestonc

merge master


Project: http://git-wip-us.apache.org/repos/asf/asterixdb/repo
Commit: http://git-wip-us.apache.org/repos/asf/asterixdb/commit/0725160a
Tree: http://git-wip-us.apache.org/repos/asf/asterixdb/tree/0725160a
Diff: http://git-wip-us.apache.org/repos/asf/asterixdb/diff/0725160a

Branch: refs/heads/ecarm002/interval_join_merge
Commit: 0725160aa57181e9f82896af1552e57d87728db9
Parents: 4f9e6a8 61ec128
Author: Preston Carman 
Authored: Thu Jul 14 13:54:55 2016 -0700
Committer: Preston Carman 
Committed: Thu Jul 14 13:54:55 2016 -0700

--
 asterixdb/asterix-app/pom.xml   |  29 ++
 .../asterix/test/sqlpp/ParserTestExecutor.java  |  11 +-
 .../optimizerts/queries/query-issue697.aql  |   2 +-
 .../sum_null-with-pred.3.query.aql  |   2 +-
 .../sum_null-with-pred.3.query.aql  |   2 +-
 .../queries/boolean/not_01/not_01.3.query.aql   |   4 +-
 .../everysat_04/everysat_04.3.query.aql |   8 +-
 .../somesat_06/somesat_06.3.query.aql   |   8 +-
 .../has-param1/has-param1.3.query.aql   |   2 +-
 .../substring-after-5.3.query.aql   |   2 +-
 .../substring-after-6.3.query.aql   |   2 +-
 .../tinysocial-suite.13.query.aql   |   2 +-
 .../q13_customer_distribution.3.query.aql   |   2 +-
 .../q16_parts_supplier_relationship.3.query.aql |   4 +-
 .../q13_customer_distribution.3.query.aql   |   2 +-
 .../q16_parts_supplier_relationship.3.query.aql |   4 +-
 .../single-line-definition.1.ddl.aql|  31 +++
 .../single-line-definition.2.query.aql  |  28 ++
 .../single-line-definition.3.ddl.aql|  27 ++
 .../udf18/udf18.3.query.aql |   2 +-
 .../queries_sqlpp/dapd/q2-6/q2-6.1.ddl.sqlpp|  55 
 .../queries_sqlpp/dapd/q2-6/q2-6.2.update.sqlpp |  19 ++
 .../queries_sqlpp/dapd/q2-6/q2-6.3.query.sqlpp  |  35 +++
 .../group-by/gby-expr-2/gby-expr-2.1.ddl.sqlpp  |  39 +++
 .../gby-expr-2/gby-expr-2.2.update.sqlpp|  43 +++
 .../gby-expr-2/gby-expr-2.3.query.sqlpp |  27 ++
 .../group-by/gby-expr-3/gby-expr-3.1.ddl.sqlpp  |  39 +++
 .../gby-expr-3/gby-expr-3.2.update.sqlpp|  43 +++
 .../gby-expr-3/gby-expr-3.3.query.sqlpp |  33 +++
 .../group-by/gby-expr/gby-expr.1.ddl.sqlpp  |  39 +++
 .../group-by/gby-expr/gby-expr.2.update.sqlpp   |  43 +++
 .../group-by/gby-expr/gby-expr.3.query.sqlpp|  27 ++
 .../group-by/sugar-01-2/sugar-01-2.1.ddl.sqlpp  |  29 ++
 .../sugar-01-2/sugar-01-2.2.update.sqlpp|  32 +++
 .../sugar-01-2/sugar-01-2.3.query.sqlpp |  26 ++
 .../group-by/sugar-02-2/sugar-02-2.1.ddl.sqlpp  |  39 +++
 .../sugar-02-2/sugar-02-2.2.update.sqlpp|  43 +++
 .../sugar-02-2/sugar-02-2.3.query.sqlpp |  27 ++
 .../sugar-02-2/sugar-02-2.4.query.sqlpp |  31 +++
 .../group-by/sugar-03-2/sugar-03-2.1.ddl.sqlpp  |  40 +++
 .../sugar-03-2/sugar-03-2.2.update.sqlpp|  42 +++
 .../sugar-03-2/sugar-03-2.3.query.sqlpp |  32 +++
 .../group-by/sugar-04-2/sugar-04-2.1.ddl.sqlpp  |  40 +++
 .../sugar-04-2/sugar-04-2.2.update.sqlpp|  55 
 .../sugar-04-2/sugar-04-2.3.query.sqlpp |  31 +++
 .../group-by/sugar-05-2/sugar-05-2.1.ddl.sqlpp  |  35 +++
 .../sugar-05-2/sugar-05-2.2.update.sqlpp|  59 
 .../sugar-05-2/sugar-05-2.3.query.sqlpp |  38 +++
 .../alias_negative/alias_negative.1.query.sqlpp |  24 ++
 .../q01_pricing_summary_report_2.1.ddl.sqlpp|  47 
 .../q01_pricing_summary_report_2.2.update.sqlpp |  24 ++
 .../q01_pricing_summary_report_2.3.query.sqlpp  |  43 +++
 .../single-line-definition.1.ddl.sqlpp  |  31 +++
 .../single-line-definition.2.query.sqlpp|  30 ++
 .../single-line-definition.3.ddl.sqlpp  |  27 ++
 .../group-by/gby-expr-2/gby-expr-2.1.adm|   1 +
 .../group-by/gby-expr-3/gby-expr-3.1.adm|   1 +
 .../results/group-by/gby-expr/gby-expr.1.adm|   1 +
 .../single-line-definition.1.adm|   1 +
 .../src/test/resources/runtimets/testsuite.xml  |   5 +
 .../resources/runtimets/testsuite_sqlpp.xml |  61 +
 .../asterix-lang-aql/src/main/javacc/AQL.jj | 250 ++---
 .../lang/common/clause/GroupbyClause.java   |  32 ++-
 .../asterix/lang/common/clause/LetClause.java   |  18 ++
 .../asterix/lang/common/clause/LimitClause.java |  27 +-
 .../lang/common/clause/OrderbyClause.java   |  20 ++
 .../lang/common/clause/UpdateClause.java|  22 ++
 .../asterix/lang/common/clause/WhereClause.java |  31 ++-
 .../common/expression/AbstractAccessor.java |  18 ++
 .../lang/common/expression/CallExpr.java|  21 +-
 .../lang/common/expression/FieldAccessor.java   |  18 ++
 .../lang/common/expression/FieldBinding.java|  24 ++
 .../expression/GbyVariableExpressionPair.java   |  22 ++
 .../asterix/lang/common/expression/IfExpr.java  |  25 ++

[30/50] [abbrv] asterixdb git commit: merge master

2016-10-17 Thread prestonc

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/c2e48a78/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java
--
diff --cc 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java
index a4959ab,cf36c05..f6f0862
--- 
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java
+++ 
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/base/RuleCollections.java
@@@ -36,9 -37,9 +37,10 @@@ import org.apache.asterix.optimizer.rul
  import org.apache.asterix.optimizer.rules.ExtractOrderExpressionsRule;
  import org.apache.asterix.optimizer.rules.FeedScanCollectionToUnnest;
  import org.apache.asterix.optimizer.rules.FuzzyEqRule;
- import org.apache.asterix.optimizer.rules.IfElseToSwitchCaseFunctionRule;
+ import org.apache.asterix.optimizer.rules.InjectTypeCastForSwitchCaseRule;
+ import org.apache.asterix.optimizer.rules.InjectTypeCastForUnionRule;
  import org.apache.asterix.optimizer.rules.InlineUnnestFunctionRule;
 +import org.apache.asterix.optimizer.rules.IntervalSplitPartitioningRule;
  import org.apache.asterix.optimizer.rules.IntroduceAutogenerateIDRule;
  import 
org.apache.asterix.optimizer.rules.IntroduceDynamicTypeCastForExternalFunctionRule;
  import org.apache.asterix.optimizer.rules.IntroduceDynamicTypeCastRule;

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/c2e48a78/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/util/FunctionCollection.java
--

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/c2e48a78/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/common/APIFramework.java
--
diff --cc 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/common/APIFramework.java
index 61057b4,76d0245..294f4d0
--- 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/common/APIFramework.java
+++ 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/common/APIFramework.java
@@@ -244,15 -254,11 +255,15 @@@ public class APIFramework 
  
OptimizationConfUtil.getPhysicalOptimizationConfig().setMaxFramesExternalSort(sortFrameLimit);
  
OptimizationConfUtil.getPhysicalOptimizationConfig().setMaxFramesExternalGroupBy(groupFrameLimit);
  
OptimizationConfUtil.getPhysicalOptimizationConfig().setMaxFramesForJoin(joinFrameLimit);
 +
OptimizationConfUtil.getPhysicalOptimizationConfig().setMaxFramesForJoinLeftInput(joinFrameLeftInputLimit);
 +
 +int intervalMaxDuration = (int) 
(compilerProperties.getIntervalMaxDuration());
 +
OptimizationConfUtil.getPhysicalOptimizationConfig().getMaxIntervalDuration(intervalMaxDuration);
  
- HeuristicCompilerFactoryBuilder builder = new 
HeuristicCompilerFactoryBuilder(
- AqlOptimizationContextFactory.INSTANCE);
+ HeuristicCompilerFactoryBuilder builder =
+ new 
HeuristicCompilerFactoryBuilder(AqlOptimizationContextFactory.INSTANCE);
  
builder.setPhysicalOptimizationConfig(OptimizationConfUtil.getPhysicalOptimizationConfig());
- builder.setLogicalRewrites(buildDefaultLogicalRewrites());
+ 
builder.setLogicalRewrites(buildDefaultLogicalRewrites(cExtensionManager));
  builder.setPhysicalRewrites(buildDefaultPhysicalRewrites());
  IDataFormat format = queryMetadataProvider.getFormat();
  ICompilerFactory compilerFactory = builder.create();

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/c2e48a78/asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
--

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/c2e48a78/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixCompilerProperties.java
--
diff --cc 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixCompilerProperties.java
index 0a684ed,5bfb5f8..b79c2cb
--- 
a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixCompilerProperties.java
+++ 
b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixCompilerProperties.java
@@@ -53,27 -45,17 +53,27 @@@ public class AsterixCompilerProperties 
  
  public long getJoinMemorySize() {
  return accessor.getProperty(COMPILER_JOINMEMORY_KEY, 
COMPILER_JOINMEMORY_DEFAULT,
- PropertyInterpreters.getLongPropertyInterpreter());
+ PropertyInterpreters.getLongBytePropertyInterpreter());
  }
  
 +public long getJoinLeftInputSize() {
 +return accessor.getProperty(COMPILER_JOIN_LEFTINPUT_KEY, 
COMPILER_JOIN_LEFTINPUT_DEFAULT,
 +

[01/50] [abbrv] asterixdb git commit: fix for indexing memory error.

2016-10-17 Thread prestonc

Repository: asterixdb
Updated Branches:
  refs/heads/ecarm002/interval_join_merge [created] faa3bb457


fix for indexing memory error.


Project: http://git-wip-us.apache.org/repos/asf/asterixdb/repo
Commit: http://git-wip-us.apache.org/repos/asf/asterixdb/commit/199bddd3
Tree: http://git-wip-us.apache.org/repos/asf/asterixdb/tree/199bddd3
Diff: http://git-wip-us.apache.org/repos/asf/asterixdb/diff/199bddd3

Branch: refs/heads/ecarm002/interval_join_merge
Commit: 199bddd3c68ef8e8c3ee0cccf6d8cdcc38fd8189
Parents: f9443fa
Author: Preston Carman 
Authored: Wed Jun 15 19:06:51 2016 -0700
Committer: Preston Carman 
Committed: Wed Jun 15 19:06:51 2016 -0700

--
 .../joins/intervalindex/ActiveSweepManager.java | 10 +++
 .../IntervalIndexJoinOperatorDescriptor.java|  1 +
 .../intervalindex/IntervalIndexJoiner.java  | 29 ++--
 .../dataflow/std/join/AbstractMergeJoiner.java  | 12 
 .../dataflow/std/join/MergeBranchStatus.java|  6 
 .../std/join/MergeJoinOperatorDescriptor.java   |  1 +
 6 files changed, 29 insertions(+), 30 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/asterixdb/blob/199bddd3/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalindex/ActiveSweepManager.java
--
diff --git 
a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalindex/ActiveSweepManager.java
 
b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalindex/ActiveSweepManager.java
index 70785ab..84bd262 100644
--- 
a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalindex/ActiveSweepManager.java
+++ 
b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalindex/ActiveSweepManager.java
@@ -44,18 +44,18 @@ public class ActiveSweepManager {
 private EndPointIndexItem item = null;
 private final LinkedList active = new LinkedList<>();
 
-public ActiveSweepManager(IPartitionedDeletableTupleBufferManager 
bufferManager, int key, int partition,
+public ActiveSweepManager(IPartitionedDeletableTupleBufferManager 
bufferManager, int key, int joinBranch,
 Comparator endPointComparator) {
 this.bufferManager = bufferManager;
 this.key = key;
-this.partition = partition;
+this.partition = joinBranch;
 indexQueue = new PriorityQueue<>(16, endPointComparator);
 }
 
-public boolean addTuple(ITupleAccessor leftInputAccessor, TuplePointer tp) 
throws HyracksDataException {
-if (bufferManager.insertTuple(partition, leftInputAccessor, 
leftInputAccessor.getTupleId(), tp)) {
+public boolean addTuple(ITupleAccessor accessor, TuplePointer tp) throws 
HyracksDataException {
+if (bufferManager.insertTuple(partition, accessor, 
accessor.getTupleId(), tp)) {
 EndPointIndexItem e = new EndPointIndexItem(tp, 
EndPointIndexItem.END_POINT,
-IntervalJoinUtil.getIntervalEnd(leftInputAccessor, 
leftInputAccessor.getTupleId(), key));
+IntervalJoinUtil.getIntervalEnd(accessor, 
accessor.getTupleId(), key));
 indexQueue.add(e);
 active.add(tp);
 item = indexQueue.peek();

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/199bddd3/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalindex/IntervalIndexJoinOperatorDescriptor.java
--
diff --git 
a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalindex/IntervalIndexJoinOperatorDescriptor.java
 
b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalindex/IntervalIndexJoinOperatorDescriptor.java
index c248f72..be44df3 100644
--- 
a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalindex/IntervalIndexJoinOperatorDescriptor.java
+++ 
b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalindex/IntervalIndexJoinOperatorDescriptor.java
@@ -258,6 +258,7 @@ public class IntervalIndexJoinOperatorDescriptor extends 
AbstractOperatorDescrip
 locks.getRight(partition).await();
 }
 state.indexJoiner.setFrame(RIGHT_ACTIVITY_ID, buffer);
+state.status.continueRightLoad = false;
 locks.getLeft(partition).signal();
 } catch (InterruptedException e) {
 Thread.currentThread().interrupt();

[28/50] [abbrv] asterixdb git commit: snapshot with logging

2016-10-17 Thread prestonc

snapshot with logging


Project: http://git-wip-us.apache.org/repos/asf/asterixdb/repo
Commit: http://git-wip-us.apache.org/repos/asf/asterixdb/commit/1df9a9c7
Tree: http://git-wip-us.apache.org/repos/asf/asterixdb/tree/1df9a9c7
Diff: http://git-wip-us.apache.org/repos/asf/asterixdb/diff/1df9a9c7

Branch: refs/heads/ecarm002/interval_join_merge
Commit: 1df9a9c7db4b686e75b870ee02080f8fe15ca666
Parents: 1fae6ac
Author: Preston Carman 
Authored: Tue Sep 6 15:51:12 2016 -0700
Committer: Preston Carman 
Committed: Tue Sep 6 15:51:12 2016 -0700

--
 .../joins/AbstractIntervalMergeJoinChecker.java |  6 +-
 .../IntervalIndexJoinOperatorDescriptor.java|  6 +-
 .../intervalindex/IntervalIndexJoiner.java  | 86 +---
 .../dataflow/std/join/AbstractMergeJoiner.java  | 24 +++---
 .../hyracks/dataflow/std/join/IMergeJoiner.java |  2 +
 .../std/join/MergeJoinOperatorDescriptor.java   | 18 +++-
 .../hyracks/dataflow/std/join/MergeJoiner.java  | 80 +-
 .../dataflow/std/join/NestedLoopJoin.java   |  8 +-
 .../dataflow/std/join/RunFileStream.java|  9 +-
 9 files changed, 162 insertions(+), 77 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/asterixdb/blob/1df9a9c7/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/AbstractIntervalMergeJoinChecker.java
--
diff --git 
a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/AbstractIntervalMergeJoinChecker.java
 
b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/AbstractIntervalMergeJoinChecker.java
index b461799..ec8ecda 100644
--- 
a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/AbstractIntervalMergeJoinChecker.java
+++ 
b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/AbstractIntervalMergeJoinChecker.java
@@ -69,7 +69,7 @@ public abstract class AbstractIntervalMergeJoinChecker 
implements IIntervalMerge
 IntervalJoinUtil.getIntervalPointable(accessorRight, idRight, tvp, 
ipRight);
 ipLeft.getEnd(endLeft);
 ipRight.getStart(startRight);
-return ch.compare(ipLeft.getTypeTag(), ipRight.getTypeTag(), endLeft, 
startRight) >= 0;
+return ch.compare(ipLeft.getTypeTag(), ipRight.getTypeTag(), endLeft, 
startRight) > 0;
 }
 
 @Override
@@ -79,7 +79,7 @@ public abstract class AbstractIntervalMergeJoinChecker 
implements IIntervalMerge
 IntervalJoinUtil.getIntervalPointable(accessorRight, idRight, tvp, 
ipRight);
 ipLeft.getStart(startLeft);
 ipRight.getEnd(endRight);
-return !(ch.compare(ipLeft.getTypeTag(), ipRight.getTypeTag(), 
startLeft, endRight) <= 0);
+return !(ch.compare(ipLeft.getTypeTag(), ipRight.getTypeTag(), 
startLeft, endRight) < 0);
 }
 
 @Override
@@ -102,7 +102,7 @@ public abstract class AbstractIntervalMergeJoinChecker 
implements IIntervalMerge
 IntervalJoinUtil.getIntervalPointable(accessorRight, rightTupleIndex, 
idRight, tvp, ipRight);
 ipLeft.getStart(startLeft);
 ipRight.getEnd(endRight);
-return !(ch.compare(ipLeft.getTypeTag(), ipRight.getTypeTag(), 
startLeft, endRight) <= 0);
+return !(ch.compare(ipLeft.getTypeTag(), ipRight.getTypeTag(), 
startLeft, endRight) < 0);
 }
 
 @Override

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/1df9a9c7/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalindex/IntervalIndexJoinOperatorDescriptor.java
--
diff --git 
a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalindex/IntervalIndexJoinOperatorDescriptor.java
 
b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalindex/IntervalIndexJoinOperatorDescriptor.java
index be44df3..d84fabc 100644
--- 
a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalindex/IntervalIndexJoinOperatorDescriptor.java
+++ 
b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalindex/IntervalIndexJoinOperatorDescriptor.java
@@ -37,6 +37,7 @@ import 
org.apache.hyracks.dataflow.std.base.AbstractActivityNode;
 import org.apache.hyracks.dataflow.std.base.AbstractOperatorDescriptor;
 import 
org.apache.hyracks.dataflow.std.base.AbstractUnaryInputSinkOperatorNodePushable;
 import 
org.apache.hyracks.dataflow.std.base.AbstractUnaryInputUnaryOutputOperatorNodePushable;
+import org.apache.hyracks.dataflow.std.join.MergeBranchStatus.Stage;
 import org.apache.hyracks.dataflow.std.join.MergeJoinLocks;
 
 public class

[42/50] [abbrv] asterixdb git commit: working partition join.

2016-10-17 Thread prestonc

working partition join.


Project: http://git-wip-us.apache.org/repos/asf/asterixdb/repo
Commit: http://git-wip-us.apache.org/repos/asf/asterixdb/commit/b34a426a
Tree: http://git-wip-us.apache.org/repos/asf/asterixdb/tree/b34a426a
Diff: http://git-wip-us.apache.org/repos/asf/asterixdb/diff/b34a426a

Branch: refs/heads/ecarm002/interval_join_merge
Commit: b34a426a48dbf16a2d313b2333eb0e6fb23227e4
Parents: ce59341
Author: Preston Carman 
Authored: Fri Sep 30 13:48:03 2016 -0700
Committer: Preston Carman 
Committed: Fri Sep 30 13:48:03 2016 -0700

--
 .../IntervalPartitionJoinPOperator.java |   2 +-
 .../intervalindex/IntervalIndexJoiner.java  |  62 ++--
 ...IntervalPartitionJoinOperatorDescriptor.java | 319 +++
 .../IntervalPartitionJoinTaskState.java |  33 ++
 .../IntervalPartitionJoiner.java| 288 +
 .../dataflow/common/io/RunFileReader.java   |   4 +
 .../dataflow/std/structures/RunFilePointer.java | 102 ++
 7 files changed, 778 insertions(+), 32 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/asterixdb/blob/b34a426a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/IntervalPartitionJoinPOperator.java
--
diff --git 
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/IntervalPartitionJoinPOperator.java
 
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/IntervalPartitionJoinPOperator.java
index 73d159e..af77a92 100644
--- 
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/IntervalPartitionJoinPOperator.java
+++ 
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/IntervalPartitionJoinPOperator.java
@@ -23,7 +23,7 @@ import java.util.List;
 import java.util.logging.Logger;
 
 import 
org.apache.asterix.runtime.operators.joins.IIntervalMergeJoinCheckerFactory;
-import 
org.apache.asterix.runtime.operators.joins.intervalpartition.IntervalPartitionJoinOperatorDescriptor;
+import 
org.apache.asterix.runtime.operators.joins.intervalpartition2.IntervalPartitionJoinOperatorDescriptor;
 import org.apache.hyracks.algebricks.core.algebra.base.LogicalVariable;
 import 
org.apache.hyracks.algebricks.core.algebra.operators.logical.AbstractBinaryJoinOperator.JoinKind;
 import 
org.apache.hyracks.algebricks.core.algebra.operators.logical.OrderOperator.IOrder.OrderKind;

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/b34a426a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalindex/IntervalIndexJoiner.java
--
diff --git 
a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalindex/IntervalIndexJoiner.java
 
b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalindex/IntervalIndexJoiner.java
index bac2f45..8b7a12e 100644
--- 
a/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalindex/IntervalIndexJoiner.java
+++ 
b/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/joins/intervalindex/IntervalIndexJoiner.java
@@ -61,7 +61,7 @@ public class IntervalIndexJoiner extends AbstractMergeJoiner {
 private final int[] streamIndex;
 private final RunFileStream[] runFileStream;
 
-//private final LinkedList buffer = new LinkedList<>();
+//private final LinkedList buffer = new LinkedList<>();
 
 private final IIntervalMergeJoinChecker imjc;
 
@@ -342,7 +342,7 @@ public class IntervalIndexJoiner extends 
AbstractMergeJoiner {
 if 
(activeManager[LEFT_PARTITION].addTuple(inputAccessor[LEFT_PARTITION], tp)) {
 
processTupleJoin(activeManager[RIGHT_PARTITION].getActiveList(), 
memoryAccessor[RIGHT_PARTITION],
 inputAccessor[LEFT_PARTITION], true, writer);
-//buffer.add(tp);
+//buffer.add(tp);
 } else {
 // Spill case
 freezeAndSpill();
@@ -356,10 +356,10 @@ public class IntervalIndexJoiner extends 
AbstractMergeJoiner {
 } while (loadLeftTuple().isLoaded() && loadRightTuple().isLoaded() && 
!checkToProcessRightTuple());
 
 // Add Results
-//if (!buffer.isEmpty()) {
-//
processActiveJoin(activeManager[RIGHT_PARTITION].getActiveList(), 
memoryAccessor[RIGHT_PARTITION], buffer,
-//memoryAccessor[LEFT_PARTITION], true, writer);
-//}
+//if (!buffer.isEmpty()) {
+//

[29/50] [abbrv] asterixdb git commit: ASTERIXDB-1635 Fix for overlap-bins start for dates.

2016-10-17 Thread prestonc

ASTERIXDB-1635 Fix for overlap-bins start for dates.


Project: http://git-wip-us.apache.org/repos/asf/asterixdb/repo
Commit: http://git-wip-us.apache.org/repos/asf/asterixdb/commit/d4cbcb2f
Tree: http://git-wip-us.apache.org/repos/asf/asterixdb/tree/d4cbcb2f
Diff: http://git-wip-us.apache.org/repos/asf/asterixdb/diff/d4cbcb2f

Branch: refs/heads/ecarm002/interval_join_merge
Commit: d4cbcb2fe05649d02f12a0196eaa508cb5f7c623
Parents: 1df9a9c
Author: Preston Carman 
Authored: Tue Sep 6 16:10:24 2016 -0700
Committer: Preston Carman 
Committed: Tue Sep 6 16:10:24 2016 -0700

--
 .../overlap_bins/overlap_bins.1.query.aql   |   2 +-
 .../overlap_bins/overlap_bins.3.query.sqlpp |   2 +-
 .../temporal/overlap_bins/overlap_bins.1.adm|   2 +-
 .../temporal/overlap_bins/overlap_bins.3.ast|   2 +-
 .../src/site/markdown/aql/functions.md  |   9 +-
 .../om/base/temporal/DateTimeFormatUtils.java   | 200 ++-
 .../temporal/DurationArithmeticOperations.java  |  32 +--
 .../temporal/OverlapBinsDescriptor.java |  25 ++-
 8 files changed, 146 insertions(+), 128 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/asterixdb/blob/d4cbcb2f/asterixdb/asterix-app/src/test/resources/runtimets/queries/temporal/overlap_bins/overlap_bins.1.query.aql
--
diff --git 
a/asterixdb/asterix-app/src/test/resources/runtimets/queries/temporal/overlap_bins/overlap_bins.1.query.aql
 
b/asterixdb/asterix-app/src/test/resources/runtimets/queries/temporal/overlap_bins/overlap_bins.1.query.aql
index 8011ad6..bfb5527 100644
--- 
a/asterixdb/asterix-app/src/test/resources/runtimets/queries/temporal/overlap_bins/overlap_bins.1.query.aql
+++ 
b/asterixdb/asterix-app/src/test/resources/runtimets/queries/temporal/overlap_bins/overlap_bins.1.query.aql
@@ -25,5 +25,5 @@ let $itv1 := interval(time("17:23:37"), time("18:30:21"))
 let $itv2 := interval(date("1984-03-17"), date("2013-08-22"))
 let $itv3 := interval(datetime("1800-01-01T23:59:48.938"), 
datetime("2015-07-26T13:28:30.218"))
 return { "timebins": overlap-bins($itv1, time("00:00:00"), 
day-time-duration("PT30M")),
- "datebins": overlap-bins($itv2, date("1990-01-01"), 
year-month-duration("P20Y")),
+ "datebins": overlap-bins($itv2, date("1990-01-01"), 
year-month-duration("P10Y")),
  "datetimebins": overlap-bins($itv3, 
datetime("1900-01-01T00:00:00.000"), year-month-duration("P100Y")) }

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/d4cbcb2f/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/temporal/overlap_bins/overlap_bins.3.query.sqlpp
--
diff --git 
a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/temporal/overlap_bins/overlap_bins.3.query.sqlpp
 
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/temporal/overlap_bins/overlap_bins.3.query.sqlpp
index e42707f..90cd95e 100644
--- 
a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/temporal/overlap_bins/overlap_bins.3.query.sqlpp
+++ 
b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/temporal/overlap_bins/overlap_bins.3.query.sqlpp
@@ -22,6 +22,6 @@
  **/
 
 { 
'timebins':`overlap-bins`(interval(time('17:23:37'),time('18:30:21')),time('00:00:00'),`day-time-duration`('PT30M')),
-  
'datebins':`overlap-bins`(interval(date('1984-03-17'),date('2013-08-22')),date('1990-01-01'),`year-month-duration`('P20Y')),
+  
'datebins':`overlap-bins`(interval(date('1984-03-17'),date('2013-08-22')),date('1990-01-01'),`year-month-duration`('P10Y')),
   
'datetimebins':`overlap-bins`(interval(datetime('1800-01-01T23:59:48.938'),datetime('2015-07-26T13:28:30.218')),datetime('1900-01-01T00:00:00.000'),
   `year-month-duration`('P100Y'))};

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/d4cbcb2f/asterixdb/asterix-app/src/test/resources/runtimets/results/temporal/overlap_bins/overlap_bins.1.adm
--
diff --git 
a/asterixdb/asterix-app/src/test/resources/runtimets/results/temporal/overlap_bins/overlap_bins.1.adm
 
b/asterixdb/asterix-app/src/test/resources/runtimets/results/temporal/overlap_bins/overlap_bins.1.adm
index 493a1ca..14f6aae 100644
--- 
a/asterixdb/asterix-app/src/test/resources/runtimets/results/temporal/overlap_bins/overlap_bins.1.adm
+++ 
b/asterixdb/asterix-app/src/test/resources/runtimets/results/temporal/overlap_bins/overlap_bins.1.adm
@@ -1 +1 @@
-{ "timebins": [ interval(time("17:00:00.000Z"), time("17:30:00.000Z")), 
interval(time("17:30:00.000Z"), time("18:00:00.000Z")), 
interval(time("18:00:00.000Z"), time("18:30:00.000Z")), 
interval(time("18:30:00.000Z"), time("19:00:00.000Z")) ], "datebins": [

[05/50] [abbrv] asterixdb git commit: added more to the interval hint

2016-10-17 Thread prestonc

added more to the interval hint


Project: http://git-wip-us.apache.org/repos/asf/asterixdb/repo
Commit: http://git-wip-us.apache.org/repos/asf/asterixdb/commit/fd84e345
Tree: http://git-wip-us.apache.org/repos/asf/asterixdb/tree/fd84e345
Diff: http://git-wip-us.apache.org/repos/asf/asterixdb/diff/fd84e345

Branch: refs/heads/ecarm002/interval_join_merge
Commit: fd84e345ec29f9b85f32132eef1f6e7f98941866
Parents: 0f533e8
Author: Preston Carman 
Authored: Wed Jul 6 18:15:43 2016 -0700
Committer: Preston Carman 
Committed: Wed Jul 6 18:15:43 2016 -0700

--
 .../IntervalPartitionJoinPOperator.java |  20 +-
 .../rules/IntervalSplitPartitioningRule.java|   1 -
 .../asterix/optimizer/rules/util/JoinUtils.java |  21 +-
 .../interval_overlapping.11.query.aql   |   2 +-
 .../interval_overlapping.12.query.aql   |   2 +-
 .../IntervalJoinExpressionAnnotation.java   |  65 +-
 ...econdaryIndexSearchExpressionAnnotation.java |  15 +-
 .../asterix-lang-aql/src/main/javacc/AQL.jj |   4 +-
 .../IntervalPartitionJoin.java  | 649 ---
 ...IntervalPartitionJoinOperatorDescriptor.java |  16 +-
 .../IntervalPartitionJoiner.java| 649 +++
 .../IntervalPartitionUtil.java  |  26 +-
 .../AbstractExpressionAnnotation.java   |  35 +
 .../ExpressionAnnotationNoCopyImpl.java |  14 +-
 .../IndexedNLJoinExpressionAnnotation.java  |  14 +-
 15 files changed, 788 insertions(+), 745 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/asterixdb/blob/fd84e345/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/IntervalPartitionJoinPOperator.java
--
diff --git 
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/IntervalPartitionJoinPOperator.java
 
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/IntervalPartitionJoinPOperator.java
index 414d0b4..1eff2aa 100644
--- 
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/IntervalPartitionJoinPOperator.java
+++ 
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/IntervalPartitionJoinPOperator.java
@@ -33,18 +33,18 @@ import 
org.apache.hyracks.dataflow.common.data.partition.range.IRangeMap;
 public class IntervalPartitionJoinPOperator extends 
AbstractIntervalJoinPOperator {
 
 private final int memSizeInFrames;
-private final int probeTupleCount;
-private final int probeMaxDuration;
-private final int buildTupleCount;
-private final int buildMaxDuration;
+private final long probeTupleCount;
+private final long probeMaxDuration;
+private final long buildTupleCount;
+private final long buildMaxDuration;
 private final int avgTuplesInFrame;
 
 private static final Logger LOGGER = 
Logger.getLogger(IntervalPartitionJoinPOperator.class.getName());
 
 public IntervalPartitionJoinPOperator(JoinKind kind, JoinPartitioningType 
partitioningType,
 List sideLeftOfEqualities, List 
sideRightOfEqualities,
-int memSizeInFrames, int buildTupleCount, int probeTupleCount, int 
buildMaxDuration, int probeMaxDuration,
-int avgTuplesInFrame, IIntervalMergeJoinCheckerFactory mjcf, 
IRangeMap rangeMap) {
+int memSizeInFrames, long buildTupleCount, long probeTupleCount, 
long buildMaxDuration,
+long probeMaxDuration, int avgTuplesInFrame, 
IIntervalMergeJoinCheckerFactory mjcf, IRangeMap rangeMap) {
 super(kind, partitioningType, sideLeftOfEqualities, 
sideRightOfEqualities, mjcf, rangeMap);
 this.memSizeInFrames = memSizeInFrames;
 this.buildTupleCount = buildTupleCount;
@@ -62,19 +62,19 @@ public class IntervalPartitionJoinPOperator extends 
AbstractIntervalJoinPOperato
 + ".");
 }
 
-public int getProbeTupleCount() {
+public long getProbeTupleCount() {
 return probeTupleCount;
 }
 
-public int getProbeMaxDuration() {
+public long getProbeMaxDuration() {
 return probeMaxDuration;
 }
 
-public int getBuildTupleCount() {
+public long getBuildTupleCount() {
 return buildTupleCount;
 }
 
-public int getBuildMaxDuration() {
+public long getBuildMaxDuration() {
 return buildMaxDuration;
 }
 

http://git-wip-us.apache.org/repos/asf/asterixdb/blob/fd84e345/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntervalSplitPartitioningRule.java
--
diff --git

[50/50] [abbrv] asterixdb git commit: merge clean up

2016-10-17 Thread prestonc

merge clean up


Project: http://git-wip-us.apache.org/repos/asf/asterixdb/repo
Commit: http://git-wip-us.apache.org/repos/asf/asterixdb/commit/faa3bb45
Tree: http://git-wip-us.apache.org/repos/asf/asterixdb/tree/faa3bb45
Diff: http://git-wip-us.apache.org/repos/asf/asterixdb/diff/faa3bb45

Branch: refs/heads/ecarm002/interval_join_merge
Commit: faa3bb457362215393e97e44c69b05109950f33a
Parents: 9baab02
Author: Preston Carman 
Authored: Mon Oct 17 12:53:38 2016 -0700
Committer: Preston Carman 
Committed: Mon Oct 17 12:53:38 2016 -0700

--
 .../logical/visitors/FDsAndEquivClassesVisitor.java | 12 ++--
 .../logical/visitors/LogicalPropertiesVisitor.java  |  8 
 .../logical/visitors/SubstituteVariableVisitor.java | 10 --
 .../prettyprint/LogicalOperatorPrettyPrintVisitor.java  |  3 +--
 hyracks-fullstack/algebricks/algebricks-data/pom.xml|  5 -
 .../algebricks/algebricks-rewriter/pom.xml  | 12 +---
 .../hyracks-examples/hyracks-integration-tests/pom.xml  |  8 +++-
 7 files changed, 23 insertions(+), 35 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/asterixdb/blob/faa3bb45/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/FDsAndEquivClassesVisitor.java
--
diff --git 
a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/FDsAndEquivClassesVisitor.java
 
b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/FDsAndEquivClassesVisitor.java
index 2bb43c9..0eaf1b3 100644
--- 
a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/FDsAndEquivClassesVisitor.java
+++ 
b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/FDsAndEquivClassesVisitor.java
@@ -405,12 +405,6 @@ public class FDsAndEquivClassesVisitor implements 
ILogicalOperatorVisitor

[2/2] asterixdb git commit: Sonar fixes and other clean up items.

[1/2] asterixdb git commit: Interval join test questions for AQL.

[26/50] [abbrv] asterixdb git commit: Add stats tracking for interval joins.

[03/50] [abbrv] asterixdb git commit: snapshot super interval dag.

[10/50] [abbrv] asterixdb git commit: switch merge memory tracking method.

[19/50] [abbrv] asterixdb git commit: clean up.

[07/50] [abbrv] asterixdb git commit: new annontations.

[44/50] [abbrv] asterixdb git commit: new partition join algorithm

[43/50] [abbrv] asterixdb git commit: new partition join algorithm

[47/50] [abbrv] asterixdb git commit: merge master with interval tests

[12/50] [abbrv] asterixdb git commit: updated from api change.

[25/50] [abbrv] asterixdb git commit: Working version of the range connector and interval join partition.

[09/50] [abbrv] asterixdb git commit: merge with master

[49/50] [abbrv] asterixdb git commit: merge master

[18/50] [abbrv] asterixdb git commit: Range forward operators.

[34/50] [abbrv] asterixdb git commit: merge master

[04/50] [abbrv] asterixdb git commit: working mega overlapping plan.

[08/50] [abbrv] asterixdb git commit: merge with master

[17/50] [abbrv] asterixdb git commit: More clean up form interval prep.

[35/50] [abbrv] asterixdb git commit: snapshot - reuse frames in deletable partition manager.

[15/50] [abbrv] asterixdb git commit: merge master with all interval prep clean up items.

[20/50] [abbrv] asterixdb git commit: snapshot for range state transition

[11/50] [abbrv] asterixdb git commit: merge master

[30/50] [abbrv] asterixdb git commit: merge master

[01/50] [abbrv] asterixdb git commit: fix for indexing memory error.

[28/50] [abbrv] asterixdb git commit: snapshot with logging

[42/50] [abbrv] asterixdb git commit: working partition join.

[29/50] [abbrv] asterixdb git commit: ASTERIXDB-1635 Fix for overlap-bins start for dates.

[05/50] [abbrv] asterixdb git commit: added more to the interval hint

[50/50] [abbrv] asterixdb git commit: merge clean up

30 matches

Site Navigation

Mail list logo

Footer information