Change in asterixdb[master]: ASTERIXDB-1581: fix subquery decorrelation.

2016-08-28 Thread Jenkins (Code Review) 
Jenkins has posted comments on this change.

Change subject: ASTERIXDB-1581: fix subquery decorrelation.
..


Patch Set 1:

WARNING: THIS CHANGE CONTAINS CROSS-PRODUCT CHANGES IN:
* asterixdb
* hyracks-fullstack

PLEASE REVIEW CAREFULLY AND LOOK FOR API CHANGES!

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1125
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia2fa4b5b836eafee1975bd1164ae7c22199a4af0
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Yingyi Bu 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1611: reasonable names for threads

2016-08-28 Thread Michael Blow (Code Review) 
Michael Blow has posted comments on this change.

Change subject: ASTERIXDB-1611: reasonable names for threads
..


Patch Set 7: Code-Review+1

Looks good, but need validation that blocking on the recovery thread is the 
correct fix.

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1092
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I3fea28c9e0ea7c3baad1c7a053adabc71bea2a8a
Gerrit-PatchSet: 7
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Till Westmann 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1581: fix subquery decorrelation.

2016-08-28 Thread Jenkins (Code Review) 
Jenkins has posted comments on this change.

Change subject: ASTERIXDB-1581: fix subquery decorrelation.
..


Patch Set 1:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/2430/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1125
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ia2fa4b5b836eafee1975bd1164ae7c22199a4af0
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Yingyi Bu 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1581: fix subquery decorrelation.

2016-08-28 Thread Yingyi Bu (Code Review) 
Yingyi Bu has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/1125

Change subject: ASTERIXDB-1581: fix subquery decorrelation.
..

ASTERIXDB-1581: fix subquery decorrelation.

Change-Id: Ia2fa4b5b836eafee1975bd1164ae7c22199a4af0
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineAllNtsInSubplanVisitor.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineSubplanInputForNestedTupleSourceRule.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/SubplanFlatteningUtil.java
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/udfs/query-ASTERIXDB-1018.plan
M 
asterixdb/asterix-app/src/test/resources/optimizerts/results/udfs/query-ASTERIXDB-1019.plan
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/tpcds/query-ASTERIXDB-1581-2/query-ASTERIXDB-1581-2.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/tpcds/query-ASTERIXDB-1581-2/query-ASTERIXDB-1581-2.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/tpcds/query-ASTERIXDB-1581-2/query-ASTERIXDB-1581-2.3.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/tpcds/query-ASTERIXDB-1581-3/query-ASTERIXDB-1581-3.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/tpcds/query-ASTERIXDB-1581-3/query-ASTERIXDB-1581-3.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/tpcds/query-ASTERIXDB-1581-3/query-ASTERIXDB-1581-3.3.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/tpcds/query-ASTERIXDB-1581-4/query-ASTERIXDB-1581-4.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/tpcds/query-ASTERIXDB-1581-4/query-ASTERIXDB-1581-4.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/tpcds/query-ASTERIXDB-1581-4/query-ASTERIXDB-1581-4.3.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/tpcds/query-ASTERIXDB-1581/query-ASTERIXDB-1581.1.ddl.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/tpcds/query-ASTERIXDB-1581/query-ASTERIXDB-1581.2.update.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/tpcds/query-ASTERIXDB-1581/query-ASTERIXDB-1581.3.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/tpcds/query-ASTERIXDB-1581-2/query-ASTERIXDB-1581-2.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/tpcds/query-ASTERIXDB-1581-4/query-ASTERIXDB-1581-4.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/tpcds/query-ASTERIXDB-1581/query-ASTERIXDB-1581.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M 
hyracks-fullstack/algebricks/algebricks-rewriter/src/main/java/org/apache/hyracks/algebricks/rewriter/rules/EnforceStructuralPropertiesRule.java
22 files changed, 734 insertions(+), 132 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/25/1125/1

diff --git 
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineAllNtsInSubplanVisitor.java
 
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineAllNtsInSubplanVisitor.java
index b061066..3139d6e 100644
--- 
a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineAllNtsInSubplanVisitor.java
+++ 
b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/subplan/InlineAllNtsInSubplanVisitor.java
@@ -25,8 +25,8 @@
 import java.util.Iterator;
 import java.util.List;
 import java.util.Map;
-import java.util.Map.Entry;
 import java.util.Set;
+import java.util.Map.Entry;
 
 import org.apache.asterix.lang.common.util.FunctionUtil;
 import org.apache.asterix.om.base.AString;
@@ -66,7 +66,6 @@
 import 
org.apache.hyracks.algebricks.core.algebra.operators.logical.MaterializeOperator;
 import 
org.apache.hyracks.algebricks.core.algebra.operators.logical.NestedTupleSourceOperator;
 import 
org.apache.hyracks.algebricks.core.algebra.operators.logical.OrderOperator;
-import 
org.apache.hyracks.algebricks.core.algebra.operators.logical.OrderOperator.IOrder;
 import 
org.apache.hyracks.algebricks.core.algebra.operators.logical.PartitioningSplitOperator;
 import 
org.apache.hyracks.algebricks.core.algebra.operators.logical.ProjectOperator;
 import 
org.apache.hyracks.algebricks.core.algebra.operators.logical.ReplicateOperator;
@@ -78,6 +77,7 @@
 import 
org.apache.hyracks.algebricks.core.algebra.operators.logical.UnionAllOperator;
 import 
org.apache.hyracks.algebricks.core.algebra.operators.logical.UnnestMapOperator;
 import 
org.apache.hyracks.algebricks.core.algebra.operators.logical.UnnestOperator;
+import

Change in asterixdb[master]: Fix Numeric Overflow For Units > GB

2016-08-28 Thread Michael Blow (Code Review) 
Michael Blow has submitted this change and it was merged.

Change subject: Fix Numeric Overflow For Units > GB
..


Fix Numeric Overflow For Units > GB

refactoring / cleanup
handle negative numbers

Change-Id: Id5a3896ebfb38fc565b3524beed940dfbbf81d4f
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1084
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Taewoo Kim 
---
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/PropertyInterpreters.java
M 
hyracks-fullstack/hyracks/hyracks-util/src/main/java/org/apache/hyracks/util/StorageUtil.java
2 files changed, 69 insertions(+), 145 deletions(-)

Approvals:
  Taewoo Kim: Looks good to me, approved
  Jenkins: Verified; No violations found; Verified



diff --git 
a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/PropertyInterpreters.java
 
b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/PropertyInterpreters.java
index 4dd07d8..6703fcd 100644
--- 
a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/PropertyInterpreters.java
+++ 
b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/PropertyInterpreters.java
@@ -25,93 +25,41 @@
 public class PropertyInterpreters {
 
 public static IPropertyInterpreter 
getIntegerPropertyInterpreter() {
-return new IPropertyInterpreter() {
-@Override
-public Integer interpret(String s) throws IllegalArgumentException 
{
-try {
-return Integer.parseInt(s);
-} catch (NumberFormatException e) {
-throw new IllegalArgumentException(e);
-}
-}
-};
+return Integer::parseInt;
 }
 
 public static IPropertyInterpreter 
getBooleanPropertyInterpreter() {
-return new IPropertyInterpreter() {
-@Override
-public Boolean interpret(String s) throws IllegalArgumentException 
{
-return Boolean.parseBoolean(s);
-}
-};
+return Boolean::parseBoolean;
 }
 
 public static IPropertyInterpreter getLongPropertyInterpreter() {
-return new IPropertyInterpreter() {
-@Override
-public Long interpret(String s) throws IllegalArgumentException {
-try {
-return Long.parseLong(s);
-} catch (NumberFormatException e) {
-throw new IllegalArgumentException(e);
-}
-}
-};
+return Long::parseLong;
 }
 
 public static IPropertyInterpreter getLevelPropertyInterpreter() {
-return new IPropertyInterpreter() {
-@Override
-public Level interpret(String s) throws IllegalArgumentException {
-return Level.parse(s);
-}
-};
+return Level::parse;
 }
 
 public static IPropertyInterpreter getStringPropertyInterpreter() {
-return new IPropertyInterpreter() {
-@Override
-public String interpret(String s) throws IllegalArgumentException {
-return s;
-}
-};
+return s -> s;
 }
 
 public static IPropertyInterpreter getDoublePropertyInterpreter() {
-return new IPropertyInterpreter() {
-@Override
-public Double interpret(String s) throws IllegalArgumentException {
-try {
-return Double.parseDouble(s);
-} catch (NumberFormatException e) {
-throw new IllegalArgumentException(e);
-}
-}
-};
+return Double::parseDouble;
 }
 
 public static IPropertyInterpreter getLongBytePropertyInterpreter() {
-return new IPropertyInterpreter() {
-@Override
-public Long interpret(String s) throws IllegalArgumentException {
-try {
-return StorageUtil.getByteValue(s);
-} catch (NumberFormatException e) {
-throw new IllegalArgumentException(e);
-}
-}
-};
+return StorageUtil::getByteValue;
 }
 
 public static IPropertyInterpreter 
getIntegerBytePropertyInterpreter() {
-return new IPropertyInterpreter() {
-@Override
-public Integer interpret(String s) throws IllegalArgumentException 
{
-try {
-return (int) StorageUtil.getByteValue(s);
-} catch (NumberFormatException e) {
-throw new IllegalArgumentException(e);
-}
+return s -> {
+long result = StorageUtil.getByteValue(s);
+

Change in asterixdb[master]: Applied the multiway fuzzyjoin based on the prefix-based joi...

2016-08-28 Thread Chen Li (Code Review) 
Chen Li has posted comments on this change.

Change subject: Applied the multiway fuzzyjoin based on the prefix-based join 
and the selectFuzzyJoin testCases.
..


Patch Set 4:

(1 comment)

https://asterix-gerrit.ics.uci.edu/#/c/1076/4/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/AsterixExtractFunctionsFromJoinConditionRule.java
File 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/AsterixExtractFunctionsFromJoinConditionRule.java:

Line 63: private boolean assignSimilarityPK(AbstractLogicalOperator joinOp, 
AbstractFunctionCallExpression fexp,
A general comment: to make the code much more readable, it will be much better 
to use a running example to explain the logic using the example.  People are 
good at doing generalization from examples.


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1076
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I8736f104905eeda763d39709e002c2b9629278cc
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Wenhai Li 
Gerrit-Reviewer: Chen Li 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Wenhai Li 
Gerrit-HasComments: Yes

[jira] [Commented] (ASTERIXDB-1614) DatasetDirectoryService.deinitState() fix intermittently fails

2016-08-28 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/ASTERIXDB-1614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15444381#comment-15444381
 ] 

ASF subversion and git services commented on ASTERIXDB-1614:


Commit 2302d875f2f6e00db4d93134373806a7bfa3033e in asterixdb's branch 
refs/heads/master from [~mblow]
[ https://git-wip-us.apache.org/repos/asf?p=asterixdb.git;h=2302d87 ]

Workaround ASTERIXDB-1614

Change-Id: I6d016b680c0991c50e1a82e27f51dfcfeb2f0d97
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1124
Tested-by: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Till Westmann 


> DatasetDirectoryService.deinitState() fix intermittently fails
> --
>
> Key: ASTERIXDB-1614
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-1614
> Project: Apache AsterixDB
>  Issue Type: Bug
>  Components: Hyracks
>Reporter: Michael Blow
>
> After merging a fix for type-correctness in 
> DatasetDirectoryService.deinitState(), tests started intermittently failing.  
> e.g. 
> https://asterix-jenkins.ics.uci.edu/view/All/job/asterix-integration-tests/208/
> java.lang.NullPointerException
>   at 
> org.apache.hyracks.control.cc.dataset.DatasetDirectoryService.registerResultPartitionLocation(DatasetDirectoryService.java:98)
>   at 
> org.apache.hyracks.control.cc.work.RegisterResultPartitionLocationWork.run(RegisterResultPartitionLocationWork.java:58)
>   at 
> org.apache.hyracks.control.common.work.WorkQueue$WorkerThread.run(WorkQueue.java:133)
> java.lang.NullPointerException
>   at 
> org.apache.hyracks.control.cc.dataset.DatasetDirectoryService.reportResultPartitionWriteCompletion(DatasetDirectoryService.java:133)
>   at 
> org.apache.hyracks.control.cc.work.ReportResultPartitionWriteCompletionWork.run(ReportResultPartitionWriteCompletionWork.java:45)
>   at 
> org.apache.hyracks.control.common.work.WorkQueue$WorkerThread.run(WorkQueue.java:133)
> org.apache.hyracks.api.exceptions.HyracksDataException: Requested JobId 
> JID:1968 doesn't exist
>   at 
> org.apache.hyracks.control.cc.dataset.DatasetDirectoryService.updatedRecords(DatasetDirectoryService.java:253)
>   at 
> org.apache.hyracks.control.cc.dataset.DatasetDirectoryService.getResultPartitionLocations(DatasetDirectoryService.java:210)
>   at 
> org.apache.hyracks.control.cc.work.GetResultPartitionLocationsWork$1.run(GetResultPartitionLocationsWork.java:59)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Change in asterixdb[master]: Workaround ASTERIXDB-1614

2016-08-28 Thread Michael Blow (Code Review) 
Michael Blow has submitted this change and it was merged.

Change subject: Workaround ASTERIXDB-1614
..


Workaround ASTERIXDB-1614

Change-Id: I6d016b680c0991c50e1a82e27f51dfcfeb2f0d97
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1124
Tested-by: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Till Westmann 
---
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/dataset/DatasetDirectoryService.java
1 file changed, 2 insertions(+), 1 deletion(-)

Approvals:
  Till Westmann: Looks good to me, approved
  Jenkins: Verified; Verified

Objections:
  Jenkins: Violations found



diff --git 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/dataset/DatasetDirectoryService.java
 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/dataset/DatasetDirectoryService.java
index c3c41a4..4d7d1c3 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/dataset/DatasetDirectoryService.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/dataset/DatasetDirectoryService.java
@@ -200,7 +200,8 @@
 
 @Override
 public void deinitState(JobId jobId) {
-jobResultLocations.remove(jobId);
+// See ASTERIXDB-1614 - DatasetDirectoryService.deinitState() fix 
intermittently fails
+// jobResultLocations.remove(jobId);
 }
 
 @Override

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1124
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I6d016b680c0991c50e1a82e27f51dfcfeb2f0d97
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Michael Blow 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann

Change in asterixdb[master]: Workaround ASTERIXDB-1614

2016-08-28 Thread Till Westmann (Code Review) 
Till Westmann has posted comments on this change.

Change subject: Workaround ASTERIXDB-1614
..


Patch Set 1: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1124
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I6d016b680c0991c50e1a82e27f51dfcfeb2f0d97
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Michael Blow 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-HasComments: No

Change in asterixdb[master]: Fix Numeric Overflow For Units > GB

2016-08-28 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: Fix Numeric Overflow For Units > GB
..


Patch Set 3: Code-Review+2

LGTM

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1084
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id5a3896ebfb38fc565b3524beed940dfbbf81d4f
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Michael Blow 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Till Westmann 
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1088: fixed a secondary index compact issue

2016-08-28 Thread Taewoo Kim (Code Review) 
Taewoo Kim has submitted this change and it was merged.

Change subject: ASTERIXDB-1088: fixed a secondary index compact issue
..


ASTERIXDB-1088: fixed a secondary index compact issue

 - Secondary index is now properly compacted

Change-Id: Ide9191fce61b767c08d2298070e64d455ee34f0b
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1123
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Taewoo Kim 
---
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
1 file changed, 8 insertions(+), 3 deletions(-)

Approvals:
  Taewoo Kim: Looks good to me, approved
  Jenkins: Verified; No violations found; Verified



diff --git 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
index 24c678c..6390b9d 100644
--- 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
+++ 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
@@ -2487,9 +2487,14 @@
 if (ds.getDatasetType() == DatasetType.INTERNAL) {
 for (int j = 0; j < indexes.size(); j++) {
 if (indexes.get(j).isSecondaryIndex()) {
-jobsToExecute
-.add(DatasetOperations.compactDatasetJobSpec(
-dataverse, datasetName, 
metadataProvider));
+CompiledIndexCompactStatement cics = new 
CompiledIndexCompactStatement(dataverseName,
+datasetName, indexes.get(j).getIndexName(), 
indexes.get(j).getKeyFieldNames(),
+indexes.get(j).getKeyFieldTypes(), 
indexes.get(j).isEnforcingKeyFileds(),
+indexes.get(j).getGramLength(), 
indexes.get(j).getIndexType());
+List keySourceIndicators = 
indexes.get(j).getKeyFieldSourceIndicators();
+
+
jobsToExecute.add(IndexOperations.buildSecondaryIndexCompactJobSpec(cics, 
aRecordType,
+metaRecordType, keySourceIndicators, 
enforcedType, metadataProvider));
 }
 }
 } else {

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1123
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Ide9191fce61b767c08d2298070e64d455ee34f0b
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Young-Seok Kim

Change in asterixdb[master]: ASTERIXDB-1088: fixed a secondary index compact issue

2016-08-28 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: ASTERIXDB-1088: fixed a secondary index compact issue
..


Patch Set 2: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1123
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ide9191fce61b767c08d2298070e64d455ee34f0b
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Young-Seok Kim 
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1088: fixed a secondary index compact issue

2016-08-28 Thread Taewoo Kim (Code Review) 
Taewoo Kim has posted comments on this change.

Change subject: ASTERIXDB-1088: fixed a secondary index compact issue
..


Patch Set 2:

Forwarding Young-Seok's +2 (just rebased)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1123
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ide9191fce61b767c08d2298070e64d455ee34f0b
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Taewoo Kim 
Gerrit-Reviewer: Young-Seok Kim 
Gerrit-HasComments: No

Change in asterixdb[master]: Workaround ASTERIXDB-1614

2016-08-28 Thread Jenkins (Code Review) 
Jenkins has posted comments on this change.

Change subject: Workaround ASTERIXDB-1614
..


Patch Set 1: Integration-Tests+1

Integration Tests Successful

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/490/ : 
SUCCESS

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1124
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I6d016b680c0991c50e1a82e27f51dfcfeb2f0d97
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Michael Blow 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1088: fixed a secondary index compact issue

2016-08-28 Thread Jenkins (Code Review) 
Jenkins has posted comments on this change.

Change subject: ASTERIXDB-1088: fixed a secondary index compact issue
..


Patch Set 2: Integration-Tests+1

Integration Tests Successful

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/489/ : 
SUCCESS

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1123
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ide9191fce61b767c08d2298070e64d455ee34f0b
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Young-Seok Kim 
Gerrit-HasComments: No

Change in asterixdb[master]: Workaround ASTERIXDB-1614

2016-08-28 Thread Jenkins (Code Review) 
Jenkins has posted comments on this change.

Change subject: Workaround ASTERIXDB-1614
..


Patch Set 1:

Integration Tests Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/490/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1124
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I6d016b680c0991c50e1a82e27f51dfcfeb2f0d97
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Michael Blow 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1088: fixed a secondary index compact issue

2016-08-28 Thread Jenkins (Code Review) 
Jenkins has posted comments on this change.

Change subject: ASTERIXDB-1088: fixed a secondary index compact issue
..


Patch Set 2:

Integration Tests Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/489/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1123
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ide9191fce61b767c08d2298070e64d455ee34f0b
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Young-Seok Kim 
Gerrit-HasComments: No

Change in asterixdb[master]: Workaround ASTERIXDB-1614

2016-08-28 Thread Jenkins (Code Review) 
Jenkins has posted comments on this change.

Change subject: Workaround ASTERIXDB-1614
..


Patch Set 1:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/2429/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1124
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I6d016b680c0991c50e1a82e27f51dfcfeb2f0d97
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Michael Blow 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-HasComments: No

Change in asterixdb[master]: Workaround ASTERIXDB-1614

2016-08-28 Thread Michael Blow (Code Review) 
Michael Blow has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/1124

Change subject: Workaround ASTERIXDB-1614
..

Workaround ASTERIXDB-1614

Change-Id: I6d016b680c0991c50e1a82e27f51dfcfeb2f0d97
---
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/dataset/DatasetDirectoryService.java
1 file changed, 2 insertions(+), 1 deletion(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/24/1124/1

diff --git 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/dataset/DatasetDirectoryService.java
 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/dataset/DatasetDirectoryService.java
index c3c41a4..4d7d1c3 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/dataset/DatasetDirectoryService.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/dataset/DatasetDirectoryService.java
@@ -200,7 +200,8 @@
 
 @Override
 public void deinitState(JobId jobId) {
-jobResultLocations.remove(jobId);
+// See ASTERIXDB-1614 - DatasetDirectoryService.deinitState() fix 
intermittently fails
+// jobResultLocations.remove(jobId);
 }
 
 @Override

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1124
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I6d016b680c0991c50e1a82e27f51dfcfeb2f0d97
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Michael Blow

[jira] [Created] (ASTERIXDB-1614) DatasetDirectoryService.deinitState() fix intermittently fails

2016-08-28 Thread Michael Blow (JIRA) 
Michael Blow created ASTERIXDB-1614:
---

 Summary: DatasetDirectoryService.deinitState() fix intermittently 
fails
 Key: ASTERIXDB-1614
 URL: https://issues.apache.org/jira/browse/ASTERIXDB-1614
 Project: Apache AsterixDB
  Issue Type: Bug
  Components: Hyracks
Reporter: Michael Blow


After merging a fix for type-correctness in 
DatasetDirectoryService.deinitState(), tests started intermittently failing.  

e.g. 
https://asterix-jenkins.ics.uci.edu/view/All/job/asterix-integration-tests/208/

java.lang.NullPointerException
at 
org.apache.hyracks.control.cc.dataset.DatasetDirectoryService.registerResultPartitionLocation(DatasetDirectoryService.java:98)
at 
org.apache.hyracks.control.cc.work.RegisterResultPartitionLocationWork.run(RegisterResultPartitionLocationWork.java:58)
at 
org.apache.hyracks.control.common.work.WorkQueue$WorkerThread.run(WorkQueue.java:133)
java.lang.NullPointerException
at 
org.apache.hyracks.control.cc.dataset.DatasetDirectoryService.reportResultPartitionWriteCompletion(DatasetDirectoryService.java:133)
at 
org.apache.hyracks.control.cc.work.ReportResultPartitionWriteCompletionWork.run(ReportResultPartitionWriteCompletionWork.java:45)
at 
org.apache.hyracks.control.common.work.WorkQueue$WorkerThread.run(WorkQueue.java:133)
org.apache.hyracks.api.exceptions.HyracksDataException: Requested JobId 
JID:1968 doesn't exist
at 
org.apache.hyracks.control.cc.dataset.DatasetDirectoryService.updatedRecords(DatasetDirectoryService.java:253)
at 
org.apache.hyracks.control.cc.dataset.DatasetDirectoryService.getResultPartitionLocations(DatasetDirectoryService.java:210)
at 
org.apache.hyracks.control.cc.work.GetResultPartitionLocationsWork$1.run(GetResultPartitionLocationsWork.java:59)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Change in asterixdb[master]: ASTERIXDB-1088: fixed a secondary index compact issue

2016-08-28 Thread Jenkins (Code Review) 
Jenkins has posted comments on this change.

Change subject: ASTERIXDB-1088: fixed a secondary index compact issue
..


Patch Set 2:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/2428/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1123
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ide9191fce61b767c08d2298070e64d455ee34f0b
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Young-Seok Kim 
Gerrit-HasComments: No

Change in asterixdb[master]: Add upsert option for feed

2016-08-28 Thread Jenkins (Code Review) 
Jenkins has posted comments on this change.

Change subject: Add upsert option for feed
..


Patch Set 10: Integration-Tests-1

Integration Tests Timed Out

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/488/ : 
ABORTED

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1068
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ic5133e7c6941fea4110cc9983f99502f364dc810
Gerrit-PatchSet: 10
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1088: fixed a secondary index compact issue

2016-08-28 Thread Jenkins (Code Review) 
Jenkins has posted comments on this change.

Change subject: ASTERIXDB-1088: fixed a secondary index compact issue
..


Patch Set 2:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/2427/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1123
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ide9191fce61b767c08d2298070e64d455ee34f0b
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Young-Seok Kim 
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1088: fixed a secondary index compact issue

2016-08-28 Thread Taewoo Kim (Code Review) 
Hello Young-Seok Kim, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1123

to look at the new patch set (#2).

Change subject: ASTERIXDB-1088: fixed a secondary index compact issue
..

ASTERIXDB-1088: fixed a secondary index compact issue

 - Secondary index is now properly compacted

Change-Id: Ide9191fce61b767c08d2298070e64d455ee34f0b
---
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
1 file changed, 8 insertions(+), 3 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/23/1123/2
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1123
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ide9191fce61b767c08d2298070e64d455ee34f0b
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Young-Seok Kim

Change in asterixdb[master]: Add upsert option for feed

2016-08-28 Thread Jenkins (Code Review) 
Jenkins has posted comments on this change.

Change subject: Add upsert option for feed
..


Patch Set 10:

Integration Tests Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/488/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1068
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ic5133e7c6941fea4110cc9983f99502f364dc810
Gerrit-PatchSet: 10
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: Add upsert option for feed

2016-08-28 Thread Xikui Wang (Code Review) 
Hello Till Westmann, Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1068

to look at the new patch set (#10).

Change subject: Add upsert option for feed
..

Add upsert option for feed

For ASTERIXDB-1567. Provide "upsert-feed"="true" for
feed configuration, which changes the default record
insert to upsert. Added one test case for upsert feed.

Change-Id: Ic5133e7c6941fea4110cc9983f99502f364dc810
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/LangExpressionToPlanTranslator.java
A asterixdb/asterix-app/data/tinysocial/twu_update.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/upsert-feed/upsert-feed.1.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/upsert-feed/upsert-feed.2.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/upsert-feed/upsert-feed.3.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/upsert-feed/upsert-feed.4.sleep.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/upsert-feed/upsert-feed.5.update.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/upsert-feed/upsert-feed.6.server.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/upsert-feed/upsert-feed.7.query.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries/feeds/upsert-feed/upsert-feed.8.ddl.aql
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/feeds/upsert-feed/upsert-feed.1.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataConstants.java
M 
asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/util/ExternalDataUtils.java
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/feeds/FeedMetadataUtil.java
15 files changed, 275 insertions(+), 12 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/68/1068/10
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1068
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic5133e7c6941fea4110cc9983f99502f364dc810
Gerrit-PatchSet: 10
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-Reviewer: abdullah alamoudi

Change in asterixdb[master]: Add upsert option for feed

2016-08-28 Thread Jenkins (Code Review) 
Jenkins has posted comments on this change.

Change subject: Add upsert option for feed
..


Patch Set 10:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/2426/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1068
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ic5133e7c6941fea4110cc9983f99502f364dc810
Gerrit-PatchSet: 10
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Xikui Wang 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-Reviewer: abdullah alamoudi 
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1088: fixed a secondary index compact issue

2016-08-28 Thread Young-Seok Kim (Code Review) 
Young-Seok Kim has posted comments on this change.

Change subject: ASTERIXDB-1088: fixed a secondary index compact issue
..


Patch Set 1: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1123
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ide9191fce61b767c08d2298070e64d455ee34f0b
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Young-Seok Kim 
Gerrit-HasComments: No

Change in asterixdb[master]: ASTERIXDB-1088: fixed a secondary index compact issue

2016-08-28 Thread Jenkins (Code Review) 
Jenkins has posted comments on this change.

Change subject: ASTERIXDB-1088: fixed a secondary index compact issue
..


Patch Set 1: Integration-Tests+1

Integration Tests Successful

https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-integration-tests/487/ : 
SUCCESS

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1123
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ide9191fce61b767c08d2298070e64d455ee34f0b
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Taewoo Kim 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No