Change in asterixdb[master]: 2 small fixes for the transition to jackson

[Chris Hillery (Code Review)]

Chris Hillery has posted comments on this change.

Change subject: 2 small fixes for the transition to jackson
..


Patch Set 1: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1423
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I39279a6a9bf8bffcf4c426b47610479030a53b42
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Till Westmann 
Gerrit-Reviewer: Chris Hillery 
Gerrit-Reviewer: Christopher Hillery 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: Add "select *" perf test case

[Chris Hillery (Code Review)]

Chris Hillery has submitted this change and it was merged.

Change subject: Add "select *" perf test case
..


Add "select *" perf test case

Change-Id: I392d959b50465db6873a5d8e513ef5bbb44bdf59
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1187
Reviewed-by: Ian Maxon 
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Integration-Tests: Jenkins 
---
A 
asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/select-all.sqlpp
1 file changed, 21 insertions(+), 0 deletions(-)

Approvals:
  Ian Maxon: Looks good to me, approved
  Jenkins: Verified; No violations found; Verified



diff --git 
a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/select-all.sqlpp
 
b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/select-all.sqlpp
new file mode 100644
index 000..29f104f
--- /dev/null
+++ 
b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/select-all.sqlpp
@@ -0,0 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+use SocialNetworkData;
+select * from GleambookMessages;

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1187
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I392d959b50465db6873a5d8e513ef5bbb44bdf59
Gerrit-PatchSet: 5
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Chris Hillery 
Gerrit-Reviewer: Chris Hillery 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 

Change in asterixdb[master]: Test, ignore

[Chris Hillery (Code Review)]

Chris Hillery has abandoned this change.

Change subject: Test, ignore
..


Abandoned

Was for testing only

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1190
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: I392d959b50465db6873a5d8e513ef5bbb44bdf58
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Chris Hillery 
Gerrit-Reviewer: Jenkins 

Change in asterixdb[master]: Add "select *" perf test case

[Chris Hillery (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1187

to look at the new patch set (#4).

Change subject: Add "select *" perf test case
..

Add "select *" perf test case

Change-Id: I392d959b50465db6873a5d8e513ef5bbb44bdf59
---
A 
asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/select-all.sqlpp
1 file changed, 21 insertions(+), 0 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/87/1187/4
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1187
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I392d959b50465db6873a5d8e513ef5bbb44bdf59
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Chris Hillery 
Gerrit-Reviewer: Jenkins 

Change in asterixdb[master]: Test, ignore

[Chris Hillery (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1190

to look at the new patch set (#2).

Change subject: Test, ignore
..

Test, ignore

Change-Id: I392d959b50465db6873a5d8e513ef5bbb44bdf58
---
M README.md
1 file changed, 2 insertions(+), 0 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/90/1190/2
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1190
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I392d959b50465db6873a5d8e513ef5bbb44bdf58
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Chris Hillery 
Gerrit-Reviewer: Jenkins 

Change in asterixdb[master]: Test, ignore

[Chris Hillery (Code Review)]

Chris Hillery has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/1190

Change subject: Test, ignore
..

Test, ignore

Change-Id: I392d959b50465db6873a5d8e513ef5bbb44bdf58
---
M README.md
1 file changed, 2 insertions(+), 0 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/90/1190/1

diff --git a/README.md b/README.md
index f522fc6..cdb4283 100644
--- a/README.md
+++ b/README.md
@@ -18,6 +18,8 @@
  !-->
 #AsterixDB
 
+Gerrit?
+
 AsterixDB is a BDMS (Big Data Management System) with a rich feature set that 
sets it apart from other Big Data platforms.  Its feature set makes it 
well-suited to modern needs such as web data warehousing and social data 
storage and analysis. AsterixDB has:
 
  * A semistructured NoSQL style data model (ADM) resulting from extending JSON 
with object database ideas

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1190
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I392d959b50465db6873a5d8e513ef5bbb44bdf58
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Chris Hillery 

Change in asterixdb[master]: Add "select *" perf test case

[Chris Hillery (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1187

to look at the new patch set (#3).

Change subject: Add "select *" perf test case
..

Add "select *" perf test case

Change-Id: I392d959b50465db6873a5d8e513ef5bbb44bdf59
---
A 
asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/select-all.sqlpp
1 file changed, 21 insertions(+), 0 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/87/1187/3
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1187
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I392d959b50465db6873a5d8e513ef5bbb44bdf59
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Chris Hillery 
Gerrit-Reviewer: Jenkins 

Change in asterixdb[master]: Add "select *" perf test case

[Chris Hillery (Code Review)]

Chris Hillery has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/1187

Change subject: Add "select *" perf test case
..

Add "select *" perf test case

Change-Id: I392d959b50465db6873a5d8e513ef5bbb44bdf59
---
A 
asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/select-all.sqlpp
1 file changed, 2 insertions(+), 0 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/87/1187/1

diff --git 
a/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/select-all.sqlpp
 
b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/select-all.sqlpp
new file mode 100644
index 000..47e1167
--- /dev/null
+++ 
b/asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/agg_bench/select-all.sqlpp
@@ -0,0 +1,2 @@
+use SocialNetworkData;
+select * from GleambookMessages;

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1187
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I392d959b50465db6873a5d8e513ef5bbb44bdf59
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Chris Hillery 

Change in asterixdb[master]: Depend on asterix-opt-bom with same version as AsterixDB itself

[Chris Hillery (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/1137

to look at the new patch set (#2).

Change subject: Depend on asterix-opt-bom with same version as AsterixDB itself
..

Depend on asterix-opt-bom with same version as AsterixDB itself

Change-Id: Ifd5b95bc052c87419d0e925e46f7741b97cc1132
---
M asterixdb/asterix-server/pom.xml
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/37/1137/2
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1137
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ifd5b95bc052c87419d0e925e46f7741b97cc1132
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Chris Hillery 
Gerrit-Reviewer: Chris Hillery 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 

Change in asterixdb[master]: Depend on asterix-opt-bom 1.0.0-SNAPSHOT

[Chris Hillery (Code Review)]

Chris Hillery has posted comments on this change.

Change subject: Depend on asterix-opt-bom 1.0.0-SNAPSHOT
..


Patch Set 1:

(1 comment)

https://asterix-gerrit.ics.uci.edu/#/c/1137/1/asterixdb/asterix-server/pom.xml
File asterixdb/asterix-server/pom.xml:

Line 179:   1.0.0-SNAPSHOT
> I think this should match the asterixdb version (i.e. ${project.version})
It can be anything we want. I'd almost prefer for it to be a never-changing 
constant, and 1.0.0 seems like a reasonable constant to pick. But if you think 
it makes sense to use the asterixdb version, that's fine too I guess. It means 
we need to update asterix-opt whenever we bump the asterixdb version, but I 
suppose we'll need to anyway.


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1137
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ifd5b95bc052c87419d0e925e46f7741b97cc1132
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Chris Hillery 
Gerrit-Reviewer: Chris Hillery 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Depend on asterix-opt-bom 1.0.0-SNAPSHOT

[Chris Hillery (Code Review)]

Chris Hillery has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/1137

Change subject: Depend on asterix-opt-bom 1.0.0-SNAPSHOT
..

Depend on asterix-opt-bom 1.0.0-SNAPSHOT

Change-Id: Ifd5b95bc052c87419d0e925e46f7741b97cc1132
---
M asterixdb/asterix-server/pom.xml
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/37/1137/1

diff --git a/asterixdb/asterix-server/pom.xml b/asterixdb/asterix-server/pom.xml
index 269c219..c713505 100644
--- a/asterixdb/asterix-server/pom.xml
+++ b/asterixdb/asterix-server/pom.xml
@@ -176,7 +176,7 @@
 
   org.apache.asterix
   asterix-opt-bom
-  0.1-SNAPSHOT
+  1.0.0-SNAPSHOT
   pom
 
   

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1137
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ifd5b95bc052c87419d0e925e46f7741b97cc1132
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Chris Hillery 

Change in asterixdb[master]: Add standalone performance test.

[Chris Hillery (Code Review)]

Chris Hillery has submitted this change and it was merged.

Change subject: Add standalone performance test.
..


Add standalone performance test.

Change-Id: Iaffcbd2f798402742740b8ac12f9cf41a52c
Reviewed-on: https://asterix-gerrit.ics.uci.edu/1070
Tested-by: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Ian Maxon 
---
A 
asterixdb/asterix-experiments/src/main/java/org/apache/asterix/experiment/builder/PresetClusterPerfBuilder.java
M 
asterixdb/asterix-experiments/src/main/java/org/apache/asterix/experiment/client/LSMExperimentSetRunner.java
A 
asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/bench_3_load_2.aql
3 files changed, 145 insertions(+), 2 deletions(-)

Approvals:
  Ian Maxon: Looks good to me, approved
  Jenkins: Verified; Verified

Objections:
  Jenkins: Violations found



diff --git 
a/asterixdb/asterix-experiments/src/main/java/org/apache/asterix/experiment/builder/PresetClusterPerfBuilder.java
 
b/asterixdb/asterix-experiments/src/main/java/org/apache/asterix/experiment/builder/PresetClusterPerfBuilder.java
new file mode 100644
index 000..5c136d6
--- /dev/null
+++ 
b/asterixdb/asterix-experiments/src/main/java/org/apache/asterix/experiment/builder/PresetClusterPerfBuilder.java
@@ -0,0 +1,134 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.asterix.experiment.builder;
+
+import org.apache.asterix.experiment.action.base.SequentialActionList;
+import org.apache.asterix.experiment.action.derived.RunAQLFileAction;
+import org.apache.asterix.experiment.action.derived.RunSQLPPFileAction;
+import org.apache.asterix.experiment.action.derived.SleepAction;
+import org.apache.asterix.experiment.action.derived.TimedAction;
+import org.apache.asterix.experiment.client.LSMExperimentConstants;
+import 
org.apache.asterix.experiment.client.LSMExperimentSetRunner.LSMExperimentSetRunnerConfig;
+import org.apache.asterix.experiment.client.LSMPerfConstants;
+import org.apache.http.client.HttpClient;
+import org.apache.http.impl.client.DefaultHttpClient;
+
+import javax.xml.bind.JAXBException;
+import java.io.IOException;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Scanner;
+
+/**
+ * This experiment simply runs the performance benchmark queries against
+ * a cluster which is presumed to already be running.
+ * Also, there is no orchestration server involved in this experiment builder.
+ *
+ * The only LSMExperimentSetRunner config command-line options used by this
+ * experiment are:
+ *   -ler  -rh  -rp
+ */
+public class PresetClusterPerfBuilder extends AbstractExperimentBuilder {
+
+protected final HttpClient httpClient;
+
+protected final String restHost;
+
+protected final int restPort;
+
+protected final Path localExperimentRoot;
+
+private final String countFileName;
+
+protected final String loadAQLFilePath;
+
+protected final String querySQLPPFileName;
+
+public PresetClusterPerfBuilder(LSMExperimentSetRunnerConfig config) {
+super("PresetClusterPerfBuilder");
+this.httpClient = new DefaultHttpClient();
+this.restHost = config.getRESTHost();
+this.restPort = config.getRESTPort();
+this.localExperimentRoot = Paths.get(config.getLocalExperimentRoot());
+this.countFileName = "bench_count.aql";
+this.loadAQLFilePath = "bench_3_load_2.aql";
+this.querySQLPPFileName = "agg_bench";
+}
+
+@Override
+protected void doBuild(Experiment e) throws IOException, JAXBException {
+SequentialActionList execs = new SequentialActionList();
+
+//ddl statements
+execs.add(new SleepAction(15000));
+// TODO: implement retry handler
+execs.add(new RunAQLFileAction(httpClient, restHost, restPort, 
localExperimentRoot.resolve(
+

Change in asterixdb[master]: Add standalone performance test.

[Chris Hillery (Code Review)]

Chris Hillery has posted comments on this change.

Change subject: Add standalone performance test.
..


Patch Set 1:

My thoughts exactly. :) Thanks.

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1070
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Iaffcbd2f798402742740b8ac12f9cf41a52c
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Chris Hillery 
Gerrit-Reviewer: Chris Hillery 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: Ensure RMI host is set to the cluster ip address

[Chris Hillery (Code Review)]

Chris Hillery has posted comments on this change.

Change subject: Ensure RMI host is set to the cluster ip address
..


Patch Set 2:

Mike - the client and cluster IPs for the CC are only the "public" IPs already. 
The client IP is obviously the one that external clients can see. As for 
cluster IP, the CC listens on all interfaces; the specific cluster IP is the 
one which is passed to NCs for connecting.

FYI, though, I'm no longer sure the whole public/private divide is really 
desirable. It was there primarily to support earlier limitations in Docker, but 
that's long ago. It'd still be useful if you were setting up a cluster where 
some nodes were behind NAT or something, but I can't imagine that's really a 
common use case. It might be worth stripping it out to avoid confusions.

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1065
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0fbda4ce14c892e66595d7a1cc3801f0fc2b8fba
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Michael Blow 
Gerrit-Reviewer: Chris Hillery 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 
Gerrit-HasComments: No

Change in asterixdb[master]: Add standalone performance test.

[Chris Hillery (Code Review)]

Chris Hillery has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/1070

Change subject: Add standalone performance test.
..

Add standalone performance test.

Change-Id: Iaffcbd2f798402742740b8ac12f9cf41a52c
---
A 
asterixdb/asterix-experiments/src/main/java/org/apache/asterix/experiment/builder/PresetClusterPerfBuilder.java
M 
asterixdb/asterix-experiments/src/main/java/org/apache/asterix/experiment/client/LSMExperimentSetRunner.java
A 
asterixdb/asterix-experiments/src/main/resources/ingestion-experiment-binary-and-configs/aql/bench_3_load_2.aql
3 files changed, 145 insertions(+), 2 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/70/1070/1

diff --git 
a/asterixdb/asterix-experiments/src/main/java/org/apache/asterix/experiment/builder/PresetClusterPerfBuilder.java
 
b/asterixdb/asterix-experiments/src/main/java/org/apache/asterix/experiment/builder/PresetClusterPerfBuilder.java
new file mode 100644
index 000..5c136d6
--- /dev/null
+++ 
b/asterixdb/asterix-experiments/src/main/java/org/apache/asterix/experiment/builder/PresetClusterPerfBuilder.java
@@ -0,0 +1,134 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.asterix.experiment.builder;
+
+import org.apache.asterix.experiment.action.base.SequentialActionList;
+import org.apache.asterix.experiment.action.derived.RunAQLFileAction;
+import org.apache.asterix.experiment.action.derived.RunSQLPPFileAction;
+import org.apache.asterix.experiment.action.derived.SleepAction;
+import org.apache.asterix.experiment.action.derived.TimedAction;
+import org.apache.asterix.experiment.client.LSMExperimentConstants;
+import 
org.apache.asterix.experiment.client.LSMExperimentSetRunner.LSMExperimentSetRunnerConfig;
+import org.apache.asterix.experiment.client.LSMPerfConstants;
+import org.apache.http.client.HttpClient;
+import org.apache.http.impl.client.DefaultHttpClient;
+
+import javax.xml.bind.JAXBException;
+import java.io.IOException;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Scanner;
+
+/**
+ * This experiment simply runs the performance benchmark queries against
+ * a cluster which is presumed to already be running.
+ * Also, there is no orchestration server involved in this experiment builder.
+ *
+ * The only LSMExperimentSetRunner config command-line options used by this
+ * experiment are:
+ *   -ler  -rh  -rp
+ */
+public class PresetClusterPerfBuilder extends AbstractExperimentBuilder {
+
+protected final HttpClient httpClient;
+
+protected final String restHost;
+
+protected final int restPort;
+
+protected final Path localExperimentRoot;
+
+private final String countFileName;
+
+protected final String loadAQLFilePath;
+
+protected final String querySQLPPFileName;
+
+public PresetClusterPerfBuilder(LSMExperimentSetRunnerConfig config) {
+super("PresetClusterPerfBuilder");
+this.httpClient = new DefaultHttpClient();
+this.restHost = config.getRESTHost();
+this.restPort = config.getRESTPort();
+this.localExperimentRoot = Paths.get(config.getLocalExperimentRoot());
+this.countFileName = "bench_count.aql";
+this.loadAQLFilePath = "bench_3_load_2.aql";
+this.querySQLPPFileName = "agg_bench";
+}
+
+@Override
+protected void doBuild(Experiment e) throws IOException, JAXBException {
+SequentialActionList execs = new SequentialActionList();
+
+//ddl statements
+execs.add(new SleepAction(15000));
+// TODO: implement retry handler
+execs.add(new RunAQLFileAction(httpClient, restHost, restPort, 
localExperimentRoot.resolve(
+
LSMExperimentConstants.AQL_DIR).resolve(LSMPerfConstants.BASE_TYPES)));
+execs.add(new RunAQLFileAction(httpClient, restHost, restPort, 
localExperimentRoot.resolve(
+LSMExperimentConstants.AQL_DIR).resolve("bench_3.aql")));
+
+//-- main experiment 

Change in asterixdb[master]: Ensure RMI host is set to the cluster ip address

[Chris Hillery (Code Review)]

Chris Hillery has posted comments on this change.

Change subject: Ensure RMI host is set to the cluster ip address
..


Patch Set 1: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/1065
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0fbda4ce14c892e66595d7a1cc3801f0fc2b8fba
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Michael Blow 
Gerrit-Reviewer: Chris Hillery 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-HasComments: No

Change in asterixdb[master]: Use more accurate name 'NCService' for HyracksVirtualCluster

[Chris Hillery (Code Review)]

Chris Hillery has submitted this change and it was merged.

Change subject: Use more accurate name 'NCService' for HyracksVirtualCluster
..


Use more accurate name 'NCService' for HyracksVirtualCluster

Change-Id: I2c68c8a7abde3906df2961b214f046bdd9bfd426
Reviewed-on: https://asterix-gerrit.ics.uci.edu/982
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Till Westmann 
---
M 
asterixdb/asterix-server/src/test/java/org/apache/asterix/server/test/NCServiceExecutionIT.java
R 
hyracks-fullstack/hyracks/hyracks-server/src/main/java/org/apache/hyracks/server/process/HyracksNCServiceProcess.java
M 
hyracks-fullstack/hyracks/hyracks-server/src/main/java/org/apache/hyracks/server/process/HyracksVirtualCluster.java
M 
hyracks-fullstack/hyracks/hyracks-server/src/test/java/org/apache/hyracks/server/test/NCServiceIT.java
4 files changed, 11 insertions(+), 13 deletions(-)

Approvals:
  Till Westmann: Looks good to me, approved
  Jenkins: Verified; No violations found; Verified



diff --git 
a/asterixdb/asterix-server/src/test/java/org/apache/asterix/server/test/NCServiceExecutionIT.java
 
b/asterixdb/asterix-server/src/test/java/org/apache/asterix/server/test/NCServiceExecutionIT.java
index c179103..696726f 100644
--- 
a/asterixdb/asterix-server/src/test/java/org/apache/asterix/server/test/NCServiceExecutionIT.java
+++ 
b/asterixdb/asterix-server/src/test/java/org/apache/asterix/server/test/NCServiceExecutionIT.java
@@ -111,11 +111,11 @@
 HDFSCluster.getInstance().setup(ASTERIX_APP_DIR + File.separator);
 
 cluster = new HyracksVirtualCluster(new File(APP_HOME), new 
File(ASTERIX_APP_DIR));
-cluster.addNC(
+cluster.addNCService(
 new File(CONF_DIR, "ncservice1.conf"),
 new File(LOG_DIR, "ncservice1.log")
 );
-cluster.addNC(
+cluster.addNCService(
 new File(CONF_DIR, "ncservice2.conf"),
 new File(LOG_DIR, "ncservice2.log")
 );
diff --git 
a/hyracks-fullstack/hyracks/hyracks-server/src/main/java/org/apache/hyracks/server/process/HyracksNCProcess.java
 
b/hyracks-fullstack/hyracks/hyracks-server/src/main/java/org/apache/hyracks/server/process/HyracksNCServiceProcess.java
similarity index 88%
rename from 
hyracks-fullstack/hyracks/hyracks-server/src/main/java/org/apache/hyracks/server/process/HyracksNCProcess.java
rename to 
hyracks-fullstack/hyracks/hyracks-server/src/main/java/org/apache/hyracks/server/process/HyracksNCServiceProcess.java
index 8bc1694..39971ad 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-server/src/main/java/org/apache/hyracks/server/process/HyracksNCProcess.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-server/src/main/java/org/apache/hyracks/server/process/HyracksNCServiceProcess.java
@@ -23,9 +23,9 @@
 import java.io.File;
 import java.util.List;
 
-public class HyracksNCProcess extends HyracksServerProcess {
+public class HyracksNCServiceProcess extends HyracksServerProcess {
 
-public HyracksNCProcess(File configFile, File logFile, File appHome, File 
workingDir) {
+public HyracksNCServiceProcess(File configFile, File logFile, File 
appHome, File workingDir) {
 this.configFile = configFile;
 this.logFile = logFile;
 this.appHome = appHome;
diff --git 
a/hyracks-fullstack/hyracks/hyracks-server/src/main/java/org/apache/hyracks/server/process/HyracksVirtualCluster.java
 
b/hyracks-fullstack/hyracks/hyracks-server/src/main/java/org/apache/hyracks/server/process/HyracksVirtualCluster.java
index f08bb43..cc0ecf7 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-server/src/main/java/org/apache/hyracks/server/process/HyracksVirtualCluster.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-server/src/main/java/org/apache/hyracks/server/process/HyracksVirtualCluster.java
@@ -30,7 +30,7 @@
 public class HyracksVirtualCluster {
 private final File appHome;
 private final File workingDir;
-private List ncProcs = new ArrayList<>(3);
+private List ncProcs = new ArrayList<>(3);
 private HyracksCCProcess ccProc = null;
 
 /**
@@ -51,15 +51,15 @@
  * @param configFile - full path to an ncservice.conf. May be null to 
accept all defaults.
  * @throws IOException - if there are errors starting the process.
  */
-public void addNC(File configFile, File logFile) throws IOException {
-HyracksNCProcess proc = new HyracksNCProcess(configFile, logFile, 
appHome, workingDir);
+public void addNCService(File configFile, File logFile) throws IOException 
{
+HyracksNCServiceProcess proc = new HyracksNCServiceProcess(configFile, 
logFile, appHome, workingDir);
 proc.start();
 ncProcs.add(proc);
 }
 
 /**
  * Starts the CC, initializing the cluster. Expects that any 

Change in asterixdb[master]: ASTERIXDB-1482: Added NCServiceExecutionIT, HyracksVirtualCl...

[Chris Hillery (Code Review)]

Chris Hillery has submitted this change and it was merged.

Change subject: ASTERIXDB-1482: Added NCServiceExecutionIT, 
HyracksVirtualCluster.
..


ASTERIXDB-1482: Added NCServiceExecutionIT, HyracksVirtualCluster.

NCServiceExecutionIT runs all execution tests against a local cluster
managed by the NCService deployment framework.

HyracksVirtualCluster offers programmatic NCService deployment
control along with improved HyracksNCProcess/HyracksCCProcess.

Further fixes and improvements:

1. Fix handling of iodevices/storagedir (ASTERIXDB-1482)
2. Proper handling of [nc] default section in all cases
3. Ensure asterixnc, etc. scripts are executable
4. Consolidate Ini handling
5. Pruned some dead code, including VirtualClusterDriver
6. A bit of refactoring and extended commenting

Change-Id: If3eb450782a595cf85d04a2c2e9cc732564e65e6
Reviewed-on: https://asterix-gerrit.ics.uci.edu/958
Tested-by: Jenkins 
Reviewed-by: Ian Maxon 
---
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixMetadataProperties.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixPropertiesAccessor.java
M asterixdb/asterix-server/pom.xml
A 
asterixdb/asterix-server/src/test/java/org/apache/asterix/server/test/NCServiceExecutionIT.java
A asterixdb/asterix-server/src/test/resources/NCServiceExecutionIT/cc.conf
A 
asterixdb/asterix-server/src/test/resources/NCServiceExecutionIT/ncservice1.conf
A 
asterixdb/asterix-server/src/test/resources/NCServiceExecutionIT/ncservice2.conf
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/work/TriggerNCWork.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/application/IniApplicationConfig.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/CCConfig.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/IniUtils.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/NCConfig.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-nc-service/src/main/java/org/apache/hyracks/control/nc/service/NCService.java
M hyracks-fullstack/hyracks/hyracks-server/pom.xml
D 
hyracks-fullstack/hyracks/hyracks-server/src/main/java/org/apache/hyracks/server/drivers/VirtualClusterDriver.java
M 
hyracks-fullstack/hyracks/hyracks-server/src/main/java/org/apache/hyracks/server/process/HyracksCCProcess.java
M 
hyracks-fullstack/hyracks/hyracks-server/src/main/java/org/apache/hyracks/server/process/HyracksNCProcess.java
M 
hyracks-fullstack/hyracks/hyracks-server/src/main/java/org/apache/hyracks/server/process/HyracksServerProcess.java
A 
hyracks-fullstack/hyracks/hyracks-server/src/main/java/org/apache/hyracks/server/process/HyracksVirtualCluster.java
M 
hyracks-fullstack/hyracks/hyracks-server/src/test/java/org/apache/hyracks/server/test/NCServiceIT.java
M hyracks-fullstack/hyracks/hyracks-server/src/test/resources/logging.properties
22 files changed, 536 insertions(+), 369 deletions(-)

Approvals:
  Ian Maxon: Looks good to me, approved
  Jenkins: Verified



diff --git 
a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixMetadataProperties.java
 
b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixMetadataProperties.java
index 9a8fba4..677fc78 100644
--- 
a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixMetadataProperties.java
+++ 
b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixMetadataProperties.java
@@ -39,7 +39,7 @@
 }
 
 public ClusterPartition getMetadataPartition() {
-return accessor.getMetadataPartiton();
+return accessor.getMetadataPartition();
 }
 
 public Map getStores() {
diff --git 
a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixPropertiesAccessor.java
 
b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixPropertiesAccessor.java
index 507a393..7309f0c 100644
--- 
a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixPropertiesAccessor.java
+++ 
b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixPropertiesAccessor.java
@@ -18,6 +18,7 @@
  */
 package org.apache.asterix.common.config;
 
+import java.io.File;
 import java.io.FileInputStream;
 import java.io.FileNotFoundException;
 import java.io.IOException;
@@ -53,10 

Change in asterixdb[master]: ASTERIXDB-1482: Added NCServiceExecutionIT, HyracksVirtualCl...

[Chris Hillery (Code Review)]

Chris Hillery has posted comments on this change.

Change subject: ASTERIXDB-1482: Added NCServiceExecutionIT, 
HyracksVirtualCluster.
..


Patch Set 3:

(3 comments)

https://asterix-gerrit.ics.uci.edu/#/c/958/3/asterixdb/asterix-server/pom.xml
File asterixdb/asterix-server/pom.xml:

Line 127:   
> What's this plugin about?
I copied these plugin settings from hyracks-server, where it's doing the same 
thing: making sure that the appassembler-generated .sh scripts are executable.


https://asterix-gerrit.ics.uci.edu/#/c/958/3/asterixdb/asterix-server/src/test/java/org/apache/asterix/server/test/NCServiceExecutionIT.java
File 
asterixdb/asterix-server/src/test/java/org/apache/asterix/server/test/NCServiceExecutionIT.java:

Line 135: LOGGER.info("Sleeping while cluster comes online...");
> Is there a better way to do this? Maybe trying to fetch against some port o
I'm not sure.. I suppose it could poll the rest/nodes Hyracks endpoint until 
the expected number of NCs are connected?


https://asterix-gerrit.ics.uci.edu/#/c/958/3/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/CCConfig.java
File 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/CCConfig.java:

Line 155
> Why remove this?
Because it no longer serves any purpose, with VirtualClusterDriver removed. It 
was always an ugly hack that risked being out of sync, and with the NCService 
model I'm trying to minimize the use of command-line arguments anyway.


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/958
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: If3eb450782a595cf85d04a2c2e9cc732564e65e6
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Chris Hillery 
Gerrit-Reviewer: Chris Hillery 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-HasComments: Yes

Change in asterixdb[master]: ASTERIXDB-1482: Added NCServiceExecutionIT, HyracksVirtualCl...

[Chris Hillery (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/958

to look at the new patch set (#3).

Change subject: ASTERIXDB-1482: Added NCServiceExecutionIT, 
HyracksVirtualCluster.
..

ASTERIXDB-1482: Added NCServiceExecutionIT, HyracksVirtualCluster.

NCServiceExecutionIT runs all execution tests against a local cluster
managed by the NCService deployment framework.

HyracksVirtualCluster offers programmatic NCService deployment
control along with improved HyracksNCProcess/HyracksCCProcess.

Further fixes and improvements:

1. Fix handling of iodevices/storagedir (ASTERIXDB-1482)
2. Proper handling of [nc] default section in all cases
3. Ensure asterixnc, etc. scripts are executable
4. Consolidate Ini handling
5. Pruned some dead code, including VirtualClusterDriver
6. A bit of refactoring and extended commenting

Change-Id: If3eb450782a595cf85d04a2c2e9cc732564e65e6
---
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixMetadataProperties.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixPropertiesAccessor.java
M asterixdb/asterix-server/pom.xml
A 
asterixdb/asterix-server/src/test/java/org/apache/asterix/server/test/NCServiceExecutionIT.java
A asterixdb/asterix-server/src/test/resources/NCServiceExecutionIT/cc.conf
A 
asterixdb/asterix-server/src/test/resources/NCServiceExecutionIT/ncservice1.conf
A 
asterixdb/asterix-server/src/test/resources/NCServiceExecutionIT/ncservice2.conf
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/work/TriggerNCWork.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/application/IniApplicationConfig.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/CCConfig.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/IniUtils.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/NCConfig.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-nc-service/src/main/java/org/apache/hyracks/control/nc/service/NCService.java
M hyracks-fullstack/hyracks/hyracks-server/pom.xml
D 
hyracks-fullstack/hyracks/hyracks-server/src/main/java/org/apache/hyracks/server/drivers/VirtualClusterDriver.java
M 
hyracks-fullstack/hyracks/hyracks-server/src/main/java/org/apache/hyracks/server/process/HyracksCCProcess.java
M 
hyracks-fullstack/hyracks/hyracks-server/src/main/java/org/apache/hyracks/server/process/HyracksNCProcess.java
M 
hyracks-fullstack/hyracks/hyracks-server/src/main/java/org/apache/hyracks/server/process/HyracksServerProcess.java
A 
hyracks-fullstack/hyracks/hyracks-server/src/main/java/org/apache/hyracks/server/process/HyracksVirtualCluster.java
M 
hyracks-fullstack/hyracks/hyracks-server/src/test/java/org/apache/hyracks/server/test/NCServiceIT.java
M hyracks-fullstack/hyracks/hyracks-server/src/test/resources/logging.properties
22 files changed, 536 insertions(+), 369 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/58/958/3
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/958
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If3eb450782a595cf85d04a2c2e9cc732564e65e6
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Chris Hillery 
Gerrit-Reviewer: Chris Hillery 
Gerrit-Reviewer: Jenkins 

Change in asterixdb[master]: Added NCServiceExecutionIT and programmatic NCService deploy...

[Chris Hillery (Code Review)]

Chris Hillery has posted comments on this change.

Change subject: Added NCServiceExecutionIT and programmatic NCService 
deployment control.
..


Patch Set 2:

I do not believe the cross-project changes here are dangerous. Largely I 
deleted VirtualClusterDriver which was unused by Asterix or VXQuery. I also 
deleted the now-unused toCommandLine() methods on CCConfig and NCConfig. There 
are API changes in HyracksCCProcess and HyracksNCProcess, but those classes 
were unused outside of VirtualClusterDriver.

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/958
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: If3eb450782a595cf85d04a2c2e9cc732564e65e6
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Chris Hillery 
Gerrit-Reviewer: Chris Hillery 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: Added NCServiceExecutionIT and programmatic NCService deploy...

[Chris Hillery (Code Review)]

Chris Hillery has uploaded a new patch set (#2).

Change subject: Added NCServiceExecutionIT and programmatic NCService 
deployment control.
..

Added NCServiceExecutionIT and programmatic NCService deployment control.

NCServiceExecutionIT runs all execution tests against a local cluster
managed by the NCService deployment framework.

Programmatic NCService control comprises new HyracksVirtualCluster class
and improved HyracksNCProcess/HyracksCCProcess.

Further fixes and improvements:

1. Fix handling of iodevices/storagedir
2. Proper handling of [nc] default section in all cases
3. Ensure asterixnc, etc. scripts are executable
4. Consolidate Ini handling
5. Pruned some dead code, including VirtualClusterDriver
6. A bit of refactoring and extended commenting

Change-Id: If3eb450782a595cf85d04a2c2e9cc732564e65e6
---
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixMetadataProperties.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixPropertiesAccessor.java
M asterixdb/asterix-server/pom.xml
A 
asterixdb/asterix-server/src/test/java/org/apache/asterix/server/test/NCServiceExecutionIT.java
A asterixdb/asterix-server/src/test/resources/NCServiceExecutionIT/cc.conf
A 
asterixdb/asterix-server/src/test/resources/NCServiceExecutionIT/ncservice1.conf
A 
asterixdb/asterix-server/src/test/resources/NCServiceExecutionIT/ncservice2.conf
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/work/TriggerNCWork.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/application/IniApplicationConfig.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/CCConfig.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/IniUtils.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/NCConfig.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-nc-service/src/main/java/org/apache/hyracks/control/nc/service/NCService.java
M hyracks-fullstack/hyracks/hyracks-server/pom.xml
D 
hyracks-fullstack/hyracks/hyracks-server/src/main/java/org/apache/hyracks/server/drivers/VirtualClusterDriver.java
M 
hyracks-fullstack/hyracks/hyracks-server/src/main/java/org/apache/hyracks/server/process/HyracksCCProcess.java
M 
hyracks-fullstack/hyracks/hyracks-server/src/main/java/org/apache/hyracks/server/process/HyracksNCProcess.java
M 
hyracks-fullstack/hyracks/hyracks-server/src/main/java/org/apache/hyracks/server/process/HyracksServerProcess.java
A 
hyracks-fullstack/hyracks/hyracks-server/src/main/java/org/apache/hyracks/server/process/HyracksVirtualCluster.java
M 
hyracks-fullstack/hyracks/hyracks-server/src/test/java/org/apache/hyracks/server/test/NCServiceIT.java
M hyracks-fullstack/hyracks/hyracks-server/src/test/resources/logging.properties
22 files changed, 536 insertions(+), 369 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/58/958/2
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/958
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If3eb450782a595cf85d04a2c2e9cc732564e65e6
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Chris Hillery 
Gerrit-Reviewer: Jenkins 

Change in asterixdb[master]: Added NCServiceExecutionIT and implemented several fixes.

[Chris Hillery (Code Review)]

Chris Hillery has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/958

Change subject: Added NCServiceExecutionIT and implemented several fixes.
..

Added NCServiceExecutionIT and implemented several fixes.

1. Fix handling of iodevices/storagedir
2. Proper handling of [nc] default section in all cases
3. Consolidate Ini handling
4. Pruned some dead code, and a bit of refactoring

Change-Id: If3eb450782a595cf85d04a2c2e9cc732564e65e6
---
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixMetadataProperties.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixPropertiesAccessor.java
M asterixdb/asterix-server/pom.xml
A 
asterixdb/asterix-server/src/test/java/org/apache/asterix/server/test/NCServiceExecutionIT.java
A asterixdb/asterix-server/src/test/resources/NCServiceExecutionIT/cc.conf
A 
asterixdb/asterix-server/src/test/resources/NCServiceExecutionIT/ncservice1.conf
A 
asterixdb/asterix-server/src/test/resources/NCServiceExecutionIT/ncservice2.conf
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/work/TriggerNCWork.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/application/IniApplicationConfig.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/CCConfig.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/IniUtils.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/NCConfig.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-nc-service/src/main/java/org/apache/hyracks/control/nc/service/NCService.java
M hyracks-fullstack/hyracks/hyracks-server/pom.xml
D 
hyracks-fullstack/hyracks/hyracks-server/src/main/java/org/apache/hyracks/server/drivers/VirtualClusterDriver.java
M 
hyracks-fullstack/hyracks/hyracks-server/src/main/java/org/apache/hyracks/server/process/HyracksCCProcess.java
M 
hyracks-fullstack/hyracks/hyracks-server/src/main/java/org/apache/hyracks/server/process/HyracksNCProcess.java
M 
hyracks-fullstack/hyracks/hyracks-server/src/main/java/org/apache/hyracks/server/process/HyracksServerProcess.java
A 
hyracks-fullstack/hyracks/hyracks-server/src/main/java/org/apache/hyracks/server/process/HyracksVirtualCluster.java
M 
hyracks-fullstack/hyracks/hyracks-server/src/test/java/org/apache/hyracks/server/test/NCServiceIT.java
M hyracks-fullstack/hyracks/hyracks-server/src/test/resources/logging.properties
22 files changed, 536 insertions(+), 369 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/58/958/1

diff --git 
a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixMetadataProperties.java
 
b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixMetadataProperties.java
index 9a8fba4..677fc78 100644
--- 
a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixMetadataProperties.java
+++ 
b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixMetadataProperties.java
@@ -39,7 +39,7 @@
 }
 
 public ClusterPartition getMetadataPartition() {
-return accessor.getMetadataPartiton();
+return accessor.getMetadataPartition();
 }
 
 public Map getStores() {
diff --git 
a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixPropertiesAccessor.java
 
b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixPropertiesAccessor.java
index 507a393..7309f0c 100644
--- 
a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixPropertiesAccessor.java
+++ 
b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixPropertiesAccessor.java
@@ -18,6 +18,7 @@
  */
 package org.apache.asterix.common.config;
 
+import java.io.File;
 import java.io.FileInputStream;
 import java.io.FileNotFoundException;
 import java.io.IOException;
@@ -53,10 +54,12 @@
 private final List nodeNames = new ArrayList<>();;
 private final Map stores = new HashMap<>();;
 private final Map coredumpConfig = new HashMap<>();
+
+// This can be removed when asterix-configuration.xml is no longer 
required.
 private final Map asterixConfigurationParams;
 private final IApplicationConfig cfg;
 private final Map transactionLogDirs = new HashMap<>();
-private final Map 

Change in asterixdb[master]: Doc for single node installation using NCService

[Chris Hillery (Code Review)]

Chris Hillery has posted comments on this change.

Change subject: Doc for single node installation using NCService
..


Patch Set 7: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/895
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id0a5ad6e88d72ce2fcaaeabc79fd86c374005ad3
Gerrit-PatchSet: 7
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Till Westmann 
Gerrit-Reviewer: Chris Hillery 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-HasComments: No

Change in asterixdb[master]: Doc for single node installation using NCService

[Chris Hillery (Code Review)]

Chris Hillery has posted comments on this change.

Change subject: Doc for single node installation using NCService
..


Patch Set 2:

(3 comments)

https://asterix-gerrit.ics.uci.edu/#/c/895/2/asterixdb/asterix-doc/src/site/markdown/ncservice.md
File asterixdb/asterix-doc/src/site/markdown/ncservice.md:

Line 79: $ export 
BINDIR=[..]/asterix-server-0.8.9-SNAPSHOT-binary-assembly/bin
Other than this line, I believe these instructions would work for both an 
in-source build as well as a binary .zip download. Can this one line be made 
more generic with a little description of what "BINDIR" should point to / 
contain?


Line 81: $ $BINDIR/asterixncservice -config-file blue.conf &> 
blue-service.log &
&> is a bash-ism which won't work in strict bourne shell... probably should use 
"blue.conf > blue-service.log 2>&1 &"


Line 97: 13198 NCDriver
Worth adding a note that the NCDriver logs will be in $app.home/logs ?


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/895
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id0a5ad6e88d72ce2fcaaeabc79fd86c374005ad3
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Till Westmann 
Gerrit-Reviewer: Chris Hillery 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: Yes

Change in asterixdb[master]: Fixes and tweaks for running Asterix atop new config-managem...

[Chris Hillery (Code Review)]

Chris Hillery has submitted this change and it was merged.

Change subject: Fixes and tweaks for running Asterix atop new config-management 
framework.
..


Fixes and tweaks for running Asterix atop new config-management framework.

Change-Id: Ie3027c8c839f25ea858790bd3340187f4b11f213
Reviewed-on: https://asterix-gerrit.ics.uci.edu/874
Reviewed-by: Jenkins 
Tested-by: Jenkins 
Reviewed-by: Till Westmann 
Reviewed-by: Michael Blow 
---
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixPropertiesAccessor.java
M 
asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/resource/PersistentLocalResourceRepository.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-nc-service/src/main/java/org/apache/hyracks/control/nc/service/NCServiceConfig.java
3 files changed, 27 insertions(+), 14 deletions(-)

Approvals:
  Michael Blow: Looks good to me, approved
  Till Westmann: Looks good to me, approved
  Jenkins: Looks good to me, but someone else must approve; Verified



diff --git 
a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixPropertiesAccessor.java
 
b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixPropertiesAccessor.java
index 62bdbf5..45e3b06 100644
--- 
a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixPropertiesAccessor.java
+++ 
b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/AsterixPropertiesAccessor.java
@@ -136,12 +136,13 @@
 this.cfg = cfg;
 instanceName = cfg.getString("asterix", "instance", 
"DEFAULT_INSTANCE");
 String mdNode = null;
+nodePartitionsMap = new HashMap<>();
+int uniquePartitionId = 0;
 for (String section : cfg.getSections()) {
 if (!section.startsWith("nc/")) {
 continue;
 }
 String ncId = section.substring(3);
-nodeNames.add(ncId);
 
 if (mdNode == null) {
 // Default is first node == metadata node
@@ -156,15 +157,23 @@
 // be a default.ini? They can't be inserted by TriggerNCWork except
 // possibly for hyracks-specified values. Certainly wherever they 
are,
 // they should be platform-dependent.
-stores.put(ncId, cfg.getString(section, "iodevices", 
"/var/lib/asterixdb/data").split(","));
 coredumpConfig.put(ncId, cfg.getString(section, "coredumpdir", 
"/var/lib/asterixdb/coredump"));
 transactionLogDirs.put(ncId, cfg.getString(section, "txnlogdir", 
"/var/lib/asterixdb/txn-log"));
+String[] storeDirs = cfg.getString(section, "storagedir", 
"storage").trim().split(",");
+ClusterPartition[] nodePartitions = new 
ClusterPartition[storeDirs.length];
+for (int i = 0; i < nodePartitions.length; i++) {
+ClusterPartition partition = new 
ClusterPartition(uniquePartitionId++, ncId, i);
+clusterPartitions.put(partition.getPartitionId(), partition);
+nodePartitions[i] = partition;
+}
+stores.put(ncId, storeDirs);
+nodePartitionsMap.put(ncId, nodePartitions);
+nodeNames.add(ncId);
 }
 
 metadataNodeName = mdNode;
 asterixConfigurationParams = null;
 asterixBuildProperties = null;
-nodePartitionsMap = null;
 }
 
 public String getMetadataNodeName() {
diff --git 
a/asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/resource/PersistentLocalResourceRepository.java
 
b/asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/resource/PersistentLocalResourceRepository.java
index 561b144..fc79ae0 100644
--- 
a/asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/resource/PersistentLocalResourceRepository.java
+++ 
b/asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/resource/PersistentLocalResourceRepository.java
@@ -86,9 +86,9 @@
 throw new HyracksDataException(mountPointDir.getAbsolutePath() 
+ " doesn't exist.");
 }
 if (!mountPoint.endsWith(System.getProperty("file.separator"))) {
-mountPoints[i] = new String(mountPoint + 
System.getProperty("file.separator"));
+mountPoints[i] = mountPoint + 
System.getProperty("file.separator");
 } else {
-mountPoints[i] = new String(mountPoint);
+mountPoints[i] = mountPoint;
 }
 }
 resourceCache = 
CacheBuilder.newBuilder().maximumSize(MAX_CACHED_RESOURCES).build();
@@ -136,10 +136,10 @@
 
 String storageRootDirPath;