Change in asterixdb[master]: [ASTERIXDB-2593][FUN] TPC-DS always parallelize + gen all tables

[Murtadha Hubail (Code Review)]

Murtadha Hubail has posted comments on this change. ( 
https://asterix-gerrit.ics.uci.edu/3437 )

Change subject: [ASTERIXDB-2593][FUN] TPC-DS always parallelize + gen all tables
..


Patch Set 9: Code-Review+2

Forwarding Dmitry's +2 after rebase


--
To view, visit https://asterix-gerrit.ics.uci.edu/3437
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iff199b0c533d22bcae1caf5057788b257ba4e486
Gerrit-Change-Number: 3437
Gerrit-PatchSet: 9
Gerrit-Owner: Hussain Towaileb 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Hussain Towaileb 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Comment-Date: Fri, 28 Jun 2019 09:56:32 +
Gerrit-HasComments: No

Change in asterixdb[stabilization-f69489]: [NO ISSUE][REPL] Suspend Dataset Checkpointing on Replica Sync

[Murtadha Hubail (Code Review)]

Murtadha Hubail has uploaded this change for review. ( 
https://asterix-gerrit.ics.uci.edu/3469


Change subject: [NO ISSUE][REPL] Suspend Dataset Checkpointing on Replica Sync
..

[NO ISSUE][REPL] Suspend Dataset Checkpointing on Replica Sync

- user model changes: no
- storage format changes: no
- interface changes: yes

Details:
- Before synchronizing replicas, stop datasets checkpointing to
  prevent new files from being generated due to async IO operations
  triggered by checkpointing.
- Instead of sync'ing current files to replicas then scheduling a flush
  and sync'ing any newly generated files, just flush datasets before
  the initial sync then sync all the files in one go.

Change-Id: I058fd48bc0fb89a1e16448ce516c3410bb4d681d
---
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/transactions/ICheckpointManager.java
M 
asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/sync/ReplicaSynchronizer.java
M 
asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/service/recovery/CheckpointManager.java
3 files changed, 37 insertions(+), 11 deletions(-)



  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/69/3469/1

diff --git 
a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/transactions/ICheckpointManager.java
 
b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/transactions/ICheckpointManager.java
index 36cea55..954e399 100644
--- 
a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/transactions/ICheckpointManager.java
+++ 
b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/transactions/ICheckpointManager.java
@@ -58,4 +58,14 @@
  * @param id
  */
 void completed(TxnId id);
+
+/**
+ * Suspends checkpointing datasets
+ */
+void suspend();
+
+/**
+ * Resumes checkpointing datasets
+ */
+void resume();
 }
diff --git 
a/asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/sync/ReplicaSynchronizer.java
 
b/asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/sync/ReplicaSynchronizer.java
index 0f0b5bd..123709b 100644
--- 
a/asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/sync/ReplicaSynchronizer.java
+++ 
b/asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/sync/ReplicaSynchronizer.java
@@ -22,6 +22,7 @@

 import org.apache.asterix.common.api.INcApplicationContext;
 import org.apache.asterix.common.replication.IReplicationStrategy;
+import org.apache.asterix.common.transactions.ICheckpointManager;
 import org.apache.asterix.replication.api.PartitionReplica;
 import org.apache.asterix.replication.messaging.CheckpointPartitionIndexesTask;
 import org.apache.asterix.replication.messaging.ReplicationProtocol;
@@ -45,21 +46,25 @@
 public void sync() throws IOException {
 final Object syncLock = 
appCtx.getReplicaManager().getReplicaSyncLock();
 synchronized (syncLock) {
-syncFiles();
-checkpointReplicaIndexes();
-appCtx.getReplicationManager().register(replica);
+final ICheckpointManager checkpointManager = 
appCtx.getTransactionSubsystem().getCheckpointManager();
+try {
+// suspend checkpointing datasets to prevent async IO 
operations while sync'ing replicas
+checkpointManager.suspend();
+syncFiles();
+checkpointReplicaIndexes();
+appCtx.getReplicationManager().register(replica);
+} finally {
+checkpointManager.resume();
+}
 }
 }

 private void syncFiles() throws IOException {
 final ReplicaFilesSynchronizer fileSync = new 
ReplicaFilesSynchronizer(appCtx, replica);
-waitForReplicatedDatasetsIO();
-fileSync.sync();
 // flush replicated dataset to generate disk component for any 
remaining in-memory components
 final IReplicationStrategy replStrategy = 
appCtx.getReplicationManager().getReplicationStrategy();
 appCtx.getDatasetLifecycleManager().flushDataset(replStrategy);
 waitForReplicatedDatasetsIO();
-// sync any newly generated files
 fileSync.sync();
 }

diff --git 
a/asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/service/recovery/CheckpointManager.java
 
b/asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/service/recovery/CheckpointManager.java
index ce523db..b85742e 100644
--- 
a/asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/service/recovery/CheckpointManager.java
+++ 
b/asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/service/recovery/CheckpointManager.java
@@ -18,6 +18,10 @@
  */
 package 

Change in asterixdb[master]: Merge commit 'fa6ce67' from stabilization-f69489

[Murtadha Hubail (Code Review)]

Murtadha Hubail has submitted this change and it was merged. ( 
https://asterix-gerrit.ics.uci.edu/3465 )

Change subject: Merge commit 'fa6ce67' from stabilization-f69489
..

Merge commit 'fa6ce67' from stabilization-f69489

Change-Id: I01e1810f1c44d9ecaf569dd22577c33e8702c682
---
1 file changed, 0 insertions(+), 0 deletions(-)

Approvals:
  Jenkins: Verified; No violations found; ; Verified
  Murtadha Hubail: Looks good to me, but someone else must approve
  Michael Blow: Looks good to me, approved

Objections:
  Anon. E. Moose (1000171): Violations found




--
To view, visit https://asterix-gerrit.ics.uci.edu/3465
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I01e1810f1c44d9ecaf569dd22577c33e8702c682
Gerrit-Change-Number: 3465
Gerrit-PatchSet: 1
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Hussain Towaileb 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 

Change in asterixdb[stabilization-f69489]: [NO ISSUE][OTH] Use Absolute Path in FileReference hashCode/equals

[Murtadha Hubail (Code Review)]

Murtadha Hubail has submitted this change and it was merged. ( 
https://asterix-gerrit.ics.uci.edu/3430 )

Change subject: [NO ISSUE][OTH] Use Absolute Path in FileReference 
hashCode/equals
..

[NO ISSUE][OTH] Use Absolute Path in FileReference hashCode/equals

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- To avoid cases where unintentional false comparison result happens
  due to a relative path of one FileReference starting with "/" and
  the other's does not, use the file absolute path in hashCode and
  equals.

Change-Id: Idcb642b3c5396a8319dc0aaf050e8af6fca93039
Reviewed-on: https://asterix-gerrit.ics.uci.edu/3430
Reviewed-by: Michael Blow 
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Integration-Tests: Jenkins 
---
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/io/FileReference.java
1 file changed, 2 insertions(+), 2 deletions(-)

Approvals:
  Michael Blow: Looks good to me, approved
  Jenkins: Verified; No violations found; Verified
  Anon. E. Moose (1000171):



diff --git 
a/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/io/FileReference.java
 
b/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/io/FileReference.java
index 4ded855..e4097ff 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/io/FileReference.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/io/FileReference.java
@@ -58,12 +58,12 @@
 if (!(o instanceof FileReference)) {
 return false;
 }
-return path.equals(((FileReference) o).path) && 
dev.equals(((FileReference) o).dev);
+return file.getAbsolutePath().equals(((FileReference) 
o).getAbsolutePath());
 }

 @Override
 public int hashCode() {
-return path.hashCode();
+return file.getAbsolutePath().hashCode();
 }

 /**

--
To view, visit https://asterix-gerrit.ics.uci.edu/3430
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: stabilization-f69489
Gerrit-MessageType: merged
Gerrit-Change-Id: Idcb642b3c5396a8319dc0aaf050e8af6fca93039
Gerrit-Change-Number: 3430
Gerrit-PatchSet: 2
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 

Change in asterixdb[stabilization-f69489]: [NO ISSUE][REPL] Wait For Dataset IO Before Replica Sync

[Murtadha Hubail (Code Review)]

Murtadha Hubail has submitted this change and it was merged. ( 
https://asterix-gerrit.ics.uci.edu/3446 )

Change subject: [NO ISSUE][REPL] Wait For Dataset IO Before Replica Sync
..

[NO ISSUE][REPL] Wait For Dataset IO Before Replica Sync

- user model changes: no
- storage format changes: no
- interface changes: yes

Details:
- Add API in DatasetLifecycleManager to wait for IO on datasets
  matching a replication strategy.
- Before synchronizing the files on a replica, wait for any
  on-going IO operations on replicated datasets to make sure
  the set of files won't change while the replica is being
  synchronized (e.g. a merge operation won't delete a file
  that we wanted to send to a replica).

Change-Id: I01ed5c9379cf7ae249faeef624d5226ea699cf22
Reviewed-on: https://asterix-gerrit.ics.uci.edu/3446
Tested-by: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Michael Blow 
---
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/api/IDatasetLifecycleManager.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
M 
asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/sync/ReplicaSynchronizer.java
3 files changed, 25 insertions(+), 0 deletions(-)

Approvals:
  Jenkins: Verified; Verified
  Anon. E. Moose (1000171):
  Michael Blow: Looks good to me, approved

Objections:
  Jenkins: Violations found



diff --git 
a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/api/IDatasetLifecycleManager.java
 
b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/api/IDatasetLifecycleManager.java
index d18b6ab..0cc3d6d 100644
--- 
a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/api/IDatasetLifecycleManager.java
+++ 
b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/api/IDatasetLifecycleManager.java
@@ -133,4 +133,12 @@
  * @throws HyracksDataException
  */
 void flushDataset(IReplicationStrategy replicationStrategy) throws 
HyracksDataException;
+
+/**
+ * Waits for all ongoing IO operations on all open datasets that are 
matching {@code replicationStrategy}.
+ *
+ * @param replicationStrategy
+ * @throws HyracksDataException
+ */
+void waitForIO(IReplicationStrategy replicationStrategy) throws 
HyracksDataException;
 }
diff --git 
a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
 
b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
index d767219..30b27d8 100644
--- 
a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
+++ 
b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
@@ -599,6 +599,15 @@
 }
 }

+@Override
+public void waitForIO(IReplicationStrategy replicationStrategy) throws 
HyracksDataException {
+for (DatasetResource dsr : datasets.values()) {
+if (dsr.isOpen() && 
replicationStrategy.isMatch(dsr.getDatasetID())) {
+dsr.getDatasetInfo().waitForIO();
+}
+}
+}
+
 private void closeIndex(IndexInfo indexInfo) throws HyracksDataException {
 if (indexInfo.isOpen()) {
 ILSMOperationTracker opTracker = 
indexInfo.getIndex().getOperationTracker();
diff --git 
a/asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/sync/ReplicaSynchronizer.java
 
b/asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/sync/ReplicaSynchronizer.java
index 09f1205..0f0b5bd 100644
--- 
a/asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/sync/ReplicaSynchronizer.java
+++ 
b/asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/sync/ReplicaSynchronizer.java
@@ -53,10 +53,12 @@

 private void syncFiles() throws IOException {
 final ReplicaFilesSynchronizer fileSync = new 
ReplicaFilesSynchronizer(appCtx, replica);
+waitForReplicatedDatasetsIO();
 fileSync.sync();
 // flush replicated dataset to generate disk component for any 
remaining in-memory components
 final IReplicationStrategy replStrategy = 
appCtx.getReplicationManager().getReplicationStrategy();
 appCtx.getDatasetLifecycleManager().flushDataset(replStrategy);
+waitForReplicatedDatasetsIO();
 // sync any newly generated files
 fileSync.sync();
 }
@@ -75,4 +77,10 @@
 (PersistentLocalResourceRepository) 
appCtx.getLocalResourceRepository();
 return 
localResourceRepository.getReplicatedIndexesMaxComponentId(partition, 
replStrategy);
 }
+
+private void waitForReplicatedDatasetsIO() throws HyracksDataException {
+// wait for IO operations to ensure replicated datasets files won't 
change during replica sync

Change in asterixdb[stabilization-f69489]: [NO ISSUE][OTH] Use Absolute Path in FileReference hashCode/equals

[Murtadha Hubail (Code Review)]

Murtadha Hubail has removed a vote on this change.

Change subject: [NO ISSUE][OTH] Use Absolute Path in FileReference 
hashCode/equals
..


Removed Code-Review-2 by Murtadha Hubail 
--
To view, visit https://asterix-gerrit.ics.uci.edu/3430
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: stabilization-f69489
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: Idcb642b3c5396a8319dc0aaf050e8af6fca93039
Gerrit-Change-Number: 3430
Gerrit-PatchSet: 1
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 

Change in asterixdb[stabilization-f69489]: [NO ISSUE][REPL] Wait For Dataset IO Before Replica Sync

[Murtadha Hubail (Code Review)]

Murtadha Hubail has removed a vote on this change.

Change subject: [NO ISSUE][REPL] Wait For Dataset IO Before Replica Sync
..


Removed Code-Review-2 by Murtadha Hubail 
--
To view, visit https://asterix-gerrit.ics.uci.edu/3446
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: stabilization-f69489
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: I01ed5c9379cf7ae249faeef624d5226ea699cf22
Gerrit-Change-Number: 3446
Gerrit-PatchSet: 2
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 

Change in asterixdb[master]: [ASTERIXDB-2598][RT] Add Support For Runtime Warnings

[Murtadha Hubail (Code Review)]

Murtadha Hubail has posted comments on this change. ( 
https://asterix-gerrit.ics.uci.edu/3451 )

Change subject: [ASTERIXDB-2598][RT] Add Support For Runtime Warnings
..


Patch Set 3:

(1 comment)

https://asterix-gerrit.ics.uci.edu/#/c/3451/3/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
File asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml:

https://asterix-gerrit.ics.uci.edu/#/c/3451/3/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml@12361
PS3, Line 12361: ASX0003: Type incompatibility: function 
min/max gets incompatible input values: bigint and string
> I wonder whether we can be more precise here and specify a set of warnings
Sure, let's discuss it and agree on the way to go before we add more of those.



--
To view, visit https://asterix-gerrit.ics.uci.edu/3451
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I52fa5b807799487d62e67a8861068e1547aa629a
Gerrit-Change-Number: 3451
Gerrit-PatchSet: 3
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Comment-Date: Mon, 24 Jun 2019 20:27:08 +
Gerrit-HasComments: Yes

Change in asterixdb[master]: [ASTERIXDB-2598][RT] Add Support For Runtime Warnings

[Murtadha Hubail (Code Review)]

Murtadha Hubail has posted comments on this change. ( 
https://asterix-gerrit.ics.uci.edu/3451 )

Change subject: [ASTERIXDB-2598][RT] Add Support For Runtime Warnings
..


Patch Set 3: Code-Review+1


--
To view, visit https://asterix-gerrit.ics.uci.edu/3451
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I52fa5b807799487d62e67a8861068e1547aa629a
Gerrit-Change-Number: 3451
Gerrit-PatchSet: 3
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Comment-Date: Sat, 22 Jun 2019 21:11:20 +
Gerrit-HasComments: No

Change in asterixdb[master]: [ASTERIXDB-2598][RT] Add Support For Runtime Warnings

[Murtadha Hubail (Code Review)]

Murtadha Hubail has posted comments on this change. ( 
https://asterix-gerrit.ics.uci.edu/3451 )

Change subject: [ASTERIXDB-2598][RT] Add Support For Runtime Warnings
..


Patch Set 2: Code-Review+1


--
To view, visit https://asterix-gerrit.ics.uci.edu/3451
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I52fa5b807799487d62e67a8861068e1547aa629a
Gerrit-Change-Number: 3451
Gerrit-PatchSet: 2
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Comment-Date: Fri, 21 Jun 2019 12:49:28 +
Gerrit-HasComments: No

Change in asterixdb[master]: [ASTERIXDB-2598][RT] Add Support For Runtime Warnings

[Murtadha Hubail (Code Review)]

Hello Anon. E. Moose (1000171), Jenkins,

I'd like you to reexamine a change. Please visit

https://asterix-gerrit.ics.uci.edu/3451

to look at the new patch set (#2).

Change subject: [ASTERIXDB-2598][RT] Add Support For Runtime Warnings
..

[ASTERIXDB-2598][RT] Add Support For Runtime Warnings

- user model changes: yes
- storage format changes: no
- interface changes: yes

Details:
- Add the ability to add runtime warnings per task
  and return the generated warnings as part of the
  task profile on task completion.
- On successful job completion, aggregate warnings
  from all task profiles of a job.
- Return the generated warnings in the query service
  response as an array of "warnings" each with a code,
  which is currently hard-coded to 1, and a message.
- Fix propagating source location to scalar aggregate
  functions.
- Add a flag in test cases definition to indicate whether
  or not to check for expected warnings and default it to
  false.
- Generate warnings when min/max functions encounter
  incomparable type or unsupported input.
- Add support in test framework to extract warnings
  along with extracting the result field.
- Add support in test framework to validate generated
  and expected warnings.
- Add test cases for min/max generated warnings.

Change-Id: I52fa5b807799487d62e67a8861068e1547aa629a
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/IStatementExecutor.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/common/ResultMetadata.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/NCQueryServiceServlet.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryServiceServlet.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/message/ExecuteStatementRequestMessage.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/message/ExecuteStatementResponseMessage.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/ExecutionWarning.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/JobResultCallback.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/app/result/ResultPrinterTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/common/CancellationTestExecutor.java
A 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/common/ExtractedResult.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/common/IPollTask.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/common/ResultExtractor.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/common/TestExecutor.java
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/min-max-incompatible-types/min-max-incompatible-types.1.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/min-max-incompatible-types/min-max-incompatible-types.2.query.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/min-max-incompatible-types/min-max-incompatible-types.1.adm
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/min-max-incompatible-types/min-max-incompatible-types.2.adm
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
A 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/utils/WarningUtil.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/scalar/AbstractScalarAggregateDescriptor.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractMinMaxAggregateFunction.java
M asterixdb/asterix-test-framework/src/main/resources/Catalog.xsd
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/context/IHyracksTaskContext.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/exceptions/IFormattedException.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/exceptions/SourceLocation.java
A 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/exceptions/Warning.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/job/profiling/om/TaskProfile.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/Joblet.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/Task.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-nc/src/main/java/org/apache/hyracks/control/nc/work/NotifyTaskCompleteWork.java
M 
hyracks-fullstack/hyracks/hyracks-test-support/src/main/java/org/apache/hyracks/test/support/TestTaskContext.java
M 

Change in asterixdb[master]: [NO ISSUE][OTH] Use Request Stats When Printing Results on NC

[Murtadha Hubail (Code Review)]

Murtadha Hubail has submitted this change and it was merged. ( 
https://asterix-gerrit.ics.uci.edu/3444 )

Change subject: [NO ISSUE][OTH] Use Request Stats When Printing Results on NC
..

[NO ISSUE][OTH] Use Request Stats When Printing Results on NC

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- When printing a result on an NC, use the stats object
  of the original request rather than the object received
  from the CC as part of the execute statement response
  message. This is done because the metrics returned to the
  user are generated from the stats object of the request.
- Add metrics test case to sqlpp test suite to test returning
  metrics from NCQueryServiceServlet.

Change-Id: I54c3b3e72cfb0c2499a95bfc0f514dd5d6774293
Reviewed-on: https://asterix-gerrit.ics.uci.edu/3444
Integration-Tests: Jenkins 
Tested-by: Jenkins 
Reviewed-by: Ali Alsuliman 
---
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/NCQueryServiceServlet.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/NcResultPrinter.java
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
3 files changed, 12 insertions(+), 3 deletions(-)

Approvals:
  Jenkins: Verified; Verified
  Anon. E. Moose (1000171):
  Ali Alsuliman: Looks good to me, approved

Objections:
  Jenkins: Violations found



diff --git 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/NCQueryServiceServlet.java
 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/NCQueryServiceServlet.java
index 7d7bad2..254f92a 100644
--- 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/NCQueryServiceServlet.java
+++ 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/NCQueryServiceServlet.java
@@ -119,7 +119,7 @@
 }
 if (hasResult(responseMsg)) {
 responsePrinter.addResultPrinter(
-new NcResultPrinter(appCtx, responseMsg, getResultSet(), 
delivery, sessionOutput));
+new NcResultPrinter(appCtx, responseMsg, getResultSet(), 
delivery, sessionOutput, stats));
 }
 buildResponseResults(responsePrinter, sessionOutput, 
responseMsg.getExecutionPlans());
 }
diff --git 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/NcResultPrinter.java
 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/NcResultPrinter.java
index 2989d2c..000dd6a 100644
--- 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/NcResultPrinter.java
+++ 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/NcResultPrinter.java
@@ -43,14 +43,17 @@
 private final IApplicationContext appCtx;
 private final IResultSet resultSet;
 private final SessionOutput sessionOutput;
+private final IStatementExecutor.Stats stats;

 public NcResultPrinter(IApplicationContext appCtx, 
ExecuteStatementResponseMessage responseMsg,
-IResultSet resultSet, IStatementExecutor.ResultDelivery delivery, 
SessionOutput sessionOutput) {
+IResultSet resultSet, IStatementExecutor.ResultDelivery delivery, 
SessionOutput sessionOutput,
+IStatementExecutor.Stats stats) {
 this.appCtx = appCtx;
 this.responseMsg = responseMsg;
 this.delivery = delivery;
 this.resultSet = resultSet;
 this.sessionOutput = sessionOutput;
+this.stats = stats;
 }

 @Override
@@ -58,7 +61,6 @@
 IStatementExecutor.ResultMetadata resultMetadata = 
responseMsg.getMetadata();
 List> resultSets = 
resultMetadata.getResultSets();
 if (delivery == IStatementExecutor.ResultDelivery.IMMEDIATE && 
!resultSets.isEmpty()) {
-final IStatementExecutor.Stats stats = responseMsg.getStats();
 
stats.setProcessedObjects(responseMsg.getStats().getProcessedObjects());
 for (int i = 0; i < resultSets.size(); i++) {
 Triple rsmd = 
resultSets.get(i);
diff --git 
a/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml 
b/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
index 15c0702..7191d03 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
@@ -12342,4 +12342,11 @@
   
 
   
+  
+
+  
+full-scan
+  
+
+  
 

--
To view, visit https://asterix-gerrit.ics.uci.edu/3444
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I54c3b3e72cfb0c2499a95bfc0f514dd5d6774293
Gerrit-Change-Number: 3444
Gerrit-PatchSet: 3
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. 

Change in asterixdb[master]: [NO ISSUE][MTD] Fix Reading Dataset Compression From Dataset Record

[Murtadha Hubail (Code Review)]

Murtadha Hubail has submitted this change and it was merged. ( 
https://asterix-gerrit.ics.uci.edu/3445 )

Change subject: [NO ISSUE][MTD] Fix Reading Dataset Compression From Dataset 
Record
..

[NO ISSUE][MTD] Fix Reading Dataset Compression From Dataset Record

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Fix reading the dataset compression scheme by reading its
  value from the compression record rather than the dataset record.

Change-Id: I3bdb1f64143fed43dc9eb37cf8a2fadbfd37dfde
Reviewed-on: https://asterix-gerrit.ics.uci.edu/3445
Contrib: Jenkins 
Tested-by: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Murtadha Hubail 
Reviewed-by: Wail Alkowaileet 
---
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entitytupletranslators/DatasetTupleTranslator.java
1 file changed, 2 insertions(+), 1 deletion(-)

Approvals:
  Jenkins: Verified; ; Verified
  Murtadha Hubail: Looks good to me, but someone else must approve
  Wail Alkowaileet: Looks good to me, approved

Objections:
  Jenkins: Violations found



diff --git 
a/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entitytupletranslators/DatasetTupleTranslator.java
 
b/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entitytupletranslators/DatasetTupleTranslator.java
index 27978ab..7f8b9bf6 100644
--- 
a/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entitytupletranslators/DatasetTupleTranslator.java
+++ 
b/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entitytupletranslators/DatasetTupleTranslator.java
@@ -282,7 +282,8 @@
 final ARecordType compressionType = (ARecordType) 
datasetType.getFieldTypes()[compressionIndex];
 final int schemeIndex = compressionType
 
.getFieldIndex(MetadataRecordTypes.DATASET_ARECORD_DATASET_COMPRESSION_SCHEME_FIELD_NAME);
-return ((AString) 
datasetRecord.getValueByPos(schemeIndex)).getStringValue();
+final ARecord compressionRecord = (ARecord) 
datasetRecord.getValueByPos(compressionIndex);
+return ((AString) 
compressionRecord.getValueByPos(schemeIndex)).getStringValue();
 }
 return CompressionManager.NONE;
 }

--
To view, visit https://asterix-gerrit.ics.uci.edu/3445
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I3bdb1f64143fed43dc9eb37cf8a2fadbfd37dfde
Gerrit-Change-Number: 3445
Gerrit-PatchSet: 2
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Reviewer: Wail Alkowaileet 

Change in asterixdb[stabilization-f69489]: [NO ISSUE][REPL] Wait For Dataset IO Before Replica Sync

[Murtadha Hubail (Code Review)]

Murtadha Hubail has posted comments on this change. ( 
https://asterix-gerrit.ics.uci.edu/3446 )

Change subject: [NO ISSUE][REPL] Wait For Dataset IO Before Replica Sync
..


Patch Set 2: Code-Review-2

Holding off the merge of this change


--
To view, visit https://asterix-gerrit.ics.uci.edu/3446
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: stabilization-f69489
Gerrit-MessageType: comment
Gerrit-Change-Id: I01ed5c9379cf7ae249faeef624d5226ea699cf22
Gerrit-Change-Number: 3446
Gerrit-PatchSet: 2
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Comment-Date: Mon, 17 Jun 2019 20:44:25 +
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE] Increase Result Set TTL to 30 mins

[Murtadha Hubail (Code Review)]

Murtadha Hubail has posted comments on this change. ( 
https://asterix-gerrit.ics.uci.edu/3447 )

Change subject: [NO ISSUE] Increase Result Set TTL to 30 mins
..


Patch Set 1: Code-Review+2


--
To view, visit https://asterix-gerrit.ics.uci.edu/3447
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I352924a865a2bd93f7254afaa0855e2caa715e39
Gerrit-Change-Number: 3447
Gerrit-PatchSet: 1
Gerrit-Owner: Hussain Towaileb 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Hussain Towaileb 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Comment-Date: Mon, 17 Jun 2019 13:25:26 +
Gerrit-HasComments: No

Change in asterixdb[stabilization-f69489]: [NO ISSUE][REPL] Wait For Dataset IO Before Replica Sync

[Murtadha Hubail (Code Review)]

Murtadha Hubail has posted comments on this change. ( 
https://asterix-gerrit.ics.uci.edu/3446 )

Change subject: [NO ISSUE][REPL] Wait For Dataset IO Before Replica Sync
..


Patch Set 2: Code-Review+1


--
To view, visit https://asterix-gerrit.ics.uci.edu/3446
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: stabilization-f69489
Gerrit-MessageType: comment
Gerrit-Change-Id: I01ed5c9379cf7ae249faeef624d5226ea699cf22
Gerrit-Change-Number: 3446
Gerrit-PatchSet: 2
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Comment-Date: Mon, 17 Jun 2019 06:58:09 +
Gerrit-HasComments: No

Change in asterixdb[stabilization-f69489]: [NO ISSUE][REPL] Wait For Dataset IO Before Replica Sync

[Murtadha Hubail (Code Review)]

Murtadha Hubail has posted comments on this change. ( 
https://asterix-gerrit.ics.uci.edu/3446 )

Change subject: [NO ISSUE][REPL] Wait For Dataset IO Before Replica Sync
..


Patch Set 1:

(2 comments)

https://asterix-gerrit.ics.uci.edu/#/c/3446/1/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/api/IDatasetLifecycleManager.java
File 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/api/IDatasetLifecycleManager.java:

https://asterix-gerrit.ics.uci.edu/#/c/3446/1/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/api/IDatasetLifecycleManager.java@138
PS1, Line 138: on-going
> ongoing
Done


https://asterix-gerrit.ics.uci.edu/#/c/3446/1/asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/sync/ReplicaSynchronizer.java
File 
asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/sync/ReplicaSynchronizer.java:

https://asterix-gerrit.ics.uci.edu/#/c/3446/1/asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/sync/ReplicaSynchronizer.java@84
PS1, Line 84: 
appCtx.getDatasetLifecycleManager().waitForIO(replStrategy);
> what prevents new IO from starting after ensuring that any ongoing operatio
replica sychronization is done when the cluster isn't active, so no new 
operations will be performed that might trigger IO operations. This change 
waits for ongoing IO before the initial sync, then waits for any merge 
operations that might happen due to the flush triggered in line 60 before 
completing the sync.



--
To view, visit https://asterix-gerrit.ics.uci.edu/3446
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: stabilization-f69489
Gerrit-MessageType: comment
Gerrit-Change-Id: I01ed5c9379cf7ae249faeef624d5226ea699cf22
Gerrit-Change-Number: 3446
Gerrit-PatchSet: 1
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Comment-Date: Sun, 16 Jun 2019 23:16:17 +
Gerrit-HasComments: Yes

Change in asterixdb[stabilization-f69489]: [NO ISSUE][REPL] Wait For Dataset IO Before Replica Sync

[Murtadha Hubail (Code Review)]

Hello Anon. E. Moose (1000171), Jenkins, Michael Blow,

I'd like you to reexamine a change. Please visit

https://asterix-gerrit.ics.uci.edu/3446

to look at the new patch set (#2).

Change subject: [NO ISSUE][REPL] Wait For Dataset IO Before Replica Sync
..

[NO ISSUE][REPL] Wait For Dataset IO Before Replica Sync

- user model changes: no
- storage format changes: no
- interface changes: yes

Details:
- Add API in DatasetLifecycleManager to wait for IO on datasets
  matching a replication strategy.
- Before synchronizing the files on a replica, wait for any
  on-going IO operations on replicated datasets to make sure
  the set of files won't change while the replica is being
  synchronized (e.g. a merge operation won't delete a file
  that we wanted to send to a replica).

Change-Id: I01ed5c9379cf7ae249faeef624d5226ea699cf22
---
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/api/IDatasetLifecycleManager.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
M 
asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/sync/ReplicaSynchronizer.java
3 files changed, 25 insertions(+), 0 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/46/3446/2
--
To view, visit https://asterix-gerrit.ics.uci.edu/3446
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: stabilization-f69489
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I01ed5c9379cf7ae249faeef624d5226ea699cf22
Gerrit-Change-Number: 3446
Gerrit-PatchSet: 2
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 

Change in asterixdb[master]: [NO ISSUE][MTD] Fix Reading Dataset Compression From Dataset Record

[Murtadha Hubail (Code Review)]

Murtadha Hubail has removed a vote on this change.

Change subject: [NO ISSUE][MTD] Fix Reading Dataset Compression From Dataset 
Record
..


Removed Contrib-2 by Anon. E. Moose (1000171)
--
To view, visit https://asterix-gerrit.ics.uci.edu/3445
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: I3bdb1f64143fed43dc9eb37cf8a2fadbfd37dfde
Gerrit-Change-Number: 3445
Gerrit-PatchSet: 1
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Reviewer: Wail Alkowaileet 

Change in asterixdb[master]: [NO ISSUE][MTD] Fix Reading Dataset Compression From Dataset Record

[Murtadha Hubail (Code Review)]

Murtadha Hubail has posted comments on this change. ( 
https://asterix-gerrit.ics.uci.edu/3445 )

Change subject: [NO ISSUE][MTD] Fix Reading Dataset Compression From Dataset 
Record
..


Patch Set 1: Code-Review+1


--
To view, visit https://asterix-gerrit.ics.uci.edu/3445
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3bdb1f64143fed43dc9eb37cf8a2fadbfd37dfde
Gerrit-Change-Number: 3445
Gerrit-PatchSet: 1
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Comment-Date: Sun, 16 Jun 2019 02:00:44 +
Gerrit-HasComments: No

Change in asterixdb[stabilization-f69489]: [NO ISSUE][REPL] Wait For Dataset IO Before Replica Sync

[Murtadha Hubail (Code Review)]

Murtadha Hubail has uploaded this change for review. ( 
https://asterix-gerrit.ics.uci.edu/3446


Change subject: [NO ISSUE][REPL] Wait For Dataset IO Before Replica Sync
..

[NO ISSUE][REPL] Wait For Dataset IO Before Replica Sync

- user model changes: no
- storage format changes: no
- interface changes: yes

Details:
- Add API in DatasetLifecycleManager to wait for IO on datasets
  matching a replication strategy,
- Before synchronizing the files on a replica, wait for any
  on-going IO operations on replicated datasets to make sure
  the set of files won't change while the replica is being
  synchronized (e.g. a merge operation won't delete a file
  that we wanted to send to a replica).

Change-Id: I01ed5c9379cf7ae249faeef624d5226ea699cf22
---
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/api/IDatasetLifecycleManager.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
M 
asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/sync/ReplicaSynchronizer.java
3 files changed, 25 insertions(+), 0 deletions(-)



  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/46/3446/1

diff --git 
a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/api/IDatasetLifecycleManager.java
 
b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/api/IDatasetLifecycleManager.java
index d18b6ab..2dfd32b 100644
--- 
a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/api/IDatasetLifecycleManager.java
+++ 
b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/api/IDatasetLifecycleManager.java
@@ -133,4 +133,12 @@
  * @throws HyracksDataException
  */
 void flushDataset(IReplicationStrategy replicationStrategy) throws 
HyracksDataException;
+
+/**
+ * Waits for all on-going IO operations on all open datasets that are 
matching {@code replicationStrategy}.
+ *
+ * @param replicationStrategy
+ * @throws HyracksDataException
+ */
+void waitForIO(IReplicationStrategy replicationStrategy) throws 
HyracksDataException;
 }
diff --git 
a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
 
b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
index d767219..30b27d8 100644
--- 
a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
+++ 
b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
@@ -599,6 +599,15 @@
 }
 }

+@Override
+public void waitForIO(IReplicationStrategy replicationStrategy) throws 
HyracksDataException {
+for (DatasetResource dsr : datasets.values()) {
+if (dsr.isOpen() && 
replicationStrategy.isMatch(dsr.getDatasetID())) {
+dsr.getDatasetInfo().waitForIO();
+}
+}
+}
+
 private void closeIndex(IndexInfo indexInfo) throws HyracksDataException {
 if (indexInfo.isOpen()) {
 ILSMOperationTracker opTracker = 
indexInfo.getIndex().getOperationTracker();
diff --git 
a/asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/sync/ReplicaSynchronizer.java
 
b/asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/sync/ReplicaSynchronizer.java
index 09f1205..0f0b5bd 100644
--- 
a/asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/sync/ReplicaSynchronizer.java
+++ 
b/asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/sync/ReplicaSynchronizer.java
@@ -53,10 +53,12 @@

 private void syncFiles() throws IOException {
 final ReplicaFilesSynchronizer fileSync = new 
ReplicaFilesSynchronizer(appCtx, replica);
+waitForReplicatedDatasetsIO();
 fileSync.sync();
 // flush replicated dataset to generate disk component for any 
remaining in-memory components
 final IReplicationStrategy replStrategy = 
appCtx.getReplicationManager().getReplicationStrategy();
 appCtx.getDatasetLifecycleManager().flushDataset(replStrategy);
+waitForReplicatedDatasetsIO();
 // sync any newly generated files
 fileSync.sync();
 }
@@ -75,4 +77,10 @@
 (PersistentLocalResourceRepository) 
appCtx.getLocalResourceRepository();
 return 
localResourceRepository.getReplicatedIndexesMaxComponentId(partition, 
replStrategy);
 }
+
+private void waitForReplicatedDatasetsIO() throws HyracksDataException {
+// wait for IO operations to ensure replicated datasets files won't 
change during replica sync
+final IReplicationStrategy replStrategy = 
appCtx.getReplicationManager().getReplicationStrategy();
+appCtx.getDatasetLifecycleManager().waitForIO(replStrategy);
+}
 }

--
To view, 

Change in asterixdb[master]: [NO ISSUE][MTD] Fix Reading Dataset Compression From Dataset Record

[Murtadha Hubail (Code Review)]

Murtadha Hubail has uploaded this change for review. ( 
https://asterix-gerrit.ics.uci.edu/3445


Change subject: [NO ISSUE][MTD] Fix Reading Dataset Compression From Dataset 
Record
..

[NO ISSUE][MTD] Fix Reading Dataset Compression From Dataset Record

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Fix reading the dataset compression scheme by reading its
  value from the compression record rather than the dataset record.

Change-Id: I3bdb1f64143fed43dc9eb37cf8a2fadbfd37dfde
---
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entitytupletranslators/DatasetTupleTranslator.java
1 file changed, 2 insertions(+), 1 deletion(-)



  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/45/3445/1

diff --git 
a/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entitytupletranslators/DatasetTupleTranslator.java
 
b/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entitytupletranslators/DatasetTupleTranslator.java
index 27978ab..7f8b9bf6 100644
--- 
a/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entitytupletranslators/DatasetTupleTranslator.java
+++ 
b/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/entitytupletranslators/DatasetTupleTranslator.java
@@ -282,7 +282,8 @@
 final ARecordType compressionType = (ARecordType) 
datasetType.getFieldTypes()[compressionIndex];
 final int schemeIndex = compressionType
 
.getFieldIndex(MetadataRecordTypes.DATASET_ARECORD_DATASET_COMPRESSION_SCHEME_FIELD_NAME);
-return ((AString) 
datasetRecord.getValueByPos(schemeIndex)).getStringValue();
+final ARecord compressionRecord = (ARecord) 
datasetRecord.getValueByPos(compressionIndex);
+return ((AString) 
compressionRecord.getValueByPos(schemeIndex)).getStringValue();
 }
 return CompressionManager.NONE;
 }

--
To view, visit https://asterix-gerrit.ics.uci.edu/3445
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I3bdb1f64143fed43dc9eb37cf8a2fadbfd37dfde
Gerrit-Change-Number: 3445
Gerrit-PatchSet: 1
Gerrit-Owner: Murtadha Hubail 

Change in asterixdb[master]: [NO ISSUE][OTH] Use Request Stats When Printing Results on NC

[Murtadha Hubail (Code Review)]

Murtadha Hubail has posted comments on this change. ( 
https://asterix-gerrit.ics.uci.edu/3444 )

Change subject: [NO ISSUE][OTH] Use Request Stats When Printing Results on NC
..


Patch Set 1: Code-Review+1


--
To view, visit https://asterix-gerrit.ics.uci.edu/3444
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I54c3b3e72cfb0c2499a95bfc0f514dd5d6774293
Gerrit-Change-Number: 3444
Gerrit-PatchSet: 1
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Comment-Date: Sat, 15 Jun 2019 11:12:23 +
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE][OTH] Use Request Stats When Printing Results on NC

[Murtadha Hubail (Code Review)]

Murtadha Hubail has uploaded this change for review. ( 
https://asterix-gerrit.ics.uci.edu/3444


Change subject: [NO ISSUE][OTH] Use Request Stats When Printing Results on NC
..

[NO ISSUE][OTH] Use Request Stats When Printing Results on NC

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- When printing a result on an NC, use the stats object
  of the original request rather than the object received
  from the CC as part of the execute statement response
  message. This is done because the metrics returned to the
  user are generated from the stats object of the request.
- Add metrics test case to sqlpp test suite to test returning
  metrics from NCQueryServiceServlet.

Change-Id: I54c3b3e72cfb0c2499a95bfc0f514dd5d6774293
---
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/NCQueryServiceServlet.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/NcResultPrinter.java
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
3 files changed, 12 insertions(+), 3 deletions(-)



  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/44/3444/1

diff --git 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/NCQueryServiceServlet.java
 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/NCQueryServiceServlet.java
index 7d7bad2..254f92a 100644
--- 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/NCQueryServiceServlet.java
+++ 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/NCQueryServiceServlet.java
@@ -119,7 +119,7 @@
 }
 if (hasResult(responseMsg)) {
 responsePrinter.addResultPrinter(
-new NcResultPrinter(appCtx, responseMsg, getResultSet(), 
delivery, sessionOutput));
+new NcResultPrinter(appCtx, responseMsg, getResultSet(), 
delivery, sessionOutput, stats));
 }
 buildResponseResults(responsePrinter, sessionOutput, 
responseMsg.getExecutionPlans());
 }
diff --git 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/NcResultPrinter.java
 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/NcResultPrinter.java
index 2989d2c..000dd6a 100644
--- 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/NcResultPrinter.java
+++ 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/NcResultPrinter.java
@@ -43,14 +43,17 @@
 private final IApplicationContext appCtx;
 private final IResultSet resultSet;
 private final SessionOutput sessionOutput;
+private final IStatementExecutor.Stats stats;

 public NcResultPrinter(IApplicationContext appCtx, 
ExecuteStatementResponseMessage responseMsg,
-IResultSet resultSet, IStatementExecutor.ResultDelivery delivery, 
SessionOutput sessionOutput) {
+IResultSet resultSet, IStatementExecutor.ResultDelivery delivery, 
SessionOutput sessionOutput,
+IStatementExecutor.Stats stats) {
 this.appCtx = appCtx;
 this.responseMsg = responseMsg;
 this.delivery = delivery;
 this.resultSet = resultSet;
 this.sessionOutput = sessionOutput;
+this.stats = stats;
 }

 @Override
@@ -58,7 +61,6 @@
 IStatementExecutor.ResultMetadata resultMetadata = 
responseMsg.getMetadata();
 List> resultSets = 
resultMetadata.getResultSets();
 if (delivery == IStatementExecutor.ResultDelivery.IMMEDIATE && 
!resultSets.isEmpty()) {
-final IStatementExecutor.Stats stats = responseMsg.getStats();
 
stats.setProcessedObjects(responseMsg.getStats().getProcessedObjects());
 for (int i = 0; i < resultSets.size(); i++) {
 Triple rsmd = 
resultSets.get(i);
diff --git 
a/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml 
b/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
index 9d5d44f..8580d49 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
@@ -12343,4 +12343,11 @@
   
 
   
+  
+
+  
+full-scan
+  
+
+  
 

--
To view, visit https://asterix-gerrit.ics.uci.edu/3444
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I54c3b3e72cfb0c2499a95bfc0f514dd5d6774293
Gerrit-Change-Number: 3444
Gerrit-PatchSet: 1
Gerrit-Owner: Murtadha Hubail 

Change in asterixdb[master]: [NO ISSUE][OTH] Skip Calling Job Result Callback For Unknown Jobs

[Murtadha Hubail (Code Review)]

Murtadha Hubail has submitted this change and it was merged. ( 
https://asterix-gerrit.ics.uci.edu/3439 )

Change subject: [NO ISSUE][OTH] Skip Calling Job Result Callback For Unknown 
Jobs
..

[NO ISSUE][OTH] Skip Calling Job Result Callback For Unknown Jobs

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Skip calling the job result callback on a job's completion
  when the job's result record is not found. This can happen
  if a job's execution time exceeds the result's TTL in which
  case we remove the job's result reference.

Change-Id: I7ba09da4292213b63d2396d21969329e86634c06
Reviewed-on: https://asterix-gerrit.ics.uci.edu/3439
Contrib: Jenkins 
Tested-by: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Murtadha Hubail 
Reviewed-by: Hussain Towaileb 
---
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/result/ResultDirectoryService.java
1 file changed, 4 insertions(+), 1 deletion(-)

Approvals:
  Jenkins: Verified; ; Verified
  Murtadha Hubail: Looks good to me, but someone else must approve
  Hussain Towaileb: Looks good to me, approved

Objections:
  Jenkins: Violations found



diff --git 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/result/ResultDirectoryService.java
 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/result/ResultDirectoryService.java
index bfecc48..9f8a7e2 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/result/ResultDirectoryService.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/result/ResultDirectoryService.java
@@ -96,7 +96,10 @@
 @Override
 public void notifyJobFinish(JobId jobId, JobStatus jobStatus, 
List exceptions) throws HyracksException {
 if (exceptions == null || exceptions.isEmpty()) {
-final ResultJobRecord resultJobRecord = 
getNonNullResultJobRecord(jobId);
+final ResultJobRecord resultJobRecord = getResultJobRecord(jobId);
+if (resultJobRecord == null) {
+return;
+}
 resultJobRecord.finish();
 jobResultCallback.completed(jobId, resultJobRecord);
 }

--
To view, visit https://asterix-gerrit.ics.uci.edu/3439
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I7ba09da4292213b63d2396d21969329e86634c06
Gerrit-Change-Number: 3439
Gerrit-PatchSet: 3
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Hussain Towaileb 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 

Change in asterixdb[master]: [NO ISSUE][OTH] Skip Calling Job Result Callback For Unknown Jobs

[Murtadha Hubail (Code Review)]

Murtadha Hubail has posted comments on this change. ( 
https://asterix-gerrit.ics.uci.edu/3439 )

Change subject: [NO ISSUE][OTH] Skip Calling Job Result Callback For Unknown 
Jobs
..


Patch Set 2: Code-Review+1


--
To view, visit https://asterix-gerrit.ics.uci.edu/3439
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7ba09da4292213b63d2396d21969329e86634c06
Gerrit-Change-Number: 3439
Gerrit-PatchSet: 2
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Comment-Date: Fri, 14 Jun 2019 01:13:24 +
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE][OTH] Skip Calling Job Result Callback For Unknown Jobs

[Murtadha Hubail (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change. Please visit

https://asterix-gerrit.ics.uci.edu/3439

to look at the new patch set (#2).

Change subject: [NO ISSUE][OTH] Skip Calling Job Result Callback For Unknown 
Jobs
..

[NO ISSUE][OTH] Skip Calling Job Result Callback For Unknown Jobs

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Skip calling the job result callback on a job's completion
  when the job's result record is not found. This can happen
  if a job's execution time exceeds the result's TTL in which
  case we remove the job's result reference.

Change-Id: I7ba09da4292213b63d2396d21969329e86634c06
---
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/result/ResultDirectoryService.java
1 file changed, 4 insertions(+), 1 deletion(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/39/3439/2
--
To view, visit https://asterix-gerrit.ics.uci.edu/3439
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7ba09da4292213b63d2396d21969329e86634c06
Gerrit-Change-Number: 3439
Gerrit-PatchSet: 2
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Jenkins 

Change in asterixdb[master]: [NO ISSUE][OTH] Skip Calling Job Result Callback For Unknown Jobs

[Murtadha Hubail (Code Review)]

Murtadha Hubail has uploaded this change for review. ( 
https://asterix-gerrit.ics.uci.edu/3439


Change subject: [NO ISSUE][OTH] Skip Calling Job Result Callback For Unknown 
Jobs
..

[NO ISSUE][OTH] Skip Calling Job Result Callback For Unknown Jobs

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Skip calling the job result callback on a job's completion
  when the job's result record is not found. This can happen
  if a job's execution time exceeds the result's TTL in which
  case we remove the job's result reference.

Change-Id: I7ba09da4292213b63d2396d21969329e86634c06
---
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/result/ResultDirectoryService.java
1 file changed, 5 insertions(+), 2 deletions(-)



  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/39/3439/1

diff --git 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/result/ResultDirectoryService.java
 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/result/ResultDirectoryService.java
index bfecc48..d5d1552 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/result/ResultDirectoryService.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/result/ResultDirectoryService.java
@@ -95,8 +95,11 @@

 @Override
 public void notifyJobFinish(JobId jobId, JobStatus jobStatus, 
List exceptions) throws HyracksException {
-if (exceptions == null || exceptions.isEmpty()) {
-final ResultJobRecord resultJobRecord = 
getNonNullResultJobRecord(jobId);
+if ((exceptions == null || exceptions.isEmpty())) {
+final ResultJobRecord resultJobRecord = getResultJobRecord(jobId);
+if (resultJobRecord == null) {
+return;
+}
 resultJobRecord.finish();
 jobResultCallback.completed(jobId, resultJobRecord);
 }

--
To view, visit https://asterix-gerrit.ics.uci.edu/3439
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I7ba09da4292213b63d2396d21969329e86634c06
Gerrit-Change-Number: 3439
Gerrit-PatchSet: 1
Gerrit-Owner: Murtadha Hubail 

Change in asterixdb[stabilization-f69489]: [NO ISSUE][OTH] Use Absolute Path in FileReference hashCode/equals

[Murtadha Hubail (Code Review)]

Murtadha Hubail has posted comments on this change. ( 
https://asterix-gerrit.ics.uci.edu/3430 )

Change subject: [NO ISSUE][OTH] Use Absolute Path in FileReference 
hashCode/equals
..


Patch Set 1: Code-Review-2

Holding off the merge of this change.


--
To view, visit https://asterix-gerrit.ics.uci.edu/3430
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: stabilization-f69489
Gerrit-MessageType: comment
Gerrit-Change-Id: Idcb642b3c5396a8319dc0aaf050e8af6fca93039
Gerrit-Change-Number: 3430
Gerrit-PatchSet: 1
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Comment-Date: Wed, 12 Jun 2019 20:21:06 +
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE][OTH] Simplify ResultJobRecord APIs

[Murtadha Hubail (Code Review)]

Murtadha Hubail has submitted this change and it was merged. ( 
https://asterix-gerrit.ics.uci.edu/3432 )

Change subject: [NO ISSUE][OTH] Simplify ResultJobRecord APIs
..

[NO ISSUE][OTH] Simplify ResultJobRecord APIs

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Simplify ResultJobRecord APIs by allowing only a single ResultSetId
  per ResultJobRecord (i.e. per job).
- Fail result partition registration when a job attempts to use
  multiple ResultSetIds or inconsistent number of partitions and
  log the inconsistency.
- Delete test ReplicateOperatorTest which duplicates
  the test PushRuntimeTest#scanReplicateWrite.

Change-Id: I37816efc92ee9f5e66f29ce74dec4c6c5bd07c6f
Reviewed-on: https://asterix-gerrit.ics.uci.edu/3432
Contrib: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Murtadha Hubail 
Tested-by: Jenkins 
Reviewed-by: Till Westmann 
---
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/result/ResultJobRecord.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/result/ResultDirectoryService.java
D 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/ReplicateOperatorTest.java
3 files changed, 37 insertions(+), 155 deletions(-)

Approvals:
  Jenkins: Verified; ; Verified
  Anon. E. Moose (1000171):
  Murtadha Hubail: Looks good to me, but someone else must approve
  Till Westmann: Looks good to me, approved

Objections:
  Jenkins: Violations found



diff --git 
a/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/result/ResultJobRecord.java
 
b/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/result/ResultJobRecord.java
index b3b0706..02762ee 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/result/ResultJobRecord.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/result/ResultJobRecord.java
@@ -21,12 +21,12 @@
 import java.io.Serializable;
 import java.util.ArrayList;
 import java.util.Arrays;
-import java.util.HashMap;
 import java.util.List;
-import java.util.Map;

 import org.apache.hyracks.api.exceptions.ErrorCode;
 import org.apache.hyracks.api.exceptions.HyracksDataException;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;

 public class ResultJobRecord implements IResultStateRecord {

@@ -77,15 +77,13 @@
 }

 private static final long serialVersionUID = 1L;
-
+private static final Logger LOGGER = LogManager.getLogger();
 private final long timestamp;
 private long jobStartTime;
 private long jobEndTime;
 private Status status;
-
+private ResultSetId rsId;
 private ResultSetMetaData resultSetMetaData;
-
-private Map resultSetMetadataMap = new 
HashMap<>();

 public ResultJobRecord() {
 this.timestamp = System.nanoTime();
@@ -116,10 +114,6 @@
 updateState(State.SUCCESS);
 }

-public void fail(ResultSetId rsId, int partition) {
-getOrCreateDirectoryRecord(rsId, partition).fail();
-}
-
 public void fail(List exceptions) {
 updateState(State.FAILED);
 status.setExceptions(exceptions);
@@ -139,47 +133,40 @@
 StringBuilder sb = new StringBuilder();
 sb.append("{ \"status\": ").append(status.toString()).append(", ");
 sb.append("\"timestamp\": ").append(timestamp).append(", ");
-sb.append("\"resultsets\": 
").append(Arrays.toString(resultSetMetadataMap.entrySet().toArray())).append(" 
}");
+sb.append("\"resultset\": ").append(resultSetMetaData).append(" }");
 return sb.toString();
 }

 public synchronized void setResultSetMetaData(ResultSetId rsId, 
IResultMetadata metadata, int nPartitions)
 throws HyracksDataException {
-ResultSetMetaData rsMd = resultSetMetadataMap.get(rsId);
-if (rsMd == null) {
-final ResultSetMetaData resultSetMetaData = new 
ResultSetMetaData(nPartitions, metadata);
-resultSetMetadataMap.put(rsId, resultSetMetaData);
-this.resultSetMetaData = resultSetMetaData;
-} else if (rsMd.getRecords().length != nPartitions) {
-throw 
HyracksDataException.create(ErrorCode.INCONSISTENT_RESULT_METADATA, 
rsId.toString());
+if (this.rsId == null) {
+this.rsId = rsId;
+this.resultSetMetaData = new ResultSetMetaData(nPartitions, 
metadata);
+} else if (!this.rsId.equals(rsId) || 
resultSetMetaData.getRecords().length != nPartitions) {
+logInconsistentMetadata(rsId, nPartitions);
+throw 
HyracksDataException.create(ErrorCode.INCONSISTENT_RESULT_METADATA, 
this.rsId.toString());
 }
-//TODO(tillw) throwing a HyracksDataException here hangs the execution 

Change in asterixdb[master]: [NO ISSUE][OTH] Simplify ResultJobRecord APIs

[Murtadha Hubail (Code Review)]

Murtadha Hubail has posted comments on this change. ( 
https://asterix-gerrit.ics.uci.edu/3432 )

Change subject: [NO ISSUE][OTH] Simplify ResultJobRecord APIs
..


Patch Set 2: Code-Review+1


--
To view, visit https://asterix-gerrit.ics.uci.edu/3432
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I37816efc92ee9f5e66f29ce74dec4c6c5bd07c6f
Gerrit-Change-Number: 3432
Gerrit-PatchSet: 2
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Comment-Date: Mon, 10 Jun 2019 17:04:01 +
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE][OTH] Simplify ResultJobRecord APIs

[Murtadha Hubail (Code Review)]

Hello Anon. E. Moose (1000171), Jenkins,

I'd like you to reexamine a change. Please visit

https://asterix-gerrit.ics.uci.edu/3432

to look at the new patch set (#2).

Change subject: [NO ISSUE][OTH] Simplify ResultJobRecord APIs
..

[NO ISSUE][OTH] Simplify ResultJobRecord APIs

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Simplify ResultJobRecord APIs by allowing only a single ResultSetId
  per ResultJobRecord (i.e. per job).
- Fail result partition registration when a job attempts to use
  multiple ResultSetIds or inconsistent number of partitions and
  log the inconsistency.
- Delete test ReplicateOperatorTest which duplicates
  the test PushRuntimeTest#scanReplicateWrite.

Change-Id: I37816efc92ee9f5e66f29ce74dec4c6c5bd07c6f
---
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/result/ResultJobRecord.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/result/ResultDirectoryService.java
D 
hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/integration/ReplicateOperatorTest.java
3 files changed, 37 insertions(+), 155 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/32/3432/2
--
To view, visit https://asterix-gerrit.ics.uci.edu/3432
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I37816efc92ee9f5e66f29ce74dec4c6c5bd07c6f
Gerrit-Change-Number: 3432
Gerrit-PatchSet: 2
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 

Change in asterixdb[master]: [NO ISSUE][OTH] Simplify ResultJobRecord APIs

[Murtadha Hubail (Code Review)]

Murtadha Hubail has uploaded this change for review. ( 
https://asterix-gerrit.ics.uci.edu/3432


Change subject: [NO ISSUE][OTH] Simplify ResultJobRecord APIs
..

[NO ISSUE][OTH] Simplify ResultJobRecord APIs

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Simplify ResultJobRecord APIs by allowing only a single ResultSetId
  per ResultJobRecord (i.e. per job).
- Fail result partition registration when a job attempts to use
  multiple ResultSetIds or inconsistent number of partitions and
  log the inconsistency.
- Remove unused methods.

Change-Id: I37816efc92ee9f5e66f29ce74dec4c6c5bd07c6f
---
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/result/ResultJobRecord.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/result/ResultDirectoryService.java
2 files changed, 37 insertions(+), 39 deletions(-)



  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/32/3432/1

diff --git 
a/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/result/ResultJobRecord.java
 
b/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/result/ResultJobRecord.java
index b3b0706..eebfd9d 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/result/ResultJobRecord.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/result/ResultJobRecord.java
@@ -21,12 +21,12 @@
 import java.io.Serializable;
 import java.util.ArrayList;
 import java.util.Arrays;
-import java.util.HashMap;
 import java.util.List;
-import java.util.Map;
 
 import org.apache.hyracks.api.exceptions.ErrorCode;
 import org.apache.hyracks.api.exceptions.HyracksDataException;
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;

 public class ResultJobRecord implements IResultStateRecord {

@@ -77,15 +77,13 @@
 }

 private static final long serialVersionUID = 1L;
-
+private static final Logger LOGGER = LogManager.getLogger();
 private final long timestamp;
 private long jobStartTime;
 private long jobEndTime;
 private Status status;
-
+private ResultSetId rsId;
 private ResultSetMetaData resultSetMetaData;
-
-private Map resultSetMetadataMap = new 
HashMap<>();

 public ResultJobRecord() {
 this.timestamp = System.nanoTime();
@@ -116,10 +114,6 @@
 updateState(State.SUCCESS);
 }

-public void fail(ResultSetId rsId, int partition) {
-getOrCreateDirectoryRecord(rsId, partition).fail();
-}
-
 public void fail(List exceptions) {
 updateState(State.FAILED);
 status.setExceptions(exceptions);
@@ -139,47 +133,40 @@
 StringBuilder sb = new StringBuilder();
 sb.append("{ \"status\": ").append(status.toString()).append(", ");
 sb.append("\"timestamp\": ").append(timestamp).append(", ");
-sb.append("\"resultsets\": 
").append(Arrays.toString(resultSetMetadataMap.entrySet().toArray())).append(" 
}");
+sb.append("\"resultset\": ").append(resultSetMetaData).append(" }");
 return sb.toString();
 }

 public synchronized void setResultSetMetaData(ResultSetId rsId, 
IResultMetadata metadata, int nPartitions)
 throws HyracksDataException {
-ResultSetMetaData rsMd = resultSetMetadataMap.get(rsId);
-if (rsMd == null) {
-final ResultSetMetaData resultSetMetaData = new 
ResultSetMetaData(nPartitions, metadata);
-resultSetMetadataMap.put(rsId, resultSetMetaData);
-this.resultSetMetaData = resultSetMetaData;
-} else if (rsMd.getRecords().length != nPartitions) {
-throw 
HyracksDataException.create(ErrorCode.INCONSISTENT_RESULT_METADATA, 
rsId.toString());
+if (this.rsId == null) {
+this.rsId = rsId;
+this.resultSetMetaData = new ResultSetMetaData(nPartitions, 
metadata);
+} else if (!this.rsId.equals(rsId) || 
resultSetMetaData.getRecords().length != nPartitions) {
+logInconsistentMetadata(rsId, nPartitions);
+throw 
HyracksDataException.create(ErrorCode.INCONSISTENT_RESULT_METADATA, 
this.rsId.toString());
 }
-//TODO(tillw) throwing a HyracksDataException here hangs the execution 
tests
 }

-public ResultSetMetaData getResultSetMetaData(ResultSetId rsId) {
-return resultSetMetadataMap.get(rsId);
-}
-
-public synchronized ResultDirectoryRecord 
getOrCreateDirectoryRecord(ResultSetId rsId, int partition) {
-ResultDirectoryRecord[] records = 
getResultSetMetaData(rsId).getRecords();
+public synchronized ResultDirectoryRecord getOrCreateDirectoryRecord(int 
partition) {
+ResultDirectoryRecord[] records = resultSetMetaData.getRecords();
 if (records[partition] == null) {
   

Change in asterixdb[master]: [NO ISSUE][OTH] Add API To Get Dataset Size

[Murtadha Hubail (Code Review)]

Murtadha Hubail has submitted this change and it was merged. ( 
https://asterix-gerrit.ics.uci.edu/3425 )

Change subject: [NO ISSUE][OTH] Add API To Get Dataset Size
..

[NO ISSUE][OTH] Add API To Get Dataset Size

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Add an API that returns the on disk total size of a dataset
  and its indexes on a node.

Change-Id: Iaff87bbe1f2417689f7827deaf03fcddd64ca7e4
Reviewed-on: https://asterix-gerrit.ics.uci.edu/3425
Contrib: Jenkins 
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Murtadha Hubail 
Reviewed-by: Michael Blow 
---
A 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/storage/DatasetCopyIdentifier.java
M 
asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/resource/PersistentLocalResourceRepository.java
2 files changed, 99 insertions(+), 0 deletions(-)

Approvals:
  Jenkins: Verified; No violations found; ; Verified
  Murtadha Hubail: Looks good to me, but someone else must approve
  Michael Blow: Looks good to me, approved



diff --git 
a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/storage/DatasetCopyIdentifier.java
 
b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/storage/DatasetCopyIdentifier.java
new file mode 100644
index 000..6500c8a
--- /dev/null
+++ 
b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/storage/DatasetCopyIdentifier.java
@@ -0,0 +1,81 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.asterix.common.storage;
+
+import java.io.Serializable;
+import java.util.Objects;
+
+public class DatasetCopyIdentifier implements Serializable {
+
+private static final long serialVersionUID = 1L;
+private final String dataset;
+private final String dataverse;
+private final String rebalance;
+
+private DatasetCopyIdentifier(String dataverse, String datasetName, String 
rebalance) {
+this.dataverse = dataverse;
+this.dataset = datasetName;
+this.rebalance = rebalance;
+}
+
+public static DatasetCopyIdentifier of(String dataverse, String 
datasetName, String rebalance) {
+return new DatasetCopyIdentifier(dataverse, datasetName, rebalance);
+}
+
+public String getDataset() {
+return dataset;
+}
+
+public String getRebalance() {
+return rebalance;
+}
+
+@Override
+public boolean equals(Object o) {
+if (this == o) {
+return true;
+}
+if (o == null || getClass() != o.getClass()) {
+return false;
+}
+DatasetCopyIdentifier that = (DatasetCopyIdentifier) o;
+return Objects.equals(dataverse, that.dataverse) && 
Objects.equals(dataset, that.dataset)
+&& Objects.equals(rebalance, that.rebalance);
+}
+
+@Override
+public int hashCode() {
+return Objects.hash(dataverse, dataset, rebalance);
+}
+
+public String getDataverse() {
+return dataverse;
+}
+
+public boolean isMatch(ResourceReference resourceReference) {
+return resourceReference.getDataverse().equals(dataverse) && 
resourceReference.getDataset().equals(dataset)
+&& resourceReference.getRebalance().equals(rebalance);
+}
+
+@Override
+public String toString() {
+return "DatasetCopyIdentifier{" + "dataset='" + dataset + '\'' + ", 
dataverse='" + dataverse + '\''
++ ", rebalance='" + rebalance + '\'' + '}';
+}
+}
\ No newline at end of file
diff --git 
a/asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/resource/PersistentLocalResourceRepository.java
 
b/asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/resource/PersistentLocalResourceRepository.java
index aef7bbd..e170779 100644
--- 
a/asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/resource/PersistentLocalResourceRepository.java
+++ 

Change in asterixdb[master]: [NO ISSUE][OTH] Add API To Get Dataset Size

[Murtadha Hubail (Code Review)]

Murtadha Hubail has removed a vote on this change.

Change subject: [NO ISSUE][OTH] Add API To Get Dataset Size
..


Removed Contrib-2 by Anon. E. Moose (1000171)
--
To view, visit https://asterix-gerrit.ics.uci.edu/3425
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: Iaff87bbe1f2417689f7827deaf03fcddd64ca7e4
Gerrit-Change-Number: 3425
Gerrit-PatchSet: 3
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Reviewer: Till Westmann 

Change in asterixdb[master]: [NO ISSUE][COMP] Better error messages for dump_index

[Murtadha Hubail (Code Review)]

Murtadha Hubail has posted comments on this change. ( 
https://asterix-gerrit.ics.uci.edu/3431 )

Change subject: [NO ISSUE][COMP] Better error messages for dump_index
..


Patch Set 1: Code-Review+2


--
To view, visit https://asterix-gerrit.ics.uci.edu/3431
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I87d75f7264c1eecf178300a1b0a67b351cce0216
Gerrit-Change-Number: 3431
Gerrit-PatchSet: 1
Gerrit-Owner: Till Westmann 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Reviewer: Till Westmann 
Gerrit-Comment-Date: Sat, 08 Jun 2019 01:51:00 +
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE][OTH] Add API To Get Dataset Size

[Murtadha Hubail (Code Review)]

Murtadha Hubail has posted comments on this change. ( 
https://asterix-gerrit.ics.uci.edu/3425 )

Change subject: [NO ISSUE][OTH] Add API To Get Dataset Size
..


Patch Set 3: Code-Review+1


--
To view, visit https://asterix-gerrit.ics.uci.edu/3425
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iaff87bbe1f2417689f7827deaf03fcddd64ca7e4
Gerrit-Change-Number: 3425
Gerrit-PatchSet: 3
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Reviewer: Till Westmann 
Gerrit-Comment-Date: Sat, 08 Jun 2019 01:46:05 +
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE][OTH] Add API To Get Dataset Size

[Murtadha Hubail (Code Review)]

Hello Anon. E. Moose (1000171), Till Westmann, Jenkins, Michael Blow,

I'd like you to reexamine a change. Please visit

https://asterix-gerrit.ics.uci.edu/3425

to look at the new patch set (#3).

Change subject: [NO ISSUE][OTH] Add API To Get Dataset Size
..

[NO ISSUE][OTH] Add API To Get Dataset Size

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Add an API that returns the on disk total size of a dataset
  and its indexes on a node.

Change-Id: Iaff87bbe1f2417689f7827deaf03fcddd64ca7e4
---
A 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/storage/DatasetCopyIdentifier.java
M 
asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/resource/PersistentLocalResourceRepository.java
2 files changed, 99 insertions(+), 0 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/25/3425/3
--
To view, visit https://asterix-gerrit.ics.uci.edu/3425
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iaff87bbe1f2417689f7827deaf03fcddd64ca7e4
Gerrit-Change-Number: 3425
Gerrit-PatchSet: 3
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Reviewer: Till Westmann 

Change in asterixdb[master]: [NO ISSUE][OTH] Add API To Get Dataset Size

[Murtadha Hubail (Code Review)]

Murtadha Hubail has posted comments on this change. ( 
https://asterix-gerrit.ics.uci.edu/3425 )

Change subject: [NO ISSUE][OTH] Add API To Get Dataset Size
..


Patch Set 2:

(1 comment)

https://asterix-gerrit.ics.uci.edu/#/c/3425/2/asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/resource/PersistentLocalResourceRepository.java
File 
asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/resource/PersistentLocalResourceRepository.java:

https://asterix-gerrit.ics.uci.edu/#/c/3425/2/asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/resource/PersistentLocalResourceRepository.java@616
PS2, Line 616:
 : private static boolean isMatch(DatasetCopyIdentifier 
datasetIdentifier, ResourceReference resourceReference) {
 : return 
resourceReference.getDataverse().equals(datasetIdentifier.getDataverse())
 : && 
resourceReference.getDataset().equals(datasetIdentifier.getDataset())
 : && 
resourceReference.getRebalance().equals(datasetIdentifier.getRebalance());
 : }
> remove
Done



--
To view, visit https://asterix-gerrit.ics.uci.edu/3425
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iaff87bbe1f2417689f7827deaf03fcddd64ca7e4
Gerrit-Change-Number: 3425
Gerrit-PatchSet: 2
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Reviewer: Till Westmann 
Gerrit-Comment-Date: Sat, 08 Jun 2019 00:23:49 +
Gerrit-HasComments: Yes

Change in asterixdb[master]: [NO ISSUE][OTH] Add API To Get Dataset Size

[Murtadha Hubail (Code Review)]

Murtadha Hubail has posted comments on this change. ( 
https://asterix-gerrit.ics.uci.edu/3425 )

Change subject: [NO ISSUE][OTH] Add API To Get Dataset Size
..


Patch Set 1:

(1 comment)

https://asterix-gerrit.ics.uci.edu/#/c/3425/1/asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/resource/PersistentLocalResourceRepository.java
File 
asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/resource/PersistentLocalResourceRepository.java:

https://asterix-gerrit.ics.uci.edu/#/c/3425/1/asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/resource/PersistentLocalResourceRepository.java@618
PS1, Line 618: return 
resourceReference.getDataverse().equals(datasetIdentifier.getDataverse())
 : && 
resourceReference.getDataset().equals(datasetIdentifier.getDataset())
 : && 
resourceReference.getRebalance().equals(datasetIdentifier.getRebalance());
> should we locate this logic inside of the DatasetCopyIdentifier, to avoid i
Done



--
To view, visit https://asterix-gerrit.ics.uci.edu/3425
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iaff87bbe1f2417689f7827deaf03fcddd64ca7e4
Gerrit-Change-Number: 3425
Gerrit-PatchSet: 1
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Reviewer: Till Westmann 
Gerrit-Comment-Date: Fri, 07 Jun 2019 16:48:21 +
Gerrit-HasComments: Yes

Change in asterixdb[master]: [NO ISSUE][OTH] Add API To Get Dataset Size

[Murtadha Hubail (Code Review)]

Hello Anon. E. Moose (1000171), Till Westmann, Jenkins, Michael Blow,

I'd like you to reexamine a change. Please visit

https://asterix-gerrit.ics.uci.edu/3425

to look at the new patch set (#2).

Change subject: [NO ISSUE][OTH] Add API To Get Dataset Size
..

[NO ISSUE][OTH] Add API To Get Dataset Size

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Add an API that returns the on disk total size of a dataset
  and its indexes on a node.

Change-Id: Iaff87bbe1f2417689f7827deaf03fcddd64ca7e4
---
A 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/storage/DatasetCopyIdentifier.java
M 
asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/resource/PersistentLocalResourceRepository.java
2 files changed, 105 insertions(+), 0 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/25/3425/2
--
To view, visit https://asterix-gerrit.ics.uci.edu/3425
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iaff87bbe1f2417689f7827deaf03fcddd64ca7e4
Gerrit-Change-Number: 3425
Gerrit-PatchSet: 2
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Reviewer: Till Westmann 

Change in asterixdb[master]: [NO ISSUE][OTH] Add API To Get Dataset Size

[Murtadha Hubail (Code Review)]

Murtadha Hubail has posted comments on this change. ( 
https://asterix-gerrit.ics.uci.edu/3425 )

Change subject: [NO ISSUE][OTH] Add API To Get Dataset Size
..


Patch Set 1: Code-Review+1


--
To view, visit https://asterix-gerrit.ics.uci.edu/3425
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iaff87bbe1f2417689f7827deaf03fcddd64ca7e4
Gerrit-Change-Number: 3425
Gerrit-PatchSet: 1
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Reviewer: Till Westmann 
Gerrit-Comment-Date: Fri, 07 Jun 2019 15:07:52 +
Gerrit-HasComments: No

Change in asterixdb[stabilization-f69489]: [NO ISSUE][OTH] Use Absolute Path in FileReference hashCode/equals

[Murtadha Hubail (Code Review)]

Murtadha Hubail has uploaded this change for review. ( 
https://asterix-gerrit.ics.uci.edu/3430


Change subject: [NO ISSUE][OTH] Use Absolute Path in FileReference 
hashCode/equals
..

[NO ISSUE][OTH] Use Absolute Path in FileReference hashCode/equals

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- To avoid cases where unintentional false comparison result happens
  due to a relative path of one FileReference starting with "/" and
  the other's does not, use the file absolute path in hashCode and
  equals.

Change-Id: Idcb642b3c5396a8319dc0aaf050e8af6fca93039
---
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/io/FileReference.java
1 file changed, 2 insertions(+), 2 deletions(-)



  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/30/3430/1

diff --git 
a/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/io/FileReference.java
 
b/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/io/FileReference.java
index 4ded855..e4097ff 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/io/FileReference.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/io/FileReference.java
@@ -58,12 +58,12 @@
 if (!(o instanceof FileReference)) {
 return false;
 }
-return path.equals(((FileReference) o).path) && 
dev.equals(((FileReference) o).dev);
+return file.getAbsolutePath().equals(((FileReference) 
o).getAbsolutePath());
 }

 @Override
 public int hashCode() {
-return path.hashCode();
+return file.getAbsolutePath().hashCode();
 }

 /**

--
To view, visit https://asterix-gerrit.ics.uci.edu/3430
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: stabilization-f69489
Gerrit-MessageType: newchange
Gerrit-Change-Id: Idcb642b3c5396a8319dc0aaf050e8af6fca93039
Gerrit-Change-Number: 3430
Gerrit-PatchSet: 1
Gerrit-Owner: Murtadha Hubail 

Change in asterixdb[master]: [NO ISSUE][OTH] Add API To Get Dataset Size

[Murtadha Hubail (Code Review)]

Murtadha Hubail has uploaded this change for review. ( 
https://asterix-gerrit.ics.uci.edu/3425


Change subject: [NO ISSUE][OTH] Add API To Get Dataset Size
..

[NO ISSUE][OTH] Add API To Get Dataset Size

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Add an API that returns the on disk total size of a dataset
  and its indexes on a node.

Change-Id: Iaff87bbe1f2417689f7827deaf03fcddd64ca7e4
---
A 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/storage/DatasetCopyIdentifier.java
M 
asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/resource/PersistentLocalResourceRepository.java
2 files changed, 100 insertions(+), 0 deletions(-)



  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/25/3425/1

diff --git 
a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/storage/DatasetCopyIdentifier.java
 
b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/storage/DatasetCopyIdentifier.java
new file mode 100644
index 000..bc91fe4
--- /dev/null
+++ 
b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/storage/DatasetCopyIdentifier.java
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.asterix.common.storage;
+
+import java.io.Serializable;
+import java.util.Objects;
+
+public class DatasetCopyIdentifier implements Serializable {
+
+private static final long serialVersionUID = 1L;
+private final String dataset;
+private final String dataverse;
+private final String rebalance;
+
+private DatasetCopyIdentifier(String dataverse, String datasetName, String 
rebalance) {
+this.dataverse = dataverse;
+this.dataset = datasetName;
+this.rebalance = rebalance;
+}
+
+public static DatasetCopyIdentifier of(String dataverse, String 
datasetName, String rebalance) {
+return new DatasetCopyIdentifier(dataverse, datasetName, rebalance);
+}
+
+public String getDataset() {
+return dataset;
+}
+
+public String getRebalance() {
+return rebalance;
+}
+
+@Override
+public boolean equals(Object o) {
+if (this == o) {
+return true;
+}
+if (o == null || getClass() != o.getClass()) {
+return false;
+}
+DatasetCopyIdentifier that = (DatasetCopyIdentifier) o;
+return Objects.equals(dataverse, that.dataverse) && 
Objects.equals(dataset, that.dataset)
+&& Objects.equals(rebalance, that.rebalance);
+}
+
+@Override
+public int hashCode() {
+return Objects.hash(dataverse, dataset, rebalance);
+}
+
+public String getDataverse() {
+return dataverse;
+}
+
+@Override
+public String toString() {
+return "DatasetCopyIdentifier{" + "dataset='" + dataset + '\'' + ", 
dataverse='" + dataverse + '\''
++ ", rebalance='" + rebalance + '\'' + '}';
+}
+}
\ No newline at end of file
diff --git 
a/asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/resource/PersistentLocalResourceRepository.java
 
b/asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/resource/PersistentLocalResourceRepository.java
index aef7bbd..d6efa2b 100644
--- 
a/asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/resource/PersistentLocalResourceRepository.java
+++ 
b/asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/resource/PersistentLocalResourceRepository.java
@@ -47,6 +47,7 @@
 import org.apache.asterix.common.replication.IReplicationManager;
 import org.apache.asterix.common.replication.IReplicationStrategy;
 import org.apache.asterix.common.replication.ReplicationJob;
+import org.apache.asterix.common.storage.DatasetCopyIdentifier;
 import org.apache.asterix.common.storage.DatasetResourceReference;
 import org.apache.asterix.common.storage.IIndexCheckpointManager;
 import org.apache.asterix.common.storage.IIndexCheckpointManagerProvider;
@@ -557,6 +558,23 @@

Change in asterixdb[master]: Merge commit '79b243' from stabilization-f69489

[Murtadha Hubail (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change. Please visit

https://asterix-gerrit.ics.uci.edu/3418

to look at the new patch set (#2).

Change subject: Merge commit '79b243' from stabilization-f69489
..

Merge commit '79b243' from stabilization-f69489

Change-Id: Ie2387663ab0cd4544eb9ec6f12ec2ba74136ef83
---
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
2 files changed, 0 insertions(+), 5 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/18/3418/2
--
To view, visit https://asterix-gerrit.ics.uci.edu/3418
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie2387663ab0cd4544eb9ec6f12ec2ba74136ef83
Gerrit-Change-Number: 3418
Gerrit-PatchSet: 2
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Jenkins 

Change in asterixdb[master]: z

[Murtadha Hubail (Code Review)]

Murtadha Hubail has uploaded this change for review. ( 
https://asterix-gerrit.ics.uci.edu/3418


Change subject: z
..

z

Change-Id: Ie2387663ab0cd4544eb9ec6f12ec2ba74136ef83
---
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
2 files changed, 0 insertions(+), 5 deletions(-)



  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/18/3418/1

diff --git 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
index bb4c546..1e92f16 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
@@ -218,13 +218,8 @@
 IIPCI ciIPCI = new ClientInterfaceIPCI(this, jobIdFactory);
 clientIPC =
 new IPCSystem(new 
InetSocketAddress(ccConfig.getClientListenAddress(), 
ccConfig.getClientListenPort()),
-<<< HEAD   (84f734 [ASTERIXDB-2579][DOC] Document uuid/upsert/insert 
behavior)
 networkSecurityManager.getSocketChannelFactory(), 
ciIPCI,
 new 
JavaSerializationBasedPayloadSerializerDeserializer());
-webServer = new WebServer(this, ccConfig.getConsoleListenPort());
-===
-ciIPCI, new 
JavaSerializationBasedPayloadSerializerDeserializer());
->>> BRANCH (79b243 [NO ISSUE][OTH] Make ClusterControllerService 
Extensible)
 clusterIPC.start();
 clientIPC.start();
 startWebServers();

--
To view, visit https://asterix-gerrit.ics.uci.edu/3418
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ie2387663ab0cd4544eb9ec6f12ec2ba74136ef83
Gerrit-Change-Number: 3418
Gerrit-PatchSet: 1
Gerrit-Owner: Murtadha Hubail 

Change in asterixdb[master]: [NO ISSUE][OTH] Make ClusterControllerService Extensible

[Murtadha Hubail (Code Review)]

Murtadha Hubail has abandoned this change. ( 
https://asterix-gerrit.ics.uci.edu/3417 )

Change subject: [NO ISSUE][OTH] Make ClusterControllerService Extensible
..


Abandoned

merged in stabilization-f69489
--
To view, visit https://asterix-gerrit.ics.uci.edu/3417
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: abandon
Gerrit-Change-Id: I87ec46a29b2421d88df15494fe6331a0ae5d37f4
Gerrit-Change-Number: 3417
Gerrit-PatchSet: 1
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 

Change in asterixdb[stabilization-f69489]: [NO ISSUE][OTH] Make ClusterControllerService Extensible

[Murtadha Hubail (Code Review)]

Murtadha Hubail has submitted this change and it was merged. ( 
https://asterix-gerrit.ics.uci.edu/3416 )

Change subject: [NO ISSUE][OTH] Make ClusterControllerService Extensible
..

[NO ISSUE][OTH] Make ClusterControllerService Extensible

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Allow configuring the web servers started on hyracks
  ClusterControllerService.

Change-Id: Icd2a048a6152105c831183db2440e9dc5b3a1eb5
Reviewed-on: https://asterix-gerrit.ics.uci.edu/3416
Tested-by: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Michael Blow 
Contrib: Michael Blow 
---
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
1 file changed, 11 insertions(+), 4 deletions(-)

Approvals:
  Jenkins: Verified; Verified
  Michael Blow: Looks good to me, approved;

Objections:
  Jenkins: Violations found



diff --git 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
index b5dacfb..91397a8 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
@@ -211,10 +211,9 @@
 clientIPC =
 new IPCSystem(new 
InetSocketAddress(ccConfig.getClientListenAddress(), 
ccConfig.getClientListenPort()),
 ciIPCI, new 
JavaSerializationBasedPayloadSerializerDeserializer());
-webServer = new WebServer(this, ccConfig.getConsoleListenPort());
 clusterIPC.start();
 clientIPC.start();
-webServer.start();
+startWebServers();
 info = new ClusterControllerInfo(ccId, 
ccConfig.getClientPublicAddress(), ccConfig.getClientPublicPort(),
 ccConfig.getConsolePublicPort());
 timer.schedule(sweeper, 0, ccConfig.getDeadNodeSweepThreshold());
@@ -226,6 +225,11 @@
 connectNCs();
 LOGGER.log(Level.INFO, "Started ClusterControllerService");
 notifyApplication();
+}
+
+protected void startWebServers() throws Exception {
+webServer = new WebServer(this, ccConfig.getConsoleListenPort());
+webServer.start();
 }

 private void startApplication() throws Exception {
@@ -317,7 +321,7 @@
 public void stop() throws Exception {
 LOGGER.log(Level.INFO, "Stopping ClusterControllerService");
 stopApplication();
-webServer.stop();
+stopWebServers();
 sweeper.cancel();
 workQueue.stop();
 executor.shutdownNow();
@@ -327,8 +331,11 @@
 LOGGER.log(Level.INFO, "Stopped ClusterControllerService");
 }

-private void stopApplication() throws Exception {
+protected void stopWebServers() throws Exception {
+webServer.stop();
+}

+private void stopApplication() throws Exception {
 application.stop();
 }


--
To view, visit https://asterix-gerrit.ics.uci.edu/3416
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: stabilization-f69489
Gerrit-MessageType: merged
Gerrit-Change-Id: Icd2a048a6152105c831183db2440e9dc5b3a1eb5
Gerrit-Change-Number: 3416
Gerrit-PatchSet: 2
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 

Change in asterixdb[stabilization-f69489]: [NO ISSUE][OTH] Make ClusterControllerService Extensible

[Murtadha Hubail (Code Review)]

Murtadha Hubail has restored this change. ( 
https://asterix-gerrit.ics.uci.edu/3416 )

Change subject: [NO ISSUE][OTH] Make ClusterControllerService Extensible
..


Restored
--
To view, visit https://asterix-gerrit.ics.uci.edu/3416
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: stabilization-f69489
Gerrit-MessageType: restore
Gerrit-Change-Id: Icd2a048a6152105c831183db2440e9dc5b3a1eb5
Gerrit-Change-Number: 3416
Gerrit-PatchSet: 1
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 

Change in asterixdb[stabilization-f69489]: [NO ISSUE][OTH] Make ClusterControllerService Extensible

[Murtadha Hubail (Code Review)]

Murtadha Hubail has abandoned this change. ( 
https://asterix-gerrit.ics.uci.edu/3416 )

Change subject: [NO ISSUE][OTH] Make ClusterControllerService Extensible
..


Abandoned

proposed on master
--
To view, visit https://asterix-gerrit.ics.uci.edu/3416
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: stabilization-f69489
Gerrit-MessageType: abandon
Gerrit-Change-Id: Icd2a048a6152105c831183db2440e9dc5b3a1eb5
Gerrit-Change-Number: 3416
Gerrit-PatchSet: 1
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 

Change in asterixdb[stabilization-f69489]: [NO ISSUE][CONF] Make Starting Hyracks Console Configurable

[Murtadha Hubail (Code Review)]

Murtadha Hubail has abandoned this change. ( 
https://asterix-gerrit.ics.uci.edu/3415 )

Change subject: [NO ISSUE][CONF] Make Starting Hyracks Console Configurable
..


Abandoned
--
To view, visit https://asterix-gerrit.ics.uci.edu/3415
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: stabilization-f69489
Gerrit-MessageType: abandon
Gerrit-Change-Id: If0050b298431b150690959b6d35db69182777ff6
Gerrit-Change-Number: 3415
Gerrit-PatchSet: 1
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 

Change in asterixdb[master]: [NO ISSUE][OTH] Make ClusterControllerService Extensible

[Murtadha Hubail (Code Review)]

Murtadha Hubail has uploaded this change for review. ( 
https://asterix-gerrit.ics.uci.edu/3417


Change subject: [NO ISSUE][OTH] Make ClusterControllerService Extensible
..

[NO ISSUE][OTH] Make ClusterControllerService Extensible

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Allow configuring the web servers started on hyracks
  ClusterControllerService.

Change-Id: I87ec46a29b2421d88df15494fe6331a0ae5d37f4
---
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
1 file changed, 11 insertions(+), 3 deletions(-)



  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/17/3417/1

diff --git 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
index de1b18d..3f6cb32 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
@@ -220,10 +220,9 @@
 new IPCSystem(new 
InetSocketAddress(ccConfig.getClientListenAddress(), 
ccConfig.getClientListenPort()),
 networkSecurityManager.getSocketChannelFactory(), 
ciIPCI,
 new 
JavaSerializationBasedPayloadSerializerDeserializer());
-webServer = new WebServer(this, ccConfig.getConsoleListenPort());
 clusterIPC.start();
 clientIPC.start();
-webServer.start();
+startWebServers();
 info = new ClusterControllerInfo(ccId, 
ccConfig.getClientPublicAddress(), ccConfig.getClientPublicPort(),
 ccConfig.getConsolePublicPort());
 timer.schedule(sweeper, 0, ccConfig.getDeadNodeSweepThreshold());
@@ -235,6 +234,11 @@
 connectNCs();
 LOGGER.log(Level.INFO, "Started ClusterControllerService");
 notifyApplication();
+}
+
+protected void startWebServers() throws Exception {
+webServer = new WebServer(this, ccConfig.getConsoleListenPort());
+webServer.start();
 }

 private void startApplication() throws Exception {
@@ -326,7 +330,7 @@
 public void stop() throws Exception {
 LOGGER.log(Level.INFO, "Stopping ClusterControllerService");
 stopApplication();
-webServer.stop();
+stopWebServers();
 sweeper.cancel();
 workQueue.stop();
 executor.shutdownNow();
@@ -336,6 +340,10 @@
 LOGGER.log(Level.INFO, "Stopped ClusterControllerService");
 }

+protected void stopWebServers() throws Exception {
+webServer.stop();
+}
+
 private void stopApplication() throws Exception {

 application.stop();

--
To view, visit https://asterix-gerrit.ics.uci.edu/3417
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I87ec46a29b2421d88df15494fe6331a0ae5d37f4
Gerrit-Change-Number: 3417
Gerrit-PatchSet: 1
Gerrit-Owner: Murtadha Hubail 

Change in asterixdb[stabilization-f69489]: [NO ISSUE][OTH] Make ClusterControllerService Extensible

[Murtadha Hubail (Code Review)]

Murtadha Hubail has uploaded this change for review. ( 
https://asterix-gerrit.ics.uci.edu/3416


Change subject: [NO ISSUE][OTH] Make ClusterControllerService Extensible
..

[NO ISSUE][OTH] Make ClusterControllerService Extensible

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Allow configuring the web servers started on hyracks
  ClusterControllerService.

Change-Id: Icd2a048a6152105c831183db2440e9dc5b3a1eb5
---
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
1 file changed, 11 insertions(+), 4 deletions(-)



  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/16/3416/1

diff --git 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
index b5dacfb..91397a8 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
@@ -211,10 +211,9 @@
 clientIPC =
 new IPCSystem(new 
InetSocketAddress(ccConfig.getClientListenAddress(), 
ccConfig.getClientListenPort()),
 ciIPCI, new 
JavaSerializationBasedPayloadSerializerDeserializer());
-webServer = new WebServer(this, ccConfig.getConsoleListenPort());
 clusterIPC.start();
 clientIPC.start();
-webServer.start();
+startWebServers();
 info = new ClusterControllerInfo(ccId, 
ccConfig.getClientPublicAddress(), ccConfig.getClientPublicPort(),
 ccConfig.getConsolePublicPort());
 timer.schedule(sweeper, 0, ccConfig.getDeadNodeSweepThreshold());
@@ -226,6 +225,11 @@
 connectNCs();
 LOGGER.log(Level.INFO, "Started ClusterControllerService");
 notifyApplication();
+}
+
+protected void startWebServers() throws Exception {
+webServer = new WebServer(this, ccConfig.getConsoleListenPort());
+webServer.start();
 }

 private void startApplication() throws Exception {
@@ -317,7 +321,7 @@
 public void stop() throws Exception {
 LOGGER.log(Level.INFO, "Stopping ClusterControllerService");
 stopApplication();
-webServer.stop();
+stopWebServers();
 sweeper.cancel();
 workQueue.stop();
 executor.shutdownNow();
@@ -327,8 +331,11 @@
 LOGGER.log(Level.INFO, "Stopped ClusterControllerService");
 }

-private void stopApplication() throws Exception {
+protected void stopWebServers() throws Exception {
+webServer.stop();
+}

+private void stopApplication() throws Exception {
 application.stop();
 }


--
To view, visit https://asterix-gerrit.ics.uci.edu/3416
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: stabilization-f69489
Gerrit-MessageType: newchange
Gerrit-Change-Id: Icd2a048a6152105c831183db2440e9dc5b3a1eb5
Gerrit-Change-Number: 3416
Gerrit-PatchSet: 1
Gerrit-Owner: Murtadha Hubail 

Change in asterixdb[stabilization-f69489]: [NO ISSUE][CONF] Make Starting Hyracks Console Configurable

[Murtadha Hubail (Code Review)]

Murtadha Hubail has uploaded this change for review. ( 
https://asterix-gerrit.ics.uci.edu/3415


Change subject: [NO ISSUE][CONF] Make Starting Hyracks Console Configurable
..

[NO ISSUE][CONF] Make Starting Hyracks Console Configurable

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Add a config option to allow starting or disabling the
  hyracks web console.

Change-Id: If0050b298431b150690959b6d35db69182777ff6
---
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/CCConfig.java
2 files changed, 15 insertions(+), 4 deletions(-)



  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/15/3415/1

diff --git 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
index b5dacfb..5e99901 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerService.java
@@ -211,10 +211,12 @@
 clientIPC =
 new IPCSystem(new 
InetSocketAddress(ccConfig.getClientListenAddress(), 
ccConfig.getClientListenPort()),
 ciIPCI, new 
JavaSerializationBasedPayloadSerializerDeserializer());
-webServer = new WebServer(this, ccConfig.getConsoleListenPort());
+if (ccConfig.isStartWebConsole()) {
+webServer = new WebServer(this, ccConfig.getConsoleListenPort());
+webServer.start();
+}
 clusterIPC.start();
 clientIPC.start();
-webServer.start();
 info = new ClusterControllerInfo(ccId, 
ccConfig.getClientPublicAddress(), ccConfig.getClientPublicPort(),
 ccConfig.getConsolePublicPort());
 timer.schedule(sweeper, 0, ccConfig.getDeadNodeSweepThreshold());
@@ -317,7 +319,9 @@
 public void stop() throws Exception {
 LOGGER.log(Level.INFO, "Stopping ClusterControllerService");
 stopApplication();
-webServer.stop();
+if (webServer != null) {
+webServer.stop();
+}
 sweeper.cancel();
 workQueue.stop();
 executor.shutdownNow();
diff --git 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/CCConfig.java
 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/CCConfig.java
index 57cf339..c2c967b 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/CCConfig.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/CCConfig.java
@@ -74,7 +74,8 @@
 JOB_MANAGER_CLASS(STRING, 
"org.apache.hyracks.control.cc.job.JobManager"),
 ENFORCE_FRAME_WRITER_PROTOCOL(BOOLEAN, false),
 CORES_MULTIPLIER(INTEGER, 3),
-CONTROLLER_ID(SHORT, (short) 0x);
+CONTROLLER_ID(SHORT, (short) 0x),
+START_WEB_CONSOLE(BOOLEAN, true);

 private final IOptionType parser;
 private Object defaultValue;
@@ -188,6 +189,8 @@
 + "execution level";
 case CONTROLLER_ID:
 return "The 16-bit (0-65535) id of this Cluster 
Controller";
+case START_WEB_CONSOLE:
+return "a flag indication if hyracks web console should be 
started";
 default:
 throw new IllegalStateException("NYI: " + this);
 }
@@ -434,4 +437,8 @@
 public CcId getCcId() {
 return CcId.valueOf(getAppConfig().getShort(Option.CONTROLLER_ID));
 }
+
+public boolean isStartWebConsole() {
+return getAppConfig().getBoolean(Option.START_WEB_CONSOLE);
+}
 }

--
To view, visit https://asterix-gerrit.ics.uci.edu/3415
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: stabilization-f69489
Gerrit-MessageType: newchange
Gerrit-Change-Id: If0050b298431b150690959b6d35db69182777ff6
Gerrit-Change-Number: 3415
Gerrit-PatchSet: 1
Gerrit-Owner: Murtadha Hubail 

Change in asterixdb[master]: [NO ISSUE][CONF] Mark KeyStore Pass As Hidden Config

[Murtadha Hubail (Code Review)]

Murtadha Hubail has submitted this change and it was merged. ( 
https://asterix-gerrit.ics.uci.edu/3413 )

Change subject: [NO ISSUE][CONF] Mark KeyStore Pass As Hidden Config
..

[NO ISSUE][CONF] Mark KeyStore Pass As Hidden Config

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Mark KeyStore pass as a hidden config to avoid
  displaying the password on APIs that use the hidden
  flag to certain config.

Change-Id: I1c27c3b229ca7d8dd44717c21cd096f0090a7278
Reviewed-on: https://asterix-gerrit.ics.uci.edu/3413
Contrib: Jenkins 
Integration-Tests: Jenkins 
Tested-by: Jenkins 
Reviewed-by: Ali Alsuliman 
---
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/CCConfig.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/NCConfig.java
2 files changed, 9 insertions(+), 0 deletions(-)

Approvals:
  Jenkins: Verified; ; Verified
  Anon. E. Moose (1000171):
  Ali Alsuliman: Looks good to me, approved

Objections:
  Jenkins: Violations found



diff --git 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/CCConfig.java
 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/CCConfig.java
index a38ee65..4f35aca 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/CCConfig.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/CCConfig.java
@@ -211,6 +211,11 @@
 public String usageDefaultOverride(IApplicationConfig accessor, 
Function optionPrinter) {
 return defaultValueDescription;
 }
+
+@Override
+public boolean hidden() {
+return this == KEY_STORE_PASSWORD;
+}
 }

 private List appArgs = new ArrayList<>();
diff --git 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/NCConfig.java
 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/NCConfig.java
index dd92798..e947d7a 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/NCConfig.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/NCConfig.java
@@ -247,6 +247,10 @@
 return defaultValueDescription;
 }

+@Override
+public boolean hidden() {
+return this == KEY_STORE_PASSWORD;
+}
 }

 public String getReplicationPublicAddress() {

--
To view, visit https://asterix-gerrit.ics.uci.edu/3413
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I1c27c3b229ca7d8dd44717c21cd096f0090a7278
Gerrit-Change-Number: 3413
Gerrit-PatchSet: 2
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Reviewer: Till Westmann 

Change in asterixdb-bad[master]: [NO ISSUE][BAD] Coordinated Change for ResponsePrinter

[Murtadha Hubail (Code Review)]

Murtadha Hubail has submitted this change and it was merged. ( 
https://asterix-gerrit.ics.uci.edu/3411 )

Change subject: [NO ISSUE][BAD] Coordinated Change for ResponsePrinter
..

[NO ISSUE][BAD] Coordinated Change for ResponsePrinter

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Coordinated change for ResponsePrinter.

Change-Id: I9255ef88ea03df629ffe3ca2f1d9b08504459f06
---
M asterix-bad/src/main/java/org/apache/asterix/bad/BADJobService.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADQueryTranslatorFactory.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADStatementExecutor.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADGlobalRecoveryManager.java
4 files changed, 20 insertions(+), 12 deletions(-)

Approvals:
  Jenkins: Verified
  Till Westmann: Looks good to me, approved



diff --git 
a/asterix-bad/src/main/java/org/apache/asterix/bad/BADJobService.java 
b/asterix-bad/src/main/java/org/apache/asterix/bad/BADJobService.java
index a711a9e..cd3883a 100644
--- a/asterix-bad/src/main/java/org/apache/asterix/bad/BADJobService.java
+++ b/asterix-bad/src/main/java/org/apache/asterix/bad/BADJobService.java
@@ -32,13 +32,14 @@

 import org.apache.asterix.active.ActivityState;
 import org.apache.asterix.active.EntityId;
-import org.apache.asterix.api.http.server.ResultUtil;
 import org.apache.asterix.app.active.ActiveNotificationHandler;
 import org.apache.asterix.app.result.ResultReader;
+import org.apache.asterix.app.result.fields.ResultsPrinter;
 import org.apache.asterix.app.translator.QueryTranslator;
 import org.apache.asterix.bad.lang.BADParserFactory;
 import org.apache.asterix.bad.lang.BADStatementExecutor;
 import org.apache.asterix.bad.metadata.DeployedJobSpecEventListener;
+import org.apache.asterix.common.api.IResponsePrinter;
 import org.apache.asterix.common.dataflow.ICcApplicationContext;
 import org.apache.asterix.common.transactions.ITxnIdFactory;
 import org.apache.asterix.lang.common.base.Statement;
@@ -139,9 +140,10 @@

 if (listener.getType() == 
DeployedJobSpecEventListener.PrecompiledType.QUERY) {
 ResultReader resultReader = new ResultReader(resultSet, jobId, new 
ResultSetId(0));
-
-ResultUtil.printResults(appCtx, resultReader, 
statementExecutor.getSessionOutput(),
-new IStatementExecutor.Stats(), null);
+IResponsePrinter printer = statementExecutor.getResponsePrinter();
+printer.addResultPrinter(new ResultsPrinter(appCtx, resultReader, 
null, new IStatementExecutor.Stats(),
+statementExecutor.getSessionOutput()));
+printer.printResults();
 }

 LOGGER.log(Level.SEVERE,
diff --git 
a/asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADQueryTranslatorFactory.java
 
b/asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADQueryTranslatorFactory.java
index 8f5d520..ab5a96e 100644
--- 
a/asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADQueryTranslatorFactory.java
+++ 
b/asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADQueryTranslatorFactory.java
@@ -22,6 +22,7 @@

 import org.apache.asterix.app.translator.DefaultStatementExecutorFactory;
 import org.apache.asterix.app.translator.QueryTranslator;
+import org.apache.asterix.common.api.IResponsePrinter;
 import org.apache.asterix.common.context.IStorageComponentProvider;
 import org.apache.asterix.common.dataflow.ICcApplicationContext;
 import org.apache.asterix.compiler.provider.ILangCompilationProvider;
@@ -32,7 +33,8 @@

 @Override
 public QueryTranslator create(ICcApplicationContext appCtx, 
List statements, SessionOutput output,
-ILangCompilationProvider compilationProvider, 
IStorageComponentProvider storageComponentProvider) {
-return new BADStatementExecutor(appCtx, statements, output, 
compilationProvider, executorService);
+ILangCompilationProvider compilationProvider, 
IStorageComponentProvider storageComponentProvider,
+IResponsePrinter printer) {
+return new BADStatementExecutor(appCtx, statements, output, 
compilationProvider, executorService, printer);
 }
 }
diff --git 
a/asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADStatementExecutor.java
 
b/asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADStatementExecutor.java
index 9cb57e1..e57a2e5 100644
--- 
a/asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADStatementExecutor.java
+++ 
b/asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADStatementExecutor.java
@@ -32,6 +32,7 @@
 import org.apache.asterix.bad.metadata.Channel;
 import org.apache.asterix.bad.metadata.DeployedJobSpecEventListener;
 import org.apache.asterix.bad.metadata.Procedure;
+import org.apache.asterix.common.api.IResponsePrinter;
 import org.apache.asterix.common.dataflow.ICcApplicationContext;
 import 

Change in asterixdb[master]: [NO ISSUE][CONF] Mark KeyStore Pass As Hidden Config

[Murtadha Hubail (Code Review)]

Murtadha Hubail has uploaded this change for review. ( 
https://asterix-gerrit.ics.uci.edu/3413


Change subject: [NO ISSUE][CONF] Mark KeyStore Pass As Hidden Config
..

[NO ISSUE][CONF] Mark KeyStore Pass As Hidden Config

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Mark KeyStore pass as a hidden config to avoid
  displaying the password on APIs that use the hidden
  flag to certain config.

Change-Id: I1c27c3b229ca7d8dd44717c21cd096f0090a7278
---
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/CCConfig.java
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/NCConfig.java
2 files changed, 9 insertions(+), 0 deletions(-)



  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/13/3413/1

diff --git 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/CCConfig.java
 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/CCConfig.java
index a38ee65..4f35aca 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/CCConfig.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/CCConfig.java
@@ -211,6 +211,11 @@
 public String usageDefaultOverride(IApplicationConfig accessor, 
Function optionPrinter) {
 return defaultValueDescription;
 }
+
+@Override
+public boolean hidden() {
+return this == KEY_STORE_PASSWORD;
+}
 }

 private List appArgs = new ArrayList<>();
diff --git 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/NCConfig.java
 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/NCConfig.java
index dd92798..e947d7a 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/NCConfig.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/controllers/NCConfig.java
@@ -247,6 +247,10 @@
 return defaultValueDescription;
 }

+@Override
+public boolean hidden() {
+return this == KEY_STORE_PASSWORD;
+}
 }

 public String getReplicationPublicAddress() {

--
To view, visit https://asterix-gerrit.ics.uci.edu/3413
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I1c27c3b229ca7d8dd44717c21cd096f0090a7278
Gerrit-Change-Number: 3413
Gerrit-PatchSet: 1
Gerrit-Owner: Murtadha Hubail 

Change in asterixdb[master]: [NO ISSUE][OTH] Introduce ResponsePrinter

[Murtadha Hubail (Code Review)]

Hello Anon. E. Moose (1000171), Jenkins,

I'd like you to reexamine a change. Please visit

https://asterix-gerrit.ics.uci.edu/3410

to look at the new patch set (#3).

Change subject: [NO ISSUE][OTH] Introduce ResponsePrinter
..

[NO ISSUE][OTH] Introduce ResponsePrinter

- user model changes: no
- storage format changes: no
- interface changes: yes

Details:
- Introduce ResponsePrinter that is responsible for printing
  the returned response from query service.
- Make ResponsePrinter responsible for adding separators between
  fields in the returned response.
- Introduce IResponseFieldPrinter and encapsulate each response
  field in its own printer.
- Introduce ICodedMessage and use it for any object that
  has code<->message (e.g errors and warnings).

Change-Id: I797e6615a72c886391fed26281fc648b38fa748f
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/IStatementExecutor.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/IStatementExecutorFactory.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/SessionOutput.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/common/APIFramework.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/AbstractQueryApiServlet.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/ApiServlet.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/NCQueryServiceServlet.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryResultApiServlet.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryServiceServlet.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryStatusApiServlet.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/ResultUtil.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/java/AsterixJavaClient.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/message/ExecuteStatementRequestMessage.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/ExecutionError.java
R 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/ExecutionWarning.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/ResponseMertics.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/ResponsePrinter.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/ResultHandle.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/AbstractCodedMessagePrinter.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/ClientContextIdPrinter.java
C 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/ErrorsPrinter.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/ExplainOnlyResultsPrinter.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/MetricsPrinter.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/NcResultPrinter.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/ParseOnlyResultPrinter.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/PlansPrinter.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/RequestIdPrinter.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/ResultHandlePrinter.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/ResultsPrinter.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/SignaturePrinter.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/StatusPrinter.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/TypePrinter.java
C 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/WarningsPrinter.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/DefaultStatementExecutorFactory.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/app/result/ResultPrinterTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/active/ActiveStatsTest.java
C 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/api/ICodedMessage.java
C 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/api/IResponseFieldPrinter.java
A 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/api/IResponsePrinter.java
41 files changed, 1,356 insertions(+), 420 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/10/3410/3
--
To view, 

Change in asterixdb-bad[master]: [NO ISSUE][BAD] Coordinated Change for ResponsePrinter

[Murtadha Hubail (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change. Please visit

https://asterix-gerrit.ics.uci.edu/3411

to look at the new patch set (#4).

Change subject: [NO ISSUE][BAD] Coordinated Change for ResponsePrinter
..

[NO ISSUE][BAD] Coordinated Change for ResponsePrinter

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Coordinated change for ResponsePrinter.

Change-Id: I9255ef88ea03df629ffe3ca2f1d9b08504459f06
---
M asterix-bad/src/main/java/org/apache/asterix/bad/BADJobService.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADQueryTranslatorFactory.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADStatementExecutor.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADGlobalRecoveryManager.java
4 files changed, 20 insertions(+), 12 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb-bad 
refs/changes/11/3411/4
--
To view, visit https://asterix-gerrit.ics.uci.edu/3411
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb-bad
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I9255ef88ea03df629ffe3ca2f1d9b08504459f06
Gerrit-Change-Number: 3411
Gerrit-PatchSet: 4
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Jenkins 

Change in asterixdb[master]: [NO ISSUE][OTH] Introduce ResponsePrinter

[Murtadha Hubail (Code Review)]

Hello Anon. E. Moose (1000171), Jenkins,

I'd like you to reexamine a change. Please visit

https://asterix-gerrit.ics.uci.edu/3410

to look at the new patch set (#2).

Change subject: [NO ISSUE][OTH] Introduce ResponsePrinter
..

[NO ISSUE][OTH] Introduce ResponsePrinter

- user model changes: no
- storage format changes: no
- interface changes: yes

Details:
- Introduce ResponsePrinter that is responsible for printing
  the returned response from query service.
- Make ResponsePrinter responsible for adding separators between
  fields in the returned response.
- Introduce IResponseFieldPrinter and encapsulate each response
  field in its own printer.
- Introduce ICodedMessage and use it for any object that
  has code<->message (e.g errors and warnings).

Change-Id: I797e6615a72c886391fed26281fc648b38fa748f
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/IStatementExecutor.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/IStatementExecutorFactory.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/SessionOutput.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/common/APIFramework.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/AbstractQueryApiServlet.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/ApiServlet.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/NCQueryServiceServlet.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryResultApiServlet.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryServiceServlet.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryStatusApiServlet.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/ResultUtil.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/java/AsterixJavaClient.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/message/ExecuteStatementRequestMessage.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/ExecutionError.java
R 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/ExecutionWarning.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/ResponseMertics.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/ResponsePrinter.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/ResultHandle.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/AbstractCodedMessagePrinter.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/ClientContextIdPrinter.java
C 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/ErrorsPrinter.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/ExplainOnlyResultsPrinter.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/MetricsPrinter.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/NcResultPrinter.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/ParseOnlyResultPrinter.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/PlansPrinter.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/RequestIdPrinter.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/ResultHandlePrinter.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/ResultsPrinter.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/SignaturePrinter.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/StatusPrinter.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/TypePrinter.java
C 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/fields/WarningsPrinter.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/DefaultStatementExecutorFactory.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/FeedOperations.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/app/result/ResultPrinterTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/active/ActiveStatsTest.java
C 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/api/ICodedMessage.java
C 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/api/IResponseFieldPrinter.java
A 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/api/IResponsePrinter.java
41 files changed, 1,354 insertions(+), 421 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/10/3410/2
--
To view, 

Change in asterixdb-bad[master]: [NO ISSUE][BAD] Coordinated Change for ResponsePrinter

[Murtadha Hubail (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change. Please visit

https://asterix-gerrit.ics.uci.edu/3411

to look at the new patch set (#3).

Change subject: [NO ISSUE][BAD] Coordinated Change for ResponsePrinter
..

[NO ISSUE][BAD] Coordinated Change for ResponsePrinter

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Coordinated change for ResponsePrinter.

Change-Id: I9255ef88ea03df629ffe3ca2f1d9b08504459f06
---
M asterix-bad/src/main/java/org/apache/asterix/bad/BADJobService.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADQueryTranslatorFactory.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADStatementExecutor.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADGlobalRecoveryManager.java
4 files changed, 20 insertions(+), 12 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb-bad 
refs/changes/11/3411/3
--
To view, visit https://asterix-gerrit.ics.uci.edu/3411
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb-bad
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I9255ef88ea03df629ffe3ca2f1d9b08504459f06
Gerrit-Change-Number: 3411
Gerrit-PatchSet: 3
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Jenkins 

Change in asterixdb-bad[master]: [NO ISSUE][BAD] Coordinated Change for ResponsePrinter

[Murtadha Hubail (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change. Please visit

https://asterix-gerrit.ics.uci.edu/3411

to look at the new patch set (#2).

Change subject: [NO ISSUE][BAD] Coordinated Change for ResponsePrinter
..

[NO ISSUE][BAD] Coordinated Change for ResponsePrinter

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Coordinated change for ResponsePrinter.

Change-Id: I9255ef88ea03df629ffe3ca2f1d9b08504459f06
---
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADQueryTranslatorFactory.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADStatementExecutor.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADGlobalRecoveryManager.java
3 files changed, 14 insertions(+), 8 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb-bad 
refs/changes/11/3411/2
--
To view, visit https://asterix-gerrit.ics.uci.edu/3411
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb-bad
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I9255ef88ea03df629ffe3ca2f1d9b08504459f06
Gerrit-Change-Number: 3411
Gerrit-PatchSet: 2
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Jenkins 

Change in asterixdb-bad[master]: [NO ISSUE][BAD] Coordinated Change for ResponsePrinter

[Murtadha Hubail (Code Review)]

Murtadha Hubail has uploaded this change for review. ( 
https://asterix-gerrit.ics.uci.edu/3411


Change subject: [NO ISSUE][BAD] Coordinated Change for ResponsePrinter
..

[NO ISSUE][BAD] Coordinated Change for ResponsePrinter

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Coordinated change for ResponsePrinter.

Change-Id: I9255ef88ea03df629ffe3ca2f1d9b08504459f06
---
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADQueryTranslatorFactory.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADStatementExecutor.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADGlobalRecoveryManager.java
3 files changed, 13 insertions(+), 7 deletions(-)



  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb-bad 
refs/changes/11/3411/1

diff --git 
a/asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADQueryTranslatorFactory.java
 
b/asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADQueryTranslatorFactory.java
index 8f5d520..ab5a96e 100644
--- 
a/asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADQueryTranslatorFactory.java
+++ 
b/asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADQueryTranslatorFactory.java
@@ -22,6 +22,7 @@

 import org.apache.asterix.app.translator.DefaultStatementExecutorFactory;
 import org.apache.asterix.app.translator.QueryTranslator;
+import org.apache.asterix.common.api.IResponsePrinter;
 import org.apache.asterix.common.context.IStorageComponentProvider;
 import org.apache.asterix.common.dataflow.ICcApplicationContext;
 import org.apache.asterix.compiler.provider.ILangCompilationProvider;
@@ -32,7 +33,8 @@

 @Override
 public QueryTranslator create(ICcApplicationContext appCtx, 
List statements, SessionOutput output,
-ILangCompilationProvider compilationProvider, 
IStorageComponentProvider storageComponentProvider) {
-return new BADStatementExecutor(appCtx, statements, output, 
compilationProvider, executorService);
+ILangCompilationProvider compilationProvider, 
IStorageComponentProvider storageComponentProvider,
+IResponsePrinter printer) {
+return new BADStatementExecutor(appCtx, statements, output, 
compilationProvider, executorService, printer);
 }
 }
diff --git 
a/asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADStatementExecutor.java
 
b/asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADStatementExecutor.java
index 9cb57e1..e57a2e5 100644
--- 
a/asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADStatementExecutor.java
+++ 
b/asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADStatementExecutor.java
@@ -32,6 +32,7 @@
 import org.apache.asterix.bad.metadata.Channel;
 import org.apache.asterix.bad.metadata.DeployedJobSpecEventListener;
 import org.apache.asterix.bad.metadata.Procedure;
+import org.apache.asterix.common.api.IResponsePrinter;
 import org.apache.asterix.common.dataflow.ICcApplicationContext;
 import org.apache.asterix.common.exceptions.CompilationException;
 import org.apache.asterix.common.functions.FunctionSignature;
@@ -57,8 +58,8 @@
 public class BADStatementExecutor extends QueryTranslator {

 public BADStatementExecutor(ICcApplicationContext appCtx, List 
statements, SessionOutput output,
-ILangCompilationProvider compliationProvider, ExecutorService 
executorService) {
-super(appCtx, statements, output, compliationProvider, 
executorService);
+ILangCompilationProvider compliationProvider, ExecutorService 
executorService, IResponsePrinter printer) {
+super(appCtx, statements, output, compliationProvider, 
executorService, printer);
 }

 //TODO: Most of this file could go away if we had metadata dependencies
diff --git 
a/asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADGlobalRecoveryManager.java
 
b/asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADGlobalRecoveryManager.java
index 38dc16d..0cc59e5 100644
--- 
a/asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADGlobalRecoveryManager.java
+++ 
b/asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADGlobalRecoveryManager.java
@@ -30,6 +30,7 @@
 import org.apache.asterix.active.EntityId;
 import org.apache.asterix.active.IActiveEntityEventsListener;
 import org.apache.asterix.app.active.ActiveNotificationHandler;
+import org.apache.asterix.app.result.ResponsePrinter;
 import org.apache.asterix.app.result.ResultReader;
 import org.apache.asterix.app.translator.DefaultStatementExecutorFactory;
 import org.apache.asterix.app.translator.RequestParameters;
@@ -94,10 +95,12 @@
 throws Exception {
 SessionConfig sessionConfig =
 new SessionConfig(SessionConfig.OutputFormat.ADM, true, true, 
true, SessionConfig.PlanFormat.STRING);
-
+final SessionOutput sessionOutput = new SessionOutput(sessionConfig, 
null);
 

Change in asterixdb[master]: [NO ISSUE][OTH] Skip Result Metadata Updates on No Result

[Murtadha Hubail (Code Review)]

Murtadha Hubail has submitted this change and it was merged. ( 
https://asterix-gerrit.ics.uci.edu/3408 )

Change subject: [NO ISSUE][OTH] Skip Result Metadata Updates on No Result
..

[NO ISSUE][OTH] Skip Result Metadata Updates on No Result

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Do not attempt to update the result metadata when a job
  has no result to avoid NPE.

Change-Id: I1ccae6003048f45344a5b7843a25c61d6a7bd366
Reviewed-on: https://asterix-gerrit.ics.uci.edu/3408
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Hussain Towaileb 
---
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/JobResultCallback.java
1 file changed, 6 insertions(+), 1 deletion(-)

Approvals:
  Jenkins: Verified; No violations found; Verified
  Hussain Towaileb: Looks good to me, approved

Objections:
  Anon. E. Moose (1000171): Violations found



diff --git 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/JobResultCallback.java
 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/JobResultCallback.java
index 30814c6..ecf92a0 100644
--- 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/JobResultCallback.java
+++ 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/JobResultCallback.java
@@ -25,6 +25,7 @@
 import org.apache.hyracks.api.job.JobId;
 import org.apache.hyracks.api.result.IJobResultCallback;
 import org.apache.hyracks.api.result.ResultJobRecord;
+import org.apache.hyracks.api.result.ResultSetMetaData;
 import org.apache.hyracks.control.cc.ClusterControllerService;
 import org.apache.hyracks.control.cc.job.IJobManager;
 import org.apache.hyracks.control.cc.job.JobRun;
@@ -53,7 +54,11 @@
 }

 private void updateResultMetadata(JobId jobId, ResultJobRecord 
resultJobRecord) {
-final ResultMetadata metadata = (ResultMetadata) 
resultJobRecord.getResultSetMetaData().getMetadata();
+final ResultSetMetaData resultSetMetaData = 
resultJobRecord.getResultSetMetaData();
+if (resultSetMetaData == null) {
+return;
+}
+final ResultMetadata metadata = (ResultMetadata) 
resultSetMetaData.getMetadata();
 metadata.setJobDuration(resultJobRecord.getJobDuration());
 metadata.setProcessedObjects(getJobProccssedObjects(jobId));
 }

--
To view, visit https://asterix-gerrit.ics.uci.edu/3408
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I1ccae6003048f45344a5b7843a25c61d6a7bd366
Gerrit-Change-Number: 3408
Gerrit-PatchSet: 3
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Hussain Towaileb 
Gerrit-Reviewer: Hussain Towaileb 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Reviewer: Till Westmann 

Change in asterixdb[master]: [NO ISSUE][OTH] Skip Result Metadata Updates on No Result

[Murtadha Hubail (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change. Please visit

https://asterix-gerrit.ics.uci.edu/3408

to look at the new patch set (#2).

Change subject: [NO ISSUE][OTH] Skip Result Metadata Updates on No Result
..

[NO ISSUE][OTH] Skip Result Metadata Updates on No Result

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Do not attempt to update the result metadata when a job
  has no result to avoid NPE.

Change-Id: I1ccae6003048f45344a5b7843a25c61d6a7bd366
---
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/JobResultCallback.java
1 file changed, 6 insertions(+), 1 deletion(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/08/3408/2
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/3408
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1ccae6003048f45344a5b7843a25c61d6a7bd366
Gerrit-Change-Number: 3408
Gerrit-PatchSet: 2
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Jenkins 

Change in asterixdb[master]: [NO ISSUE][OTH] Skip Result Metadata Updates on No Result

[Murtadha Hubail (Code Review)]

Murtadha Hubail has uploaded this change for review. ( 
https://asterix-gerrit.ics.uci.edu/3408


Change subject: [NO ISSUE][OTH] Skip Result Metadata Updates on No Result
..

[NO ISSUE][OTH] Skip Result Metadata Updates on No Result

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Do not attempt to update the result metadata when a job
  has not result to avoid NPE.

Change-Id: I1ccae6003048f45344a5b7843a25c61d6a7bd366
---
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/JobResultCallback.java
1 file changed, 6 insertions(+), 1 deletion(-)



  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/08/3408/1

diff --git 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/JobResultCallback.java
 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/JobResultCallback.java
index 30814c6..ecf92a0 100644
--- 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/JobResultCallback.java
+++ 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/JobResultCallback.java
@@ -25,6 +25,7 @@
 import org.apache.hyracks.api.job.JobId;
 import org.apache.hyracks.api.result.IJobResultCallback;
 import org.apache.hyracks.api.result.ResultJobRecord;
+import org.apache.hyracks.api.result.ResultSetMetaData;
 import org.apache.hyracks.control.cc.ClusterControllerService;
 import org.apache.hyracks.control.cc.job.IJobManager;
 import org.apache.hyracks.control.cc.job.JobRun;
@@ -53,7 +54,11 @@
 }

 private void updateResultMetadata(JobId jobId, ResultJobRecord 
resultJobRecord) {
-final ResultMetadata metadata = (ResultMetadata) 
resultJobRecord.getResultSetMetaData().getMetadata();
+final ResultSetMetaData resultSetMetaData = 
resultJobRecord.getResultSetMetaData();
+if (resultSetMetaData == null) {
+return;
+}
+final ResultMetadata metadata = (ResultMetadata) 
resultSetMetaData.getMetadata();
 metadata.setJobDuration(resultJobRecord.getJobDuration());
 metadata.setProcessedObjects(getJobProccssedObjects(jobId));
 }

--
To view, visit https://asterix-gerrit.ics.uci.edu/3408
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I1ccae6003048f45344a5b7843a25c61d6a7bd366
Gerrit-Change-Number: 3408
Gerrit-PatchSet: 1
Gerrit-Owner: Murtadha Hubail 

Change in asterixdb[master]: [NO ISSUE][API] Return Async Query Results As JSON

[Murtadha Hubail (Code Review)]

Hello Anon. E. Moose (1000171), Jenkins,

I'd like you to reexamine a change. Please visit

https://asterix-gerrit.ics.uci.edu/3401

to look at the new patch set (#2).

Change subject: [NO ISSUE][API] Return Async Query Results As JSON
..

[NO ISSUE][API] Return Async Query Results As JSON

- user model changes: yes
- storage format changes: no
- interface changes: yes

Details:
- Add ResultMetadata that can hold metadata about a job's
  result.
- Add required CC functions to get ResultMetadata.
- Add IJobResultCallback that is called when a job's result
  is written successfully.
- Calculate a job's processed objects on the job completion.
- Calculate a job's duration on the job completion.
- Use the async query request format when serving the result
  from QueryResultApiServlet.
- Return a proper JSON envelop in QueryResultApiServlet
  which includes the results as well as metrics fields.
- Add test case for returning JSON response for async
  query.

Change-Id: Ic4812a14925099a677f9e77a0040f881d2600724
---
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/algebra/base/ILangExpressionToPlanTranslator.java
M 
asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/LangExpressionToPlanTranslator.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/common/APIFramework.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/common/ResultMetadata.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryResultApiServlet.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryServiceServlet.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/JobResultCallback.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/result/ResultReader.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/CCApplication.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/app/resource/PlanStagesGeneratorTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/common/TestExecutor.java
M 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/async-deferred/AsyncDeferredQueries.xml
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/async-deferred/async-json/async-json.1.async.sqlpp
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/async-deferred/async-json/async-json.2.pollget.uri
A 
asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/async-deferred/async-json/async-json.3.get.uri
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/async-deferred/async-json/async-json.1.ignore
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/async-deferred/async-json/async-json.2.regex
A 
asterixdb/asterix-app/src/test/resources/runtimets/results/async-deferred/async-json/async-json.3.json
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/MetadataProvider.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/metadata/IMetadataProvider.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/DistributeResultOperator.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/logical/visitors/OperatorDeepCopyVisitor.java
M 
hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/operators/physical/DistributeResultPOperator.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/application/ICCApplication.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/client/HyracksClientInterfaceFunctions.java
A 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/result/IJobResultCallback.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/result/IResultDirectory.java
A 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/result/IResultMetadata.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/result/IResultPartitionManager.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/result/IResultSetReader.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/result/ResultJobRecord.java
M 
hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/result/ResultSetMetaData.java
M 
hyracks-fullstack/hyracks/hyracks-client/src/main/java/org/apache/hyracks/client/result/ResultDirectory.java
M 
hyracks-fullstack/hyracks/hyracks-client/src/main/java/org/apache/hyracks/client/result/ResultDirectoryRemoteProxy.java
M 

Change in asterixdb[master]: [NO ISSUE][*DB][CLUS] Add support for a rebalance required cluster state

[Murtadha Hubail (Code Review)]

Murtadha Hubail has posted comments on this change. ( 
https://asterix-gerrit.ics.uci.edu/3394 )

Change subject: [NO ISSUE][*DB][CLUS] Add support for a rebalance required 
cluster state
..


Patch Set 1: Code-Review+2


--
To view, visit https://asterix-gerrit.ics.uci.edu/3394
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I863f8bf3fe1ce8d59522c9a28a1283006ffa414c
Gerrit-Change-Number: 3394
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Blow 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Reviewer: Till Westmann 
Gerrit-Comment-Date: Wed, 15 May 2019 19:23:17 +
Gerrit-HasComments: No

Change in asterixdb[master]: [ASTERIXDB-2567][RT] Fix serializing nullable record fields

[Murtadha Hubail (Code Review)]

Murtadha Hubail has posted comments on this change. ( 
https://asterix-gerrit.ics.uci.edu/3398 )

Change subject: [ASTERIXDB-2567][RT] Fix serializing nullable record fields
..


Patch Set 4: Code-Review+2


--
To view, visit https://asterix-gerrit.ics.uci.edu/3398
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I42105965e69c60bc4b123337296476b4d75f0ad8
Gerrit-Change-Number: 3398
Gerrit-PatchSet: 4
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Reviewer: Till Westmann 
Gerrit-Comment-Date: Wed, 15 May 2019 19:25:59 +
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE][*DB][CLUS] Add support for a rebalance required cluster state

[Murtadha Hubail (Code Review)]

Murtadha Hubail has posted comments on this change. ( 
https://asterix-gerrit.ics.uci.edu/3394 )

Change subject: [NO ISSUE][*DB][CLUS] Add support for a rebalance required 
cluster state
..


Patch Set 1:

(1 comment)

https://asterix-gerrit.ics.uci.edu/#/c/3394/1/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/utils/ClusterStateManager.java
File 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/utils/ClusterStateManager.java:

https://asterix-gerrit.ics.uci.edu/#/c/3394/1/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/utils/ClusterStateManager.java@195
PS1, Line 195: if (state != ClusterState.ACTIVE && state != 
ClusterState.RECOVERING) {
 : setState(ClusterState.PENDING);
 : }
> This ensures that we always transition from UNUSABLE -> PENDING -> RECOVERI
I doubt this is of any use since any subscribers won't be able to get the 
current state until this synchronized block is complete. We can revise this on 
another change though.



--
To view, visit https://asterix-gerrit.ics.uci.edu/3394
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I863f8bf3fe1ce8d59522c9a28a1283006ffa414c
Gerrit-Change-Number: 3394
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Blow 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Reviewer: Till Westmann 
Gerrit-Comment-Date: Wed, 15 May 2019 19:03:03 +
Gerrit-HasComments: Yes

Change in asterixdb[master]: [ASTERIXDB-2555][RT][COMP] Make hash join use logical comparison

[Murtadha Hubail (Code Review)]

Murtadha Hubail has posted comments on this change. ( 
https://asterix-gerrit.ics.uci.edu/3387 )

Change subject: [ASTERIXDB-2555][RT][COMP] Make hash join use logical comparison
..


Patch Set 4: Code-Review+2


--
To view, visit https://asterix-gerrit.ics.uci.edu/3387
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If1834967fdd913fdc76003f09636b2450d07cd5e
Gerrit-Change-Number: 3387
Gerrit-PatchSet: 4
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Reviewer: Till Westmann 
Gerrit-Comment-Date: Wed, 15 May 2019 17:11:54 +
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE][*DB][CLUS] Add support for a rebalance required cluster state

[Murtadha Hubail (Code Review)]

Murtadha Hubail has posted comments on this change. ( 
https://asterix-gerrit.ics.uci.edu/3394 )

Change subject: [NO ISSUE][*DB][CLUS] Add support for a rebalance required 
cluster state
..


Patch Set 1:

(1 comment)

https://asterix-gerrit.ics.uci.edu/#/c/3394/1/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/utils/ClusterStateManager.java
File 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/utils/ClusterStateManager.java:

https://asterix-gerrit.ics.uci.edu/#/c/3394/1/asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/utils/ClusterStateManager.java@195
PS1, Line 195: if (state != ClusterState.ACTIVE && state != 
ClusterState.RECOVERING) {
 : setState(ClusterState.PENDING);
 : }
is this needed? looks like some unnecessary transient state that will be 
changed no matter what in the checks to follow



--
To view, visit https://asterix-gerrit.ics.uci.edu/3394
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I863f8bf3fe1ce8d59522c9a28a1283006ffa414c
Gerrit-Change-Number: 3394
Gerrit-PatchSet: 1
Gerrit-Owner: Michael Blow 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Reviewer: Till Westmann 
Gerrit-Comment-Date: Wed, 15 May 2019 01:12:55 +
Gerrit-HasComments: Yes

Change in asterixdb[master]: [NO ISSUE][CLUS] Ignore Received Heartbeats Before App Initialization

[Murtadha Hubail (Code Review)]

Murtadha Hubail has submitted this change and it was merged. ( 
https://asterix-gerrit.ics.uci.edu/3395 )

Change subject: [NO ISSUE][CLUS] Ignore Received Heartbeats Before App 
Initialization
..

[NO ISSUE][CLUS] Ignore Received Heartbeats Before App Initialization

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Ignore received node heartbeats on CC when the CC executor
  has not been initialized since nodes are not registered yet.

Change-Id: I65ef92ae3179214f1efecd1bb44da4772b7a3dd9
Reviewed-on: https://asterix-gerrit.ics.uci.edu/3395
Tested-by: Jenkins 
Integration-Tests: Jenkins 
Contrib: Jenkins 
Reviewed-by: Michael Blow 
---
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerIPCI.java
1 file changed, 10 insertions(+), 3 deletions(-)

Approvals:
  Jenkins: Verified; ; Verified
  Anon. E. Moose (1000171):
  Michael Blow: Looks good to me, approved

Objections:
  Jenkins: Violations found



diff --git 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerIPCI.java
 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerIPCI.java
index 78ed9b9..e022dfe 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerIPCI.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerIPCI.java
@@ -19,6 +19,7 @@
 package org.apache.hyracks.control.cc;

 import java.util.Map;
+import java.util.concurrent.ExecutorService;

 import org.apache.hyracks.api.client.NodeControllerInfo;
 import org.apache.hyracks.control.cc.work.ApplicationMessageWork;
@@ -73,9 +74,7 @@
 ccs.getWorkQueue().schedule(new 
UnregisterNodeWork(ccs.getNodeManager(), unf.getNodeId()));
 break;
 case NODE_HEARTBEAT:
-CCNCFunctions.NodeHeartbeatFunction nhf = 
(CCNCFunctions.NodeHeartbeatFunction) fn;
-ccs.getExecutor().execute(
-new NodeHeartbeatWork(ccs, nhf.getNodeId(), 
nhf.getHeartbeatData(), nhf.getNcAddress()));
+processNodeHeartbeat(ccs, fn);
 break;
 case NOTIFY_JOBLET_CLEANUP:
 CCNCFunctions.NotifyJobletCleanupFunction njcf = 
(CCNCFunctions.NotifyJobletCleanupFunction) fn;
@@ -170,4 +169,12 @@
 LOGGER.warn("Unknown function: " + fn.getFunctionId());
 }
 }
+
+private static void processNodeHeartbeat(ClusterControllerService ccs, 
CCNCFunctions.Function fn) {
+final ExecutorService executor = ccs.getExecutor();
+if (executor != null) {
+CCNCFunctions.NodeHeartbeatFunction nhf = 
(CCNCFunctions.NodeHeartbeatFunction) fn;
+executor.execute(new NodeHeartbeatWork(ccs, nhf.getNodeId(), 
nhf.getHeartbeatData(), nhf.getNcAddress()));
+}
+}
 }

--
To view, visit https://asterix-gerrit.ics.uci.edu/3395
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I65ef92ae3179214f1efecd1bb44da4772b7a3dd9
Gerrit-Change-Number: 3395
Gerrit-PatchSet: 3
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Reviewer: Till Westmann 

Change in asterixdb[master]: [NO ISSUE][CLUS] Ignore Received Heartbeats Before App Initialization

[Murtadha Hubail (Code Review)]

Hello Anon. E. Moose (1000171), Till Westmann, Jenkins, Michael Blow,

I'd like you to reexamine a change. Please visit

https://asterix-gerrit.ics.uci.edu/3395

to look at the new patch set (#2).

Change subject: [NO ISSUE][CLUS] Ignore Received Heartbeats Before App 
Initialization
..

[NO ISSUE][CLUS] Ignore Received Heartbeats Before App Initialization

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Ignore received node heartbeats on CC when the CC executor
  has not been initialized since nodes are not registered yet.

Change-Id: I65ef92ae3179214f1efecd1bb44da4772b7a3dd9
---
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerIPCI.java
1 file changed, 10 insertions(+), 3 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/95/3395/2
--
To view, visit https://asterix-gerrit.ics.uci.edu/3395
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I65ef92ae3179214f1efecd1bb44da4772b7a3dd9
Gerrit-Change-Number: 3395
Gerrit-PatchSet: 2
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Till Westmann 

Change in asterixdb[master]: [NO ISSUE][CLUS] Ignore Received Heartbeats Before App Initialization

[Murtadha Hubail (Code Review)]

Murtadha Hubail has uploaded this change for review. ( 
https://asterix-gerrit.ics.uci.edu/3395


Change subject: [NO ISSUE][CLUS] Ignore Received Heartbeats Before App 
Initialization
..

[NO ISSUE][CLUS] Ignore Received Heartbeats Before App Initialization

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Ignore received node heartbeats on CC when the CC has not
  been initialized since nodes are not registered yet.

Change-Id: I65ef92ae3179214f1efecd1bb44da4772b7a3dd9
---
M 
hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerIPCI.java
1 file changed, 9 insertions(+), 3 deletions(-)



  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/95/3395/1

diff --git 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerIPCI.java
 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerIPCI.java
index 78ed9b9..97e05da 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerIPCI.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-cc/src/main/java/org/apache/hyracks/control/cc/ClusterControllerIPCI.java
@@ -19,6 +19,7 @@
 package org.apache.hyracks.control.cc;

 import java.util.Map;
+import java.util.concurrent.ExecutorService;

 import org.apache.hyracks.api.client.NodeControllerInfo;
 import org.apache.hyracks.control.cc.work.ApplicationMessageWork;
@@ -73,9 +74,7 @@
 ccs.getWorkQueue().schedule(new 
UnregisterNodeWork(ccs.getNodeManager(), unf.getNodeId()));
 break;
 case NODE_HEARTBEAT:
-CCNCFunctions.NodeHeartbeatFunction nhf = 
(CCNCFunctions.NodeHeartbeatFunction) fn;
-ccs.getExecutor().execute(
-new NodeHeartbeatWork(ccs, nhf.getNodeId(), 
nhf.getHeartbeatData(), nhf.getNcAddress()));
+processNodeHeartbeat(ccs, 
(CCNCFunctions.NodeHeartbeatFunction) fn);
 break;
 case NOTIFY_JOBLET_CLEANUP:
 CCNCFunctions.NotifyJobletCleanupFunction njcf = 
(CCNCFunctions.NotifyJobletCleanupFunction) fn;
@@ -170,4 +169,11 @@
 LOGGER.warn("Unknown function: " + fn.getFunctionId());
 }
 }
+
+private static void processNodeHeartbeat(ClusterControllerService ccs, 
CCNCFunctions.NodeHeartbeatFunction fn) {
+final ExecutorService executor = ccs.getExecutor();
+if (executor != null) {
+executor.execute(new NodeHeartbeatWork(ccs, fn.getNodeId(), 
fn.getHeartbeatData(), fn.getNcAddress()));
+}
+}
 }

--
To view, visit https://asterix-gerrit.ics.uci.edu/3395
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I65ef92ae3179214f1efecd1bb44da4772b7a3dd9
Gerrit-Change-Number: 3395
Gerrit-PatchSet: 1
Gerrit-Owner: Murtadha Hubail 

Change in asterixdb[master]: [ASTERIXDB-2563][API] Allow All Nodes To Receive Queries

[Murtadha Hubail (Code Review)]

Murtadha Hubail has submitted this change and it was merged. ( 
https://asterix-gerrit.ics.uci.edu/3388 )

Change subject: [ASTERIXDB-2563][API] Allow All Nodes To Receive Queries
..

[ASTERIXDB-2563][API] Allow All Nodes To Receive Queries

- user model changes: any node can receive query requests.
- storage format changes: no
- interface changes: no

Details:
- Add required servlets to NCs web server to allow them
  to receive query requests.
- Adapt test framework to use NCs endpoints for query requests
  when they are configured.
- Make SqlppExecutionTest use NCs endpoints for query requests.

Change-Id: I31f7937d46b4532c30e2201b3e434e62e5c4a4de
Reviewed-on: https://asterix-gerrit.ics.uci.edu/3388
Tested-by: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Till Westmann 
---
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplication.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/common/TestExecutor.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/runtime/SqlppExecutionTest.java
3 files changed, 45 insertions(+), 2 deletions(-)

Approvals:
  Jenkins: Verified; Verified
  Anon. E. Moose (1000171):
  Till Westmann: Looks good to me, approved

Objections:
  Jenkins: Violations found



diff --git 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplication.java
 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplication.java
index c3402f9..eba2049 100644
--- 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplication.java
+++ 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplication.java
@@ -18,13 +18,22 @@
  */
 package org.apache.asterix.hyracks.bootstrap;

+import static 
org.apache.asterix.api.http.server.ServletConstants.HYRACKS_CONNECTION_ATTR;
+import static org.apache.asterix.common.utils.Servlets.QUERY_RESULT;
+import static org.apache.asterix.common.utils.Servlets.QUERY_SERVICE;
+import static org.apache.asterix.common.utils.Servlets.QUERY_STATUS;
+
 import java.io.IOException;
 import java.util.ArrayList;
 import java.util.Arrays;
 import java.util.List;
 import java.util.Set;

+import org.apache.asterix.algebra.base.ILangExtension;
+import org.apache.asterix.api.http.server.NCQueryServiceServlet;
 import org.apache.asterix.api.http.server.NetDiagnosticsApiServlet;
+import org.apache.asterix.api.http.server.QueryResultApiServlet;
+import org.apache.asterix.api.http.server.QueryStatusApiServlet;
 import org.apache.asterix.api.http.server.ServletConstants;
 import org.apache.asterix.api.http.server.StorageApiServlet;
 import org.apache.asterix.app.config.ConfigValidator;
@@ -57,6 +66,7 @@
 import org.apache.asterix.common.utils.Servlets;
 import org.apache.asterix.common.utils.StorageConstants;
 import org.apache.asterix.common.utils.StoragePathUtil;
+import org.apache.asterix.compiler.provider.ILangCompilationProvider;
 import org.apache.asterix.messaging.MessagingChannelInterfaceFactory;
 import org.apache.asterix.messaging.NCMessageBroker;
 import 
org.apache.asterix.transaction.management.resource.PersistentLocalResourceRepository;
@@ -186,9 +196,16 @@
 HttpServer apiServer = new HttpServer(webManager.getBosses(), 
webManager.getWorkers(),
 externalProperties.getNcApiPort(), config);
 apiServer.setAttribute(ServletConstants.SERVICE_CONTEXT_ATTR, 
ncServiceCtx);
+apiServer.setAttribute(HYRACKS_CONNECTION_ATTR, 
getApplicationContext().getHcc());
 apiServer.addServlet(new StorageApiServlet(apiServer.ctx(), 
getApplicationContext(), Servlets.STORAGE));
 apiServer.addServlet(
 new NetDiagnosticsApiServlet(apiServer.ctx(), 
getApplicationContext(), Servlets.NET_DIAGNOSTICS));
+final ILangCompilationProvider sqlppCompilationProvider =
+
ncExtensionManager.getCompilationProvider(ILangExtension.Language.SQLPP);
+apiServer.addServlet(new NCQueryServiceServlet(apiServer.ctx(), new 
String[] { QUERY_SERVICE },
+getApplicationContext(), 
sqlppCompilationProvider.getLanguage(), sqlppCompilationProvider, null));
+apiServer.addServlet(new QueryStatusApiServlet(apiServer.ctx(), 
getApplicationContext(), QUERY_STATUS));
+apiServer.addServlet(new QueryResultApiServlet(apiServer.ctx(), 
getApplicationContext(), QUERY_RESULT));
 webManager.add(apiServer);
 }

diff --git 
a/asterixdb/asterix-app/src/test/java/org/apache/asterix/test/common/TestExecutor.java
 
b/asterixdb/asterix-app/src/test/java/org/apache/asterix/test/common/TestExecutor.java
index 5d5abdf..ab8f172 100644
--- 
a/asterixdb/asterix-app/src/test/java/org/apache/asterix/test/common/TestExecutor.java
+++ 
b/asterixdb/asterix-app/src/test/java/org/apache/asterix/test/common/TestExecutor.java
@@ -162,6 +162,7 @@

 private static final HashMap 

Change in asterixdb[stabilization-f69489]: [NO ISSUE][RT] Make StreamSelectRuntimeFactory Extensible

[Murtadha Hubail (Code Review)]

Murtadha Hubail has abandoned this change. ( 
https://asterix-gerrit.ics.uci.edu/3392 )

Change subject: [NO ISSUE][RT] Make StreamSelectRuntimeFactory Extensible
..


Abandoned
--
To view, visit https://asterix-gerrit.ics.uci.edu/3392
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: stabilization-f69489
Gerrit-MessageType: abandon
Gerrit-Change-Id: Iba02bca6a8792cd23c07f26d363162c7638c872e
Gerrit-Change-Number: 3392
Gerrit-PatchSet: 1
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Murtadha Hubail 

Change in asterixdb[stabilization-f69489]: [NO ISSUE][RT] Make StreamSelectRuntimeFactory Extensible

[Murtadha Hubail (Code Review)]

Murtadha Hubail has posted comments on this change. ( 
https://asterix-gerrit.ics.uci.edu/3392 )

Change subject: [NO ISSUE][RT] Make StreamSelectRuntimeFactory Extensible
..


Patch Set 1: Code-Review-2


--
To view, visit https://asterix-gerrit.ics.uci.edu/3392
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: stabilization-f69489
Gerrit-MessageType: comment
Gerrit-Change-Id: Iba02bca6a8792cd23c07f26d363162c7638c872e
Gerrit-Change-Number: 3392
Gerrit-PatchSet: 1
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Comment-Date: Sat, 11 May 2019 22:06:16 +
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE][OTH] Ensure HttpServer Channel is Initialized

[Murtadha Hubail (Code Review)]

Murtadha Hubail has submitted this change and it was merged. ( 
https://asterix-gerrit.ics.uci.edu/3389 )

Change subject: [NO ISSUE][OTH] Ensure HttpServer Channel is Initialized
..

[NO ISSUE][OTH] Ensure HttpServer Channel is Initialized

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Ensure the HttpServer channel is initialized when stopping
  the server to avoid NPE.

Change-Id: I5b7403e80f6118f99be46d166c6cfbee8d4305ac
Reviewed-on: https://asterix-gerrit.ics.uci.edu/3389
Contrib: Jenkins 
Tested-by: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Dmitry Lychagin 
---
M 
hyracks-fullstack/hyracks/hyracks-http/src/main/java/org/apache/hyracks/http/server/HttpServer.java
1 file changed, 4 insertions(+), 2 deletions(-)

Approvals:
  Jenkins: Verified; ; Verified
  Anon. E. Moose (1000171):
  Dmitry Lychagin: Looks good to me, approved

Objections:
  Jenkins: Violations found



diff --git 
a/hyracks-fullstack/hyracks/hyracks-http/src/main/java/org/apache/hyracks/http/server/HttpServer.java
 
b/hyracks-fullstack/hyracks/hyracks-http/src/main/java/org/apache/hyracks/http/server/HttpServer.java
index 152f6c7..d9902da 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-http/src/main/java/org/apache/hyracks/http/server/HttpServer.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-http/src/main/java/org/apache/hyracks/http/server/HttpServer.java
@@ -333,8 +333,10 @@
 } catch (Exception e) {
 LOGGER.log(Level.ERROR, "Error while shutting down http server 
executor", e);
 }
-channel.close();
-channel.closeFuture().sync();
+if (channel != null) {
+channel.close();
+channel.closeFuture().sync();
+}
 }

 public IServlet getServlet(FullHttpRequest request) {

--
To view, visit https://asterix-gerrit.ics.uci.edu/3389
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I5b7403e80f6118f99be46d166c6cfbee8d4305ac
Gerrit-Change-Number: 3389
Gerrit-PatchSet: 2
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Hussain Towaileb 
Gerrit-Reviewer: Ian Maxon 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Reviewer: Till Westmann 

Change in asterixdb[master]: [NO ISSUE][OTH] Ensure HttpServer Channel is Initialized

[Murtadha Hubail (Code Review)]

Murtadha Hubail has uploaded this change for review. ( 
https://asterix-gerrit.ics.uci.edu/3389


Change subject: [NO ISSUE][OTH] Ensure HttpServer Channel is Initialized
..

[NO ISSUE][OTH] Ensure HttpServer Channel is Initialized

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Ensure the HttpServer channel is initialized when stopping
  the server to avoid NPE.

Change-Id: I5b7403e80f6118f99be46d166c6cfbee8d4305ac
---
M 
hyracks-fullstack/hyracks/hyracks-http/src/main/java/org/apache/hyracks/http/server/HttpServer.java
1 file changed, 4 insertions(+), 2 deletions(-)



  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/89/3389/1

diff --git 
a/hyracks-fullstack/hyracks/hyracks-http/src/main/java/org/apache/hyracks/http/server/HttpServer.java
 
b/hyracks-fullstack/hyracks/hyracks-http/src/main/java/org/apache/hyracks/http/server/HttpServer.java
index 152f6c7..d9902da 100644
--- 
a/hyracks-fullstack/hyracks/hyracks-http/src/main/java/org/apache/hyracks/http/server/HttpServer.java
+++ 
b/hyracks-fullstack/hyracks/hyracks-http/src/main/java/org/apache/hyracks/http/server/HttpServer.java
@@ -333,8 +333,10 @@
 } catch (Exception e) {
 LOGGER.log(Level.ERROR, "Error while shutting down http server 
executor", e);
 }
-channel.close();
-channel.closeFuture().sync();
+if (channel != null) {
+channel.close();
+channel.closeFuture().sync();
+}
 }

 public IServlet getServlet(FullHttpRequest request) {

--
To view, visit https://asterix-gerrit.ics.uci.edu/3389
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I5b7403e80f6118f99be46d166c6cfbee8d4305ac
Gerrit-Change-Number: 3389
Gerrit-PatchSet: 1
Gerrit-Owner: Murtadha Hubail 

Change in asterixdb[master]: [ASTERIXDB-2563][API] Allow All Nodes To Receive Queries

[Murtadha Hubail (Code Review)]

Hello Anon. E. Moose (1000171), Jenkins,

I'd like you to reexamine a change. Please visit

https://asterix-gerrit.ics.uci.edu/3388

to look at the new patch set (#2).

Change subject: [ASTERIXDB-2563][API] Allow All Nodes To Receive Queries
..

[ASTERIXDB-2563][API] Allow All Nodes To Receive Queries

- user model changes: any node can receive query requests.
- storage format changes: no
- interface changes: no

Details:
- Add required servlets to NCs web server to allow them
  to receive query requests.
- Adapt test framework to use NCs endpoints for query requests
  when they are configured.
- Make SqlppExecutionTest use NCs endpoints for query requests.

Change-Id: I31f7937d46b4532c30e2201b3e434e62e5c4a4de
---
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplication.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/common/TestExecutor.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/runtime/SqlppExecutionTest.java
3 files changed, 45 insertions(+), 2 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/88/3388/2
--
To view, visit https://asterix-gerrit.ics.uci.edu/3388
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I31f7937d46b4532c30e2201b3e434e62e5c4a4de
Gerrit-Change-Number: 3388
Gerrit-PatchSet: 2
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 

Change in asterixdb[master]: [ASTERIXDB-2563][API] Allow All Nodes To Receive Queries

[Murtadha Hubail (Code Review)]

Murtadha Hubail has uploaded this change for review. ( 
https://asterix-gerrit.ics.uci.edu/3388


Change subject: [ASTERIXDB-2563][API] Allow All Nodes To Receive Queries
..

[ASTERIXDB-2563][API] Allow All Nodes To Receive Queries

- user model changes: any node can receive query requests.
- storage format changes: no
- interface changes: no

Details:
- Add required servlets to NCs web server to allow them
  to receive query requests.
- Adapt test framework to use NCs endpoints for query requests
  when they are configured.
- Make SqlppExecutionTest use NCs endpoints for query requests.

Change-Id: I31f7937d46b4532c30e2201b3e434e62e5c4a4de
---
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplication.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/common/TestExecutor.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/runtime/SqlppExecutionTest.java
3 files changed, 46 insertions(+), 2 deletions(-)



  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/88/3388/1

diff --git 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplication.java
 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplication.java
index c3402f9..eba2049 100644
--- 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplication.java
+++ 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplication.java
@@ -18,13 +18,22 @@
  */
 package org.apache.asterix.hyracks.bootstrap;

+import static 
org.apache.asterix.api.http.server.ServletConstants.HYRACKS_CONNECTION_ATTR;
+import static org.apache.asterix.common.utils.Servlets.QUERY_RESULT;
+import static org.apache.asterix.common.utils.Servlets.QUERY_SERVICE;
+import static org.apache.asterix.common.utils.Servlets.QUERY_STATUS;
+
 import java.io.IOException;
 import java.util.ArrayList;
 import java.util.Arrays;
 import java.util.List;
 import java.util.Set;

+import org.apache.asterix.algebra.base.ILangExtension;
+import org.apache.asterix.api.http.server.NCQueryServiceServlet;
 import org.apache.asterix.api.http.server.NetDiagnosticsApiServlet;
+import org.apache.asterix.api.http.server.QueryResultApiServlet;
+import org.apache.asterix.api.http.server.QueryStatusApiServlet;
 import org.apache.asterix.api.http.server.ServletConstants;
 import org.apache.asterix.api.http.server.StorageApiServlet;
 import org.apache.asterix.app.config.ConfigValidator;
@@ -57,6 +66,7 @@
 import org.apache.asterix.common.utils.Servlets;
 import org.apache.asterix.common.utils.StorageConstants;
 import org.apache.asterix.common.utils.StoragePathUtil;
+import org.apache.asterix.compiler.provider.ILangCompilationProvider;
 import org.apache.asterix.messaging.MessagingChannelInterfaceFactory;
 import org.apache.asterix.messaging.NCMessageBroker;
 import 
org.apache.asterix.transaction.management.resource.PersistentLocalResourceRepository;
@@ -186,9 +196,16 @@
 HttpServer apiServer = new HttpServer(webManager.getBosses(), 
webManager.getWorkers(),
 externalProperties.getNcApiPort(), config);
 apiServer.setAttribute(ServletConstants.SERVICE_CONTEXT_ATTR, 
ncServiceCtx);
+apiServer.setAttribute(HYRACKS_CONNECTION_ATTR, 
getApplicationContext().getHcc());
 apiServer.addServlet(new StorageApiServlet(apiServer.ctx(), 
getApplicationContext(), Servlets.STORAGE));
 apiServer.addServlet(
 new NetDiagnosticsApiServlet(apiServer.ctx(), 
getApplicationContext(), Servlets.NET_DIAGNOSTICS));
+final ILangCompilationProvider sqlppCompilationProvider =
+
ncExtensionManager.getCompilationProvider(ILangExtension.Language.SQLPP);
+apiServer.addServlet(new NCQueryServiceServlet(apiServer.ctx(), new 
String[] { QUERY_SERVICE },
+getApplicationContext(), 
sqlppCompilationProvider.getLanguage(), sqlppCompilationProvider, null));
+apiServer.addServlet(new QueryStatusApiServlet(apiServer.ctx(), 
getApplicationContext(), QUERY_STATUS));
+apiServer.addServlet(new QueryResultApiServlet(apiServer.ctx(), 
getApplicationContext(), QUERY_RESULT));
 webManager.add(apiServer);
 }

diff --git 
a/asterixdb/asterix-app/src/test/java/org/apache/asterix/test/common/TestExecutor.java
 
b/asterixdb/asterix-app/src/test/java/org/apache/asterix/test/common/TestExecutor.java
index 217b065..163ab59 100644
--- 
a/asterixdb/asterix-app/src/test/java/org/apache/asterix/test/common/TestExecutor.java
+++ 
b/asterixdb/asterix-app/src/test/java/org/apache/asterix/test/common/TestExecutor.java
@@ -161,6 +161,7 @@

 private static final HashMap runningTestServers = 
new HashMap<>();
 private static Map ncEndPoints;
+private static List ncEndPointsList = new ArrayList<>();
 private static Map replicationAddress;

 private final List 

Change in asterixdb[master]: [NO ISSUE] show aggregate functions in JSON plan

[Murtadha Hubail (Code Review)]

Murtadha Hubail has posted comments on this change. ( 
https://asterix-gerrit.ics.uci.edu/3384 )

Change subject: [NO ISSUE] show aggregate functions in JSON plan
..


Patch Set 3: Verified+1 Code-Review+2 Integration-Tests+1


--
To view, visit https://asterix-gerrit.ics.uci.edu/3384
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie4ba74d8455844e3913a4d4b5e2761b5ed23028b
Gerrit-Change-Number: 3384
Gerrit-PatchSet: 3
Gerrit-Owner: Till Westmann 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Comment-Date: Wed, 08 May 2019 23:20:19 +
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE] show aggregate functions in JSON plan

[Murtadha Hubail (Code Review)]

Murtadha Hubail has posted comments on this change. ( 
https://asterix-gerrit.ics.uci.edu/3384 )

Change subject: [NO ISSUE] show aggregate functions in JSON plan
..


Patch Set 2: Verified+1 Code-Review+2 Integration-Tests+1


--
To view, visit https://asterix-gerrit.ics.uci.edu/3384
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie4ba74d8455844e3913a4d4b5e2761b5ed23028b
Gerrit-Change-Number: 3384
Gerrit-PatchSet: 2
Gerrit-Owner: Till Westmann 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Comment-Date: Wed, 08 May 2019 23:08:52 +
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE][FAIL] Cast Exception Clean Up

[Murtadha Hubail (Code Review)]

Murtadha Hubail has submitted this change and it was merged. ( 
https://asterix-gerrit.ics.uci.edu/3382 )

Change subject: [NO ISSUE][FAIL] Cast Exception Clean Up
..

[NO ISSUE][FAIL] Cast Exception Clean Up

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Use proper exception type for cast exception.
- Move cast exception message to error properties file.
- Improve cast exception message.
- Update test cases to new error message.

Change-Id: I07c6c3c344b930ecbfaf6111a89c62b5d13868d9
Reviewed-on: https://asterix-gerrit.ics.uci.edu/3382
Contrib: Jenkins 
Sonar-Qube: Jenkins 
Tested-by: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Ian Maxon 
---
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M asterixdb/asterix-common/src/main/resources/asx_errormsg/en.properties
M 
asterixdb/asterix-om/src/main/java/org/apache/asterix/om/pointables/cast/ARecordCaster.java
4 files changed, 5 insertions(+), 4 deletions(-)

Approvals:
  Jenkins: Verified; No violations found; ; Verified
  Ian Maxon: Looks good to me, approved

Objections:
  Anon. E. Moose (1000171): Violations found



diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml 
b/asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
index 8725af0..f16b187 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
@@ -1466,7 +1466,7 @@
 
   
 query-issue410
-Field type double can't be promoted to type 
string
+Field type double cannot be promoted to type 
string
   
 
 
diff --git 
a/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml 
b/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
index 1f093d7..baeb7c5 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
@@ -7429,7 +7429,7 @@
 
   
 query-issue410
-Field type double can't be promoted to type 
string
+Field type double cannot be promoted to type 
string
 false
   
 
diff --git 
a/asterixdb/asterix-common/src/main/resources/asx_errormsg/en.properties 
b/asterixdb/asterix-common/src/main/resources/asx_errormsg/en.properties
index a69a6cf..4243837 100644
--- a/asterixdb/asterix-common/src/main/resources/asx_errormsg/en.properties
+++ b/asterixdb/asterix-common/src/main/resources/asx_errormsg/en.properties
@@ -32,6 +32,7 @@
 4005 = Unsupported replication strategy %1$s

 # Type errors
+1 = Field type %1$s cannot be promoted to type %2$s
 2,1002 = Type mismatch: function %1$s expects its %2$s input parameter to be 
of type %3$s, but the actual input type is %4$s
 3,1003 = Type incompatibility: function %1$s gets incompatible input values: 
%2$s and %3$s
 4,1004 = Unsupported type: %1$s cannot process input type %2$s
diff --git 
a/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/pointables/cast/ARecordCaster.java
 
b/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/pointables/cast/ARecordCaster.java
index 96433ac..56bdfc5 100644
--- 
a/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/pointables/cast/ARecordCaster.java
+++ 
b/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/pointables/cast/ARecordCaster.java
@@ -29,6 +29,7 @@

 import org.apache.asterix.builders.RecordBuilder;
 import org.apache.asterix.common.exceptions.ErrorCode;
+import org.apache.asterix.common.exceptions.RuntimeDataException;
 import org.apache.asterix.om.pointables.ARecordVisitablePointable;
 import org.apache.asterix.om.pointables.PointableAllocator;
 import org.apache.asterix.om.pointables.base.DefaultOpenFieldType;
@@ -222,8 +223,7 @@
 fieldPermutation[reqFnPos] = fnPos;
 openFields[fnPos] = false;
 } else {
-throw new HyracksDataException(ErrorCode.ASTERIX, 
ErrorCode.CASTING_FIELD,
-"Field type %1$s can't be promoted to type 
%2$s", inputTypeTag, requiredTypeTag);
+throw new 
RuntimeDataException(ErrorCode.CASTING_FIELD, inputTypeTag, requiredTypeTag);
 }
 }
 fnStart++;

--
To view, visit https://asterix-gerrit.ics.uci.edu/3382
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I07c6c3c344b930ecbfaf6111a89c62b5d13868d9
Gerrit-Change-Number: 3382
Gerrit-PatchSet: 2
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Ian Maxon 

Change in asterixdb[master]: [NO ISSUE][FAIL] Cast Exception Clean Up

[Murtadha Hubail (Code Review)]

Murtadha Hubail has uploaded this change for review. ( 
https://asterix-gerrit.ics.uci.edu/3382


Change subject: [NO ISSUE][FAIL] Cast Exception Clean Up
..

[NO ISSUE][FAIL] Cast Exception Clean Up

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Use proper exception type for cast exception.
- Move cast exception message to error properties file.
- Improve cast exception message.
- Update test cases to new error message.

Change-Id: I07c6c3c344b930ecbfaf6111a89c62b5d13868d9
---
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
M asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
M asterixdb/asterix-common/src/main/resources/asx_errormsg/en.properties
M 
asterixdb/asterix-om/src/main/java/org/apache/asterix/om/pointables/cast/ARecordCaster.java
4 files changed, 5 insertions(+), 4 deletions(-)



  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/82/3382/1

diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml 
b/asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
index 8725af0..f16b187 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/testsuite.xml
@@ -1466,7 +1466,7 @@
 
   
 query-issue410
-Field type double can't be promoted to type 
string
+Field type double cannot be promoted to type 
string
   
 
 
diff --git 
a/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml 
b/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
index 1f093d7..baeb7c5 100644
--- a/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
+++ b/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml
@@ -7429,7 +7429,7 @@
 
   
 query-issue410
-Field type double can't be promoted to type 
string
+Field type double cannot be promoted to type 
string
 false
   
 
diff --git 
a/asterixdb/asterix-common/src/main/resources/asx_errormsg/en.properties 
b/asterixdb/asterix-common/src/main/resources/asx_errormsg/en.properties
index a69a6cf..4243837 100644
--- a/asterixdb/asterix-common/src/main/resources/asx_errormsg/en.properties
+++ b/asterixdb/asterix-common/src/main/resources/asx_errormsg/en.properties
@@ -32,6 +32,7 @@
 4005 = Unsupported replication strategy %1$s

 # Type errors
+1 = Field type %1$s cannot be promoted to type %2$s
 2,1002 = Type mismatch: function %1$s expects its %2$s input parameter to be 
of type %3$s, but the actual input type is %4$s
 3,1003 = Type incompatibility: function %1$s gets incompatible input values: 
%2$s and %3$s
 4,1004 = Unsupported type: %1$s cannot process input type %2$s
diff --git 
a/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/pointables/cast/ARecordCaster.java
 
b/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/pointables/cast/ARecordCaster.java
index 96433ac..56bdfc5 100644
--- 
a/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/pointables/cast/ARecordCaster.java
+++ 
b/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/pointables/cast/ARecordCaster.java
@@ -29,6 +29,7 @@

 import org.apache.asterix.builders.RecordBuilder;
 import org.apache.asterix.common.exceptions.ErrorCode;
+import org.apache.asterix.common.exceptions.RuntimeDataException;
 import org.apache.asterix.om.pointables.ARecordVisitablePointable;
 import org.apache.asterix.om.pointables.PointableAllocator;
 import org.apache.asterix.om.pointables.base.DefaultOpenFieldType;
@@ -222,8 +223,7 @@
 fieldPermutation[reqFnPos] = fnPos;
 openFields[fnPos] = false;
 } else {
-throw new HyracksDataException(ErrorCode.ASTERIX, 
ErrorCode.CASTING_FIELD,
-"Field type %1$s can't be promoted to type 
%2$s", inputTypeTag, requiredTypeTag);
+throw new 
RuntimeDataException(ErrorCode.CASTING_FIELD, inputTypeTag, requiredTypeTag);
 }
 }
 fnStart++;

--
To view, visit https://asterix-gerrit.ics.uci.edu/3382
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I07c6c3c344b930ecbfaf6111a89c62b5d13868d9
Gerrit-Change-Number: 3382
Gerrit-PatchSet: 1
Gerrit-Owner: Murtadha Hubail 

Change in asterixdb[master]: [NO ISSUE][OTR] GlobalConfig Cleanup

[Murtadha Hubail (Code Review)]

Murtadha Hubail has submitted this change and it was merged. ( 
https://asterix-gerrit.ics.uci.edu/3376 )

Change subject: [NO ISSUE][OTR] GlobalConfig Cleanup
..

[NO ISSUE][OTR] GlobalConfig Cleanup

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Remove usage of DEBUG flag in GlobalConfig.
- Remove unused methods/constants in GlobalConfig.
- Move storage constants from GlobalConfig to StorageConstants.

Change-Id: Ie59a646d7d2eedaf1360c97c355f827d80917adf
Reviewed-on: https://asterix-gerrit.ics.uci.edu/3376
Contrib: Jenkins 
Tested-by: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Ali Alsuliman 
---
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/GlobalConfig.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/utils/StorageConstants.java
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/bootstrap/MetadataBootstrap.java
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/MetadataProvider.java
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/SecondaryBTreeOperationsHelper.java
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/SecondaryInvertedIndexOperationsHelper.java
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/SecondaryRTreeOperationsHelper.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/serializable/std/AbstractSerializableAvgAggregateFunction.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/serializable/std/AbstractSerializableSingleVariableStatisticsAggregateFunction.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/serializable/std/AbstractSerializableSumAggregateFunction.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractAvgAggregateFunction.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractSingleVarStatisticsAggregateFunction.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractSumAggregateFunction.java
14 files changed, 43 insertions(+), 98 deletions(-)

Approvals:
  Jenkins: Verified; ; Verified
  Anon. E. Moose (1000171):
  Ali Alsuliman: Looks good to me, approved

Objections:
  Jenkins: Violations found



diff --git 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
index 42b573f..0c19303 100644
--- 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
+++ 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
@@ -72,6 +72,7 @@
 import org.apache.asterix.common.functions.FunctionSignature;
 import org.apache.asterix.common.utils.JobUtils;
 import org.apache.asterix.common.utils.JobUtils.ProgressState;
+import org.apache.asterix.common.utils.StorageConstants;
 import org.apache.asterix.compiler.provider.ILangCompilationProvider;
 import org.apache.asterix.external.indexing.ExternalFile;
 import org.apache.asterix.external.indexing.IndexingConstants;
@@ -587,8 +588,8 @@
 sourceLoc);

 if (compactionPolicy == null) {
-compactionPolicy = GlobalConfig.DEFAULT_COMPACTION_POLICY_NAME;
-compactionPolicyProperties = 
GlobalConfig.DEFAULT_COMPACTION_POLICY_PROPERTIES;
+compactionPolicy = 
StorageConstants.DEFAULT_COMPACTION_POLICY_NAME;
+compactionPolicyProperties = 
StorageConstants.DEFAULT_COMPACTION_POLICY_PROPERTIES;
 } else {
 validateCompactionPolicy(compactionPolicy, 
compactionPolicyProperties, mdTxnCtx, false, sourceLoc);
 }
@@ -627,8 +628,8 @@
 // If the dataset has a filter and the user didn't 
specify a merge
 // policy, then we will pick the
 // correlated-prefix as the default merge policy.
-compactionPolicy = 
GlobalConfig.DEFAULT_FILTERED_DATASET_COMPACTION_POLICY_NAME;
-compactionPolicyProperties = 
GlobalConfig.DEFAULT_COMPACTION_POLICY_PROPERTIES;
+compactionPolicy = 
StorageConstants.DEFAULT_FILTERED_DATASET_COMPACTION_POLICY_NAME;
+compactionPolicyProperties = 
StorageConstants.DEFAULT_COMPACTION_POLICY_PROPERTIES;
 }
 datasetDetails = new 
InternalDatasetDetails(InternalDatasetDetails.FileStructure.BTREE,
 InternalDatasetDetails.PartitioningStrategy.HASH, 

Change in asterixdb[master]: [NO ISSUE][HYR][RT] Handle calls to close() properly in hash join

[Murtadha Hubail (Code Review)]

Murtadha Hubail has posted comments on this change. ( 
https://asterix-gerrit.ics.uci.edu/3375 )

Change subject: [NO ISSUE][HYR][RT] Handle calls to close() properly in hash 
join
..


Patch Set 1: Code-Review+2


--
To view, visit https://asterix-gerrit.ics.uci.edu/3375
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I41fff7deed5e56818bfefb4d36b8018a9f512cbb
Gerrit-Change-Number: 3375
Gerrit-PatchSet: 1
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Comment-Date: Thu, 02 May 2019 19:11:07 +
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE][OTR] GlobalConfig Cleanup

[Murtadha Hubail (Code Review)]

Murtadha Hubail has uploaded this change for review. ( 
https://asterix-gerrit.ics.uci.edu/3376


Change subject: [NO ISSUE][OTR] GlobalConfig Cleanup
..

[NO ISSUE][OTR] GlobalConfig Cleanup

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Remove usage of DEBUG flag in GlobalConfig.
- Remove unused methods/constants in GlobalConfig.
- Move storage constants from GlobalConfig to StorageConstants.

Change-Id: Ie59a646d7d2eedaf1360c97c355f827d80917adf
---
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/GlobalConfig.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/utils/StorageConstants.java
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/bootstrap/MetadataBootstrap.java
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/MetadataProvider.java
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/SecondaryBTreeOperationsHelper.java
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/SecondaryInvertedIndexOperationsHelper.java
M 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/SecondaryRTreeOperationsHelper.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/serializable/std/AbstractSerializableAvgAggregateFunction.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/serializable/std/AbstractSerializableSingleVariableStatisticsAggregateFunction.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/serializable/std/AbstractSerializableSumAggregateFunction.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractAvgAggregateFunction.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractSingleVarStatisticsAggregateFunction.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/aggregates/std/AbstractSumAggregateFunction.java
14 files changed, 43 insertions(+), 98 deletions(-)



  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/76/3376/1

diff --git 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
index 42b573f..0c19303 100644
--- 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
+++ 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java
@@ -72,6 +72,7 @@
 import org.apache.asterix.common.functions.FunctionSignature;
 import org.apache.asterix.common.utils.JobUtils;
 import org.apache.asterix.common.utils.JobUtils.ProgressState;
+import org.apache.asterix.common.utils.StorageConstants;
 import org.apache.asterix.compiler.provider.ILangCompilationProvider;
 import org.apache.asterix.external.indexing.ExternalFile;
 import org.apache.asterix.external.indexing.IndexingConstants;
@@ -587,8 +588,8 @@
 sourceLoc);

 if (compactionPolicy == null) {
-compactionPolicy = GlobalConfig.DEFAULT_COMPACTION_POLICY_NAME;
-compactionPolicyProperties = 
GlobalConfig.DEFAULT_COMPACTION_POLICY_PROPERTIES;
+compactionPolicy = 
StorageConstants.DEFAULT_COMPACTION_POLICY_NAME;
+compactionPolicyProperties = 
StorageConstants.DEFAULT_COMPACTION_POLICY_PROPERTIES;
 } else {
 validateCompactionPolicy(compactionPolicy, 
compactionPolicyProperties, mdTxnCtx, false, sourceLoc);
 }
@@ -627,8 +628,8 @@
 // If the dataset has a filter and the user didn't 
specify a merge
 // policy, then we will pick the
 // correlated-prefix as the default merge policy.
-compactionPolicy = 
GlobalConfig.DEFAULT_FILTERED_DATASET_COMPACTION_POLICY_NAME;
-compactionPolicyProperties = 
GlobalConfig.DEFAULT_COMPACTION_POLICY_PROPERTIES;
+compactionPolicy = 
StorageConstants.DEFAULT_FILTERED_DATASET_COMPACTION_POLICY_NAME;
+compactionPolicyProperties = 
StorageConstants.DEFAULT_COMPACTION_POLICY_PROPERTIES;
 }
 datasetDetails = new 
InternalDatasetDetails(InternalDatasetDetails.FileStructure.BTREE,
 InternalDatasetDetails.PartitioningStrategy.HASH, 
partitioningExprs, partitioningExprs,
diff --git 
a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/GlobalConfig.java
 

Change in asterixdb[master]: [NO ISSUE][API] Return Bad Request on Insufficient Cluster Resources

[Murtadha Hubail (Code Review)]

Murtadha Hubail has submitted this change and it was merged. ( 
https://asterix-gerrit.ics.uci.edu/3368 )

Change subject: [NO ISSUE][API] Return Bad Request on Insufficient Cluster 
Resources
..

[NO ISSUE][API] Return Bad Request on Insufficient Cluster Resources

- user model changes: yes
- storage format changes: no
- interface changes: no

Details:
- Return (400 Bad Request) when a job cannot be executed
  due to limited cluster resources.

Change-Id: I15500bfa4051a97fe3f5277d318b5a1187da225c
Reviewed-on: https://asterix-gerrit.ics.uci.edu/3368
Contrib: Jenkins 
Tested-by: Jenkins 
Integration-Tests: Jenkins 
Reviewed-by: Ali Alsuliman 
---
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryServiceServlet.java
1 file changed, 3 insertions(+), 0 deletions(-)

Approvals:
  Jenkins: Verified; ; Verified
  Ali Alsuliman: Looks good to me, approved

Objections:
  Jenkins: Violations found
  Anon. E. Moose (1000171): Violations found



diff --git 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryServiceServlet.java
 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryServiceServlet.java
index 1c66ce8..26e7430 100644
--- 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryServiceServlet.java
+++ 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryServiceServlet.java
@@ -23,6 +23,8 @@
 import static 
org.apache.asterix.common.exceptions.ErrorCode.REJECT_BAD_CLUSTER_STATE;
 import static 
org.apache.asterix.common.exceptions.ErrorCode.REJECT_NODE_UNREGISTERED;
 import static org.apache.asterix.common.exceptions.ErrorCode.REQUEST_TIMEOUT;
+import static org.apache.hyracks.api.exceptions.ErrorCode.HYRACKS;
+import static 
org.apache.hyracks.api.exceptions.ErrorCode.JOB_REQUIREMENTS_EXCEED_CAPACITY;

 import java.io.IOException;
 import java.io.PrintWriter;
@@ -675,6 +677,7 @@
 state.setStatus(ResultStatus.FATAL, 
HttpResponseStatus.SERVICE_UNAVAILABLE);
 break;
 case ASTERIX + NO_STATEMENT_PROVIDED:
+case HYRACKS + JOB_REQUIREMENTS_EXCEED_CAPACITY:
 state.setStatus(ResultStatus.FATAL, 
HttpResponseStatus.BAD_REQUEST);
 break;
 default:

--
To view, visit https://asterix-gerrit.ics.uci.edu/3368
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I15500bfa4051a97fe3f5277d318b5a1187da225c
Gerrit-Change-Number: 3368
Gerrit-PatchSet: 2
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Reviewer: Till Westmann 

Change in asterixdb[master]: [NO ISSUE][API] Return Bad Request on Insufficient Cluster Resources

[Murtadha Hubail (Code Review)]

Murtadha Hubail has uploaded this change for review. ( 
https://asterix-gerrit.ics.uci.edu/3368


Change subject: [NO ISSUE][API] Return Bad Request on Insufficient Cluster 
Resources
..

[NO ISSUE][API] Return Bad Request on Insufficient Cluster Resources

- user model changes: yes
- storage format changes: no
- interface changes: no

Details:
- Return (400 Bad Request) when a job cannot be executed
  due to limited cluster resources.

Change-Id: I15500bfa4051a97fe3f5277d318b5a1187da225c
---
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryServiceServlet.java
1 file changed, 3 insertions(+), 0 deletions(-)



  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/68/3368/1

diff --git 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryServiceServlet.java
 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryServiceServlet.java
index 1c66ce8..26e7430 100644
--- 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryServiceServlet.java
+++ 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/QueryServiceServlet.java
@@ -23,6 +23,8 @@
 import static 
org.apache.asterix.common.exceptions.ErrorCode.REJECT_BAD_CLUSTER_STATE;
 import static 
org.apache.asterix.common.exceptions.ErrorCode.REJECT_NODE_UNREGISTERED;
 import static org.apache.asterix.common.exceptions.ErrorCode.REQUEST_TIMEOUT;
+import static org.apache.hyracks.api.exceptions.ErrorCode.HYRACKS;
+import static 
org.apache.hyracks.api.exceptions.ErrorCode.JOB_REQUIREMENTS_EXCEED_CAPACITY;

 import java.io.IOException;
 import java.io.PrintWriter;
@@ -675,6 +677,7 @@
 state.setStatus(ResultStatus.FATAL, 
HttpResponseStatus.SERVICE_UNAVAILABLE);
 break;
 case ASTERIX + NO_STATEMENT_PROVIDED:
+case HYRACKS + JOB_REQUIREMENTS_EXCEED_CAPACITY:
 state.setStatus(ResultStatus.FATAL, 
HttpResponseStatus.BAD_REQUEST);
 break;
 default:

--
To view, visit https://asterix-gerrit.ics.uci.edu/3368
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I15500bfa4051a97fe3f5277d318b5a1187da225c
Gerrit-Change-Number: 3368
Gerrit-PatchSet: 1
Gerrit-Owner: Murtadha Hubail 

Change in asterixdb[master]: [NO ISSUE][STO]Ensure Bloom filters are not built for secondary indexes

[Murtadha Hubail (Code Review)]

Murtadha Hubail has posted comments on this change. ( 
https://asterix-gerrit.ics.uci.edu/3366 )

Change subject: [NO ISSUE][STO]Ensure Bloom filters are not built for secondary 
indexes
..


Patch Set 2: Code-Review+2


--
To view, visit https://asterix-gerrit.ics.uci.edu/3366
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib4272e5278bf76fa78af36ccf10d516da12c4810
Gerrit-Change-Number: 3366
Gerrit-PatchSet: 2
Gerrit-Owner: Luo Chen 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Luo Chen 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Comment-Date: Sun, 28 Apr 2019 14:46:27 +
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE][STO]Ensure Bloom filters are not built for secondary indexes

[Murtadha Hubail (Code Review)]

Murtadha Hubail has posted comments on this change. ( 
https://asterix-gerrit.ics.uci.edu/3366 )

Change subject: [NO ISSUE][STO]Ensure Bloom filters are not built for secondary 
indexes
..


Patch Set 1: Code-Review+2

(1 comment)

https://asterix-gerrit.ics.uci.edu/#/c/3366/1/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/BTreeResourceFactoryProvider.java
File 
asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/BTreeResourceFactoryProvider.java:

https://asterix-gerrit.ics.uci.edu/#/c/3366/1/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/BTreeResourceFactoryProvider.java@202
PS1, Line 202: has
have



--
To view, visit https://asterix-gerrit.ics.uci.edu/3366
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib4272e5278bf76fa78af36ccf10d516da12c4810
Gerrit-Change-Number: 3366
Gerrit-PatchSet: 1
Gerrit-Owner: Luo Chen 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Comment-Date: Sat, 27 Apr 2019 07:08:51 +
Gerrit-HasComments: Yes

Change in asterixdb[master]: [NO ISSUE] Introduce MissingNullInOutFunction annotation

[Murtadha Hubail (Code Review)]

Murtadha Hubail has posted comments on this change. ( 
https://asterix-gerrit.ics.uci.edu/3361 )

Change subject: [NO ISSUE] Introduce MissingNullInOutFunction annotation
..


Patch Set 2: Code-Review+2


--
To view, visit https://asterix-gerrit.ics.uci.edu/3361
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I253373ae4c8e23d9f5ed38f5fbbdefe9680e04b4
Gerrit-Change-Number: 3361
Gerrit-PatchSet: 2
Gerrit-Owner: Hussain Towaileb 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Hussain Towaileb 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Comment-Date: Thu, 25 Apr 2019 15:55:47 +
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE] Introduce MissingNullInOutFunction annotation

[Murtadha Hubail (Code Review)]

Murtadha Hubail has posted comments on this change. ( 
https://asterix-gerrit.ics.uci.edu/3361 )

Change subject: [NO ISSUE] Introduce MissingNullInOutFunction annotation
..


Patch Set 2: -Code-Review


--
To view, visit https://asterix-gerrit.ics.uci.edu/3361
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I253373ae4c8e23d9f5ed38f5fbbdefe9680e04b4
Gerrit-Change-Number: 3361
Gerrit-PatchSet: 2
Gerrit-Owner: Hussain Towaileb 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Hussain Towaileb 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Comment-Date: Thu, 25 Apr 2019 15:55:35 +
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE] Introduce MissingNullInOutFunction annotation

[Murtadha Hubail (Code Review)]

Murtadha Hubail has posted comments on this change. ( 
https://asterix-gerrit.ics.uci.edu/3361 )

Change subject: [NO ISSUE] Introduce MissingNullInOutFunction annotation
..


Patch Set 2: Code-Review+2


--
To view, visit https://asterix-gerrit.ics.uci.edu/3361
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I253373ae4c8e23d9f5ed38f5fbbdefe9680e04b4
Gerrit-Change-Number: 3361
Gerrit-PatchSet: 2
Gerrit-Owner: Hussain Towaileb 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Hussain Towaileb 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Comment-Date: Thu, 25 Apr 2019 15:55:25 +
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE] Introduce MissingNullInOutFunction annotation

[Murtadha Hubail (Code Review)]

Murtadha Hubail has posted comments on this change. ( 
https://asterix-gerrit.ics.uci.edu/3361 )

Change subject: [NO ISSUE] Introduce MissingNullInOutFunction annotation
..


Patch Set 1:

(3 comments)

https://asterix-gerrit.ics.uci.edu/#/c/3361/1/asterixdb/asterix-app/src/test/java/org/apache/asterix/runtime/ExceptionIT.java
File 
asterixdb/asterix-app/src/test/java/org/apache/asterix/runtime/ExceptionIT.java:

https://asterix-gerrit.ics.uci.edu/#/c/3361/1/asterixdb/asterix-app/src/test/java/org/apache/asterix/runtime/ExceptionIT.java@78
PS1, Line 78: System.out
define a logger in this class and use it instead of System.out


https://asterix-gerrit.ics.uci.edu/#/c/3361/1/asterixdb/asterix-app/src/test/java/org/apache/asterix/runtime/NullMissingTest.java
File 
asterixdb/asterix-app/src/test/java/org/apache/asterix/runtime/NullMissingTest.java:

https://asterix-gerrit.ics.uci.edu/#/c/3361/1/asterixdb/asterix-app/src/test/java/org/apache/asterix/runtime/NullMissingTest.java@89
PS1, Line 89: System.out
define a logger in this class and use it instead of System.out


https://asterix-gerrit.ics.uci.edu/#/c/3361/1/hyracks-fullstack/hyracks/hyracks-util/src/main/java/org/apache/hyracks/util/annotations/MissingNullInOutFunction.java
File 
hyracks-fullstack/hyracks/hyracks-util/src/main/java/org/apache/hyracks/util/annotations/MissingNullInOutFunction.java:

https://asterix-gerrit.ics.uci.edu/#/c/3361/1/hyracks-fullstack/hyracks/hyracks-util/src/main/java/org/apache/hyracks/util/annotations/MissingNullInOutFunction.java@19
PS1, Line 19: org.apache.hyracks.util.annotations
Move this to org.apache.asterix.common.annotations package. Hyracks shouldn't 
know about something called function.



--
To view, visit https://asterix-gerrit.ics.uci.edu/3361
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I253373ae4c8e23d9f5ed38f5fbbdefe9680e04b4
Gerrit-Change-Number: 3361
Gerrit-PatchSet: 1
Gerrit-Owner: Hussain Towaileb 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Comment-Date: Thu, 25 Apr 2019 09:17:28 +
Gerrit-HasComments: Yes

Change in asterixdb[master]: [NO ISSUE] Fix class name typo

[Murtadha Hubail (Code Review)]

Murtadha Hubail has posted comments on this change. ( 
https://asterix-gerrit.ics.uci.edu/3362 )

Change subject: [NO ISSUE] Fix class name typo
..


Patch Set 1: Code-Review+2


--
To view, visit https://asterix-gerrit.ics.uci.edu/3362
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ice71fd27ccf9ffd39e4ff368063c0715e33959db
Gerrit-Change-Number: 3362
Gerrit-PatchSet: 1
Gerrit-Owner: Hussain Towaileb 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Comment-Date: Wed, 24 Apr 2019 15:38:17 +
Gerrit-HasComments: No

Change in asterixdb[master]: [ASTERIXDB-2554][HYR] Add UTF8 and byte array comparator factories

[Murtadha Hubail (Code Review)]

Murtadha Hubail has posted comments on this change. ( 
https://asterix-gerrit.ics.uci.edu/3355 )

Change subject: [ASTERIXDB-2554][HYR] Add UTF8 and byte array comparator 
factories
..


Patch Set 1: Code-Review+2

(4 comments)

Feel free to forward the +2 after addressing the comments

https://asterix-gerrit.ics.uci.edu/#/c/3355/1//COMMIT_MSG
Commit Message:

https://asterix-gerrit.ics.uci.edu/#/c/3355/1//COMMIT_MSG@16
PS1, Line 16: PointableBinaryComparatorFactory
Let's add a comment to say why we still keep this factory and when to avoid 
using it.


https://asterix-gerrit.ics.uci.edu/#/c/3355/1/hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/src/main/java/org/apache/hyracks/data/std/primitive/ByteArrayPointable.java
File 
hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/src/main/java/org/apache/hyracks/data/std/primitive/ByteArrayPointable.java:

https://asterix-gerrit.ics.uci.edu/#/c/3355/1/hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/src/main/java/org/apache/hyracks/data/std/primitive/ByteArrayPointable.java@92
PS1, Line 92: public static int compare
move static method to the bottom


https://asterix-gerrit.ics.uci.edu/#/c/3355/1/hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/src/main/java/org/apache/hyracks/data/std/primitive/UTF8StringLowercasePointable.java
File 
hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/src/main/java/org/apache/hyracks/data/std/primitive/UTF8StringLowercasePointable.java:

https://asterix-gerrit.ics.uci.edu/#/c/3355/1/hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/src/main/java/org/apache/hyracks/data/std/primitive/UTF8StringLowercasePointable.java@73
PS1, Line 73: public static
move static method to the bottom


https://asterix-gerrit.ics.uci.edu/#/c/3355/1/hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/src/main/java/org/apache/hyracks/data/std/primitive/UTF8StringLowercaseTokenPointable.java
File 
hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/src/main/java/org/apache/hyracks/data/std/primitive/UTF8StringLowercaseTokenPointable.java:

https://asterix-gerrit.ics.uci.edu/#/c/3355/1/hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/src/main/java/org/apache/hyracks/data/std/primitive/UTF8StringLowercaseTokenPointable.java@75
PS1, Line 75: public static int
move static method to the bottom



--
To view, visit https://asterix-gerrit.ics.uci.edu/3355
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ied6a29210a3dc1ba9fd553fb0a67ff4340e4571f
Gerrit-Change-Number: 3355
Gerrit-PatchSet: 1
Gerrit-Owner: Ali Alsuliman 
Gerrit-Reviewer: Anon. E. Moose (1000171)
Gerrit-Reviewer: Dmitry Lychagin 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Michael Blow 
Gerrit-Reviewer: Murtadha Hubail 
Gerrit-Reviewer: Till Westmann 
Gerrit-Comment-Date: Tue, 23 Apr 2019 23:45:07 +
Gerrit-HasComments: Yes

Change in asterixdb[master]: [NO ISSUE][CONF] Introduce ConfigValidator

[Murtadha Hubail (Code Review)]

Murtadha Hubail has submitted this change and it was merged. ( 
https://asterix-gerrit.ics.uci.edu/3359 )

Change subject: [NO ISSUE][CONF] Introduce ConfigValidator
..

[NO ISSUE][CONF] Introduce ConfigValidator

- user model changes: no
- storage format changes: no
- interface changes: no

Details:
- Introduce ConfigValidator that can be used to validate
  runtime changes to configurations.
- On storage compression scheme configuration changes, ensure
  new value is a registered scheme.

Change-Id: I7e9dcde9c93feb2c9e2bdfe790f121b751d4c8ad
Reviewed-on: https://asterix-gerrit.ics.uci.edu/3359
Contrib: Jenkins 
Integration-Tests: Jenkins 
Tested-by: Jenkins 
Reviewed-by: Till Westmann 
---
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/config/ConfigValidator.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/NCAppRuntimeContext.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/CCApplication.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplication.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/api/IApplicationContext.java
A 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/api/IConfigValidator.java
A 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/api/IConfigValidatorFactory.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/api/INcApplicationContext.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/compression/CompressionManager.java
M 
asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/utils/CcApplicationContext.java
10 files changed, 141 insertions(+), 9 deletions(-)

Approvals:
  Jenkins: Verified; ; Verified
  Anon. E. Moose (1000171):
  Till Westmann: Looks good to me, approved

Objections:
  Jenkins: Violations found



diff --git 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/config/ConfigValidator.java
 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/config/ConfigValidator.java
new file mode 100644
index 000..138a2b4
--- /dev/null
+++ 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/config/ConfigValidator.java
@@ -0,0 +1,38 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.asterix.app.config;
+
+import org.apache.asterix.common.api.IConfigValidator;
+import org.apache.asterix.common.config.StorageProperties;
+import org.apache.asterix.runtime.compression.CompressionManager;
+import org.apache.hyracks.api.config.IOption;
+
+public class ConfigValidator implements IConfigValidator {
+
+@Override
+public void validate(IOption option, Object value) {
+boolean valid = true;
+if (option == StorageProperties.Option.STORAGE_COMPRESSION_BLOCK) {
+valid = CompressionManager.isRegisteredScheme((String) value);
+}
+if (!valid) {
+throw new IllegalArgumentException("Invalid value " + value + " 
for option " + option.name());
+}
+}
+}
diff --git 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/NCAppRuntimeContext.java
 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/NCAppRuntimeContext.java
index 8b8f5a0..182f982 100644
--- 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/NCAppRuntimeContext.java
+++ 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/NCAppRuntimeContext.java
@@ -31,6 +31,8 @@
 import java.util.stream.Collectors;

 import org.apache.asterix.active.ActiveManager;
+import org.apache.asterix.common.api.IConfigValidator;
+import org.apache.asterix.common.api.IConfigValidatorFactory;
 import org.apache.asterix.common.api.ICoordinationService;
 import org.apache.asterix.common.api.IDatasetLifecycleManager;
 import org.apache.asterix.common.api.IDatasetMemoryManager;
@@ -153,6 +155,7 @@
 private IReplicaManager replicaManager;
 private IReceptionist receptionist;
 private ICacheManager cacheManager;
+private IConfigValidator configValidator;

 public NCAppRuntimeContext(INCServiceContext ncServiceContext, 
List extensions,