subject:"Change in asterixdb\-bad\[master\]\: Redeploy channels and procedures during recovery"

Steven Jacobs has submitted this change and it was merged.

Change subject: Redeploy channels and procedures during recovery
..


Redeploy channels and procedures during recovery

Use the GlobalRecoveryManager extension to redeploy channels/procedures
Restart execution of channels during recovery
Some code cleanup
Added recovery test

Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
---
M asterix-bad/pom.xml
M asterix-bad/src/main/java/org/apache/asterix/bad/BADConstants.java
M asterix-bad/src/main/java/org/apache/asterix/bad/BADJobService.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADStatementExecutor.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/CreateChannelStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/CreateProcedureStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/ExecuteProcedureStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/BADMetadataRecordTypes.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/DeployedJobSpecEventListener.java
M asterix-bad/src/main/java/org/apache/asterix/bad/metadata/Procedure.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/ProcedureTupleTranslator.java
A 
asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADGlobalRecoveryManager.java
A 
asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADRecoveryExtension.java
M asterix-bad/src/main/resources/cc.conf
M 
asterix-bad/src/test/java/org/apache/asterix/bad/test/BADAsterixHyracksIntegrationUtil.java
M asterix-bad/src/test/java/org/apache/asterix/bad/test/BADListenerTest.java
A asterix-bad/src/test/java/org/apache/asterix/bad/test/BADRecoveryTest.java
A 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.1.ddl.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.10.update.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.11.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.12.update.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.13.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.14.ddl.sqlpp
A 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.2.script.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.3.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.4.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.5.update.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.6.sleep.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.7.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.8.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.9.query.sqlpp
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/restart_cluster/restart_cluster.11.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/restart_cluster/restart_cluster.13.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/restart_cluster/restart_cluster.3.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/restart_cluster/restart_cluster.4.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/restart_cluster/restart_cluster.7.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/restart_cluster/restart_cluster.8.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/restart_cluster/restart_cluster.9.adm
A 
asterix-bad/src/test/resources/recoveryts/scripts/recovery/restart_cluster/stop_and_start.sh
A 
asterix-bad/src/test/resources/recoveryts/scripts/setup_teardown/configure_and_validate.sh
A 
asterix-bad/src/test/resources/recoveryts/scripts/setup_teardown/stop_and_delete.sh
A asterix-bad/src/test/resources/recoveryts/testsuite.xml
R 
asterix-bad/src/test/resources/runtimets/queries/procedure/query_procedure/query_procedure.3.query.sqlpp
R 
asterix-bad/src/test/resources/runtimets/queries/procedure/query_procedure/query_procedure.5.query.sqlpp
R 
asterix-bad/src/test/resources/runtimets/queries/procedure/query_procedure_with_parameters/query_procedure_with_parameters.2.query.sqlpp
R 
asterix-bad/src/test/resources/runtimets/queries/procedure/query_procedure_with_parameters/query_procedure_with_parameters.3.query.sqlpp
M 
asterix-bad/src/test/resources/runtimets/results/procedure/create_procedure_check_metadata/create_procedure_check_metadata.1.adm
M

Change in asterixdb-bad[master]: Redeploy channels and procedures during recovery

2018-05-21 Thread Xikui Wang (Code Review)

Xikui Wang has posted comments on this change.

Change subject: Redeploy channels and procedures during recovery
..


Patch Set 15: Code-Review+2

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2641
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
Gerrit-PatchSet: 15
Gerrit-Project: asterixdb-bad
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: No

Change in asterixdb-bad[master]: Redeploy channels and procedures during recovery

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/2641

to look at the new patch set (#15).

Change subject: Redeploy channels and procedures during recovery
..

Redeploy channels and procedures during recovery

Use the GlobalRecoveryManager extension to redeploy channels/procedures
Restart execution of channels during recovery
Some code cleanup
Added recovery test

Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
---
M asterix-bad/pom.xml
M asterix-bad/src/main/java/org/apache/asterix/bad/BADConstants.java
M asterix-bad/src/main/java/org/apache/asterix/bad/BADJobService.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADStatementExecutor.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/CreateChannelStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/CreateProcedureStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/ExecuteProcedureStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/BADMetadataRecordTypes.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/DeployedJobSpecEventListener.java
M asterix-bad/src/main/java/org/apache/asterix/bad/metadata/Procedure.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/ProcedureTupleTranslator.java
A 
asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADGlobalRecoveryManager.java
A 
asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADRecoveryExtension.java
M asterix-bad/src/main/resources/cc.conf
M 
asterix-bad/src/test/java/org/apache/asterix/bad/test/BADAsterixHyracksIntegrationUtil.java
M asterix-bad/src/test/java/org/apache/asterix/bad/test/BADListenerTest.java
A asterix-bad/src/test/java/org/apache/asterix/bad/test/BADRecoveryTest.java
A 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.1.ddl.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.10.update.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.11.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.12.update.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.13.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.14.ddl.sqlpp
A 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.2.script.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.3.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.4.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.5.update.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.6.sleep.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.7.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.8.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.9.query.sqlpp
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/restart_cluster/restart_cluster.11.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/restart_cluster/restart_cluster.13.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/restart_cluster/restart_cluster.3.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/restart_cluster/restart_cluster.4.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/restart_cluster/restart_cluster.7.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/restart_cluster/restart_cluster.8.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/restart_cluster/restart_cluster.9.adm
A 
asterix-bad/src/test/resources/recoveryts/scripts/recovery/restart_cluster/stop_and_start.sh
A 
asterix-bad/src/test/resources/recoveryts/scripts/setup_teardown/configure_and_validate.sh
A 
asterix-bad/src/test/resources/recoveryts/scripts/setup_teardown/stop_and_delete.sh
A asterix-bad/src/test/resources/recoveryts/testsuite.xml
R 
asterix-bad/src/test/resources/runtimets/queries/procedure/query_procedure/query_procedure.3.query.sqlpp
R 
asterix-bad/src/test/resources/runtimets/queries/procedure/query_procedure/query_procedure.5.query.sqlpp
R 
asterix-bad/src/test/resources/runtimets/queries/procedure/query_procedure_with_parameters/query_procedure_with_parameters.2.query.sqlpp
R 
asterix-bad/src/test/resources/runtimets/queries/procedure/query_procedure_with_parameters/query_procedure_with_parameters.3.query.sqlpp
M

Change in asterixdb-bad[master]: Redeploy channels and procedures during recovery

2018-05-21 Thread Jenkins (Code Review)

Jenkins has posted comments on this change.

Change subject: Redeploy channels and procedures during recovery
..


Patch Set 15:

Build Started https://asterix-jenkins.ics.uci.edu/job/asterixbad-gerrit/295/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2641
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
Gerrit-PatchSet: 15
Gerrit-Project: asterixdb-bad
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: No

Change in asterixdb-bad[master]: Redeploy channels and procedures during recovery

2018-05-21 Thread Jenkins (Code Review)

Jenkins has posted comments on this change.

Change subject: Redeploy channels and procedures during recovery
..


Patch Set 14:

Build Started https://asterix-jenkins.ics.uci.edu/job/asterixbad-gerrit/294/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2641
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
Gerrit-PatchSet: 14
Gerrit-Project: asterixdb-bad
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: No

Change in asterixdb-bad[master]: Redeploy channels and procedures during recovery

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/2641

to look at the new patch set (#14).

Change subject: Redeploy channels and procedures during recovery
..

Redeploy channels and procedures during recovery

Use the GlobalRecoveryManager extension to redeploy channels/procedures
Restart execution of channels during recovery
Some code cleanup
Added recovery test

Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
---
M asterix-bad/pom.xml
M asterix-bad/src/main/java/org/apache/asterix/bad/BADConstants.java
M asterix-bad/src/main/java/org/apache/asterix/bad/BADJobService.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADStatementExecutor.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/CreateChannelStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/CreateProcedureStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/ExecuteProcedureStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/BADMetadataRecordTypes.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/DeployedJobSpecEventListener.java
M asterix-bad/src/main/java/org/apache/asterix/bad/metadata/Procedure.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/ProcedureTupleTranslator.java
A 
asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADGlobalRecoveryManager.java
A 
asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADRecoveryExtension.java
M asterix-bad/src/main/resources/cc.conf
M 
asterix-bad/src/test/java/org/apache/asterix/bad/test/BADAsterixHyracksIntegrationUtil.java
M asterix-bad/src/test/java/org/apache/asterix/bad/test/BADListenerTest.java
A asterix-bad/src/test/java/org/apache/asterix/bad/test/BADRecoveryTest.java
A 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.1.ddl.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.10.update.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.11.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.12.update.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.13.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.14.ddl.sqlpp
A 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.2.script.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.3.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.4.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.5.update.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.6.sleep.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.7.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.8.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.9.query.sqlpp
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/restart_cluster/restart_cluster.11.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/restart_cluster/restart_cluster.13.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/restart_cluster/restart_cluster.3.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/restart_cluster/restart_cluster.4.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/restart_cluster/restart_cluster.7.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/restart_cluster/restart_cluster.8.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/restart_cluster/restart_cluster.9.adm
A 
asterix-bad/src/test/resources/recoveryts/scripts/recovery/restart_cluster/stop_and_start.sh
A 
asterix-bad/src/test/resources/recoveryts/scripts/setup_teardown/configure_and_validate.sh
A 
asterix-bad/src/test/resources/recoveryts/scripts/setup_teardown/stop_and_delete.sh
A asterix-bad/src/test/resources/recoveryts/testsuite.xml
R 
asterix-bad/src/test/resources/runtimets/queries/procedure/query_procedure/query_procedure.3.query.sqlpp
R 
asterix-bad/src/test/resources/runtimets/queries/procedure/query_procedure/query_procedure.5.query.sqlpp
R 
asterix-bad/src/test/resources/runtimets/queries/procedure/query_procedure_with_parameters/query_procedure_with_parameters.2.query.sqlpp
R 
asterix-bad/src/test/resources/runtimets/queries/procedure/query_procedure_with_parameters/query_procedure_with_parameters.3.query.sqlpp
M

Change in asterixdb-bad[master]: Redeploy channels and procedures during recovery

2018-05-21 Thread Jenkins (Code Review)

Jenkins has posted comments on this change.

Change subject: Redeploy channels and procedures during recovery
..


Patch Set 13:

Build Started https://asterix-jenkins.ics.uci.edu/job/asterixbad-gerrit/293/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2641
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
Gerrit-PatchSet: 13
Gerrit-Project: asterixdb-bad
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: No

Change in asterixdb-bad[master]: Redeploy channels and procedures during recovery

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/2641

to look at the new patch set (#13).

Change subject: Redeploy channels and procedures during recovery
..

Redeploy channels and procedures during recovery

Use the GlobalRecoveryManager extension to redeploy channels/procedures
Restart execution of channels during recovery
Some code cleanup
Added recovery test

Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
---
M asterix-bad/pom.xml
M asterix-bad/src/main/java/org/apache/asterix/bad/BADConstants.java
M asterix-bad/src/main/java/org/apache/asterix/bad/BADJobService.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADStatementExecutor.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/CreateChannelStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/CreateProcedureStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/ExecuteProcedureStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/BADMetadataRecordTypes.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/DeployedJobSpecEventListener.java
M asterix-bad/src/main/java/org/apache/asterix/bad/metadata/Procedure.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/ProcedureTupleTranslator.java
A 
asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADGlobalRecoveryManager.java
A 
asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADRecoveryExtension.java
M asterix-bad/src/main/resources/cc.conf
M 
asterix-bad/src/test/java/org/apache/asterix/bad/test/BADAsterixHyracksIntegrationUtil.java
M asterix-bad/src/test/java/org/apache/asterix/bad/test/BADListenerTest.java
A asterix-bad/src/test/java/org/apache/asterix/bad/test/BADRecoveryTest.java
A 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.1.ddl.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.10.update.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.11.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.12.update.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.13.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.14.ddl.sqlpp
A 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.2.script.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.3.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.4.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.5.update.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.6.sleep.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.7.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.8.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/restart_cluster/restart_cluster.9.query.sqlpp
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/restart_cluster/restart_cluster.11.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/restart_cluster/restart_cluster.13.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/restart_cluster/restart_cluster.3.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/restart_cluster/restart_cluster.4.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/restart_cluster/restart_cluster.7.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/restart_cluster/restart_cluster.8.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/restart_cluster/restart_cluster.9.adm
A 
asterix-bad/src/test/resources/recoveryts/scripts/recovery/restart_cluster/stop_and_start.sh
A 
asterix-bad/src/test/resources/recoveryts/scripts/setup_teardown/configure_and_validate.sh
A 
asterix-bad/src/test/resources/recoveryts/scripts/setup_teardown/stop_and_delete.sh
A asterix-bad/src/test/resources/recoveryts/testsuite.xml
R 
asterix-bad/src/test/resources/runtimets/queries/procedure/query_procedure/query_procedure.3.query.sqlpp
R 
asterix-bad/src/test/resources/runtimets/queries/procedure/query_procedure/query_procedure.5.query.sqlpp
R 
asterix-bad/src/test/resources/runtimets/queries/procedure/query_procedure_with_parameters/query_procedure_with_parameters.2.query.sqlpp
R 
asterix-bad/src/test/resources/runtimets/queries/procedure/query_procedure_with_parameters/query_procedure_with_parameters.3.query.sqlpp
M

Change in asterixdb-bad[master]: Redeploy channels and procedures during recovery

Jenkins has posted comments on this change.

Change subject: Redeploy channels and procedures during recovery
..


Patch Set 12:

Build Started https://asterix-jenkins.ics.uci.edu/job/asterixbad-gerrit/292/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2641
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb-bad
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: No

Change in asterixdb-bad[master]: Redeploy channels and procedures during recovery

Steven Jacobs has posted comments on this change.

Change subject: Redeploy channels and procedures during recovery
..


Patch Set 11:

(1 comment)

https://asterix-gerrit.ics.uci.edu/#/c/2641/11/asterix-bad/src/test/java/org/apache/asterix/bad/test/BADRecoveryTest.java
File asterix-bad/src/test/java/org/apache/asterix/bad/test/BADRecoveryTest.java:

Line 86: if (testNumber == 1) {
> The cluster gets stopped after the 0th test (line 95) and started again bef
In case this isn't very clear from looking, the three tests are as follows:
0) Test 0 creates the channels and procedures
The cluster is restarted
1) Test 1 checks whether the channels started again successfully 
2) Test 2 checks whether the procedures are still usable


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2641
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
Gerrit-PatchSet: 11
Gerrit-Project: asterixdb-bad
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: Yes

Change in asterixdb-bad[master]: Redeploy channels and procedures during recovery

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/2641

to look at the new patch set (#12).

Change subject: Redeploy channels and procedures during recovery
..

Redeploy channels and procedures during recovery

Use the GlobalRecoveryManager extension to redeploy channels/procedures
Restart execution of channels during recovery
Some code cleanup

Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
---
M asterix-bad/pom.xml
M asterix-bad/src/main/java/org/apache/asterix/bad/BADConstants.java
M asterix-bad/src/main/java/org/apache/asterix/bad/BADJobService.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADStatementExecutor.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/CreateChannelStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/CreateProcedureStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/ExecuteProcedureStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/BADMetadataRecordTypes.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/DeployedJobSpecEventListener.java
M asterix-bad/src/main/java/org/apache/asterix/bad/metadata/Procedure.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/ProcedureTupleTranslator.java
A 
asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADGlobalRecoveryManager.java
A 
asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADRecoveryExtension.java
M asterix-bad/src/main/resources/cc.conf
M 
asterix-bad/src/test/java/org/apache/asterix/bad/test/BADAsterixHyracksIntegrationUtil.java
M asterix-bad/src/test/java/org/apache/asterix/bad/test/BADListenerTest.java
A asterix-bad/src/test/java/org/apache/asterix/bad/test/BADRecoveryTest.java
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_channels/check_channels.1.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_channels/check_channels.2.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_channels/check_channels.3.sleep.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_channels/check_channels.4.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_channels/check_channels.5.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_channels/check_channels.6.ddl.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_procedures/check_procedures.1.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_procedures/check_procedures.2.update.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_procedures/check_procedures.3.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_procedures/check_procedures.4.update.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_procedures/check_procedures.5.query.sqlpp
A asterix-bad/src/test/resources/recoveryts/queries/recovery/ddl/ddl.1.ddl.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/ddl/ddl.2.update.sqlpp
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/check_channels/check_channels.1.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/check_channels/check_channels.2.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/check_channels/check_channels.4.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/check_channels/check_channels.5.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/check_procedures/check_procedures.1.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/check_procedures/check_procedures.3.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/check_procedures/check_procedures.5.adm
A asterix-bad/src/test/resources/recoveryts/testsuite.xml
R 
asterix-bad/src/test/resources/runtimets/queries/procedure/query_procedure/query_procedure.3.query.sqlpp
R 
asterix-bad/src/test/resources/runtimets/queries/procedure/query_procedure/query_procedure.5.query.sqlpp
R 
asterix-bad/src/test/resources/runtimets/queries/procedure/query_procedure_with_parameters/query_procedure_with_parameters.2.query.sqlpp
R 
asterix-bad/src/test/resources/runtimets/queries/procedure/query_procedure_with_parameters/query_procedure_with_parameters.3.query.sqlpp
M 
asterix-bad/src/test/resources/runtimets/results/procedure/create_procedure_check_metadata/create_procedure_check_metadata.1.adm
M 
asterix-bad/src/test/resources/runtimets/results/procedure/query_procedure/query_procedure.3.adm
M 
asterix-bad/src/test/resources/runtimets/results/procedure/query_procedure/query_procedure.5.adm
M 
asterix-bad/src/test/resources/runtimets/results/procedure/query_procedure_with_parameters/query_procedure_with_parameters.2.adm
46 files changed, 620 insertions(+), 197

Change in asterixdb-bad[master]: Redeploy channels and procedures during recovery

Jenkins has posted comments on this change.

Change subject: Redeploy channels and procedures during recovery
..


Patch Set 12:

Build Started https://asterix-jenkins.ics.uci.edu/job/asterixbad-gerrit/291/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2641
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
Gerrit-PatchSet: 12
Gerrit-Project: asterixdb-bad
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: No

Change in asterixdb-bad[master]: Redeploy channels and procedures during recovery

Steven Jacobs has posted comments on this change.

Change subject: Redeploy channels and procedures during recovery
..


Patch Set 5:

(3 comments)

https://asterix-gerrit.ics.uci.edu/#/c/2641/5/asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADGlobalRecoveryManager.java
File 
asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADGlobalRecoveryManager.java:

PS5, Line 92: tionContext mdTxnCtx = 
MetadataManager.INSTANCE.beginTransaction();
: metadataProvider.setMetadataTxnContext(mdTxnCtx);
: 
: List channels = 
BADLangExtension.getAllChannels(mdTxnCtx);
: List procedures = 
BADLangExtension.getAllProcedures(mdTxnCtx);
: 
: MetadataManager.INSTANCE.commitTransaction(mdTxnCtx);
> sorry that I didn't explain my comment very well. What I meant is that it s
Done


Line 125: listener.suspend();
> the followup question is the system now is in suspend state right? Would it
the redeployJobSpec call (line 127) calls listener.resume() at the end.


PS5, Line 161: //Issue: need to store in metadata the information 
for running instances
> In other word, we are not storing the period of the repetitive procedures? 
We do store the period, but just having the period doesn't give us any 
guarantees about whether there were actually instances of the procedure running 
or not. In other words, you can create a repetitive procedure but not execute 
it.

There is a deeper question about maintaining multiple instances of repetitive 
procedures as well, which is why this is listed as TODO


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2641
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
Gerrit-PatchSet: 5
Gerrit-Project: asterixdb-bad
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: Yes

Change in asterixdb-bad[master]: Redeploy channels and procedures during recovery

Steven Jacobs has posted comments on this change.

Change subject: Redeploy channels and procedures during recovery
..

Patch Set 11:

(2 comments)

https://asterix-gerrit.ics.uci.edu/#/c/2641/11/asterix-bad/src/test/java/org/apache/asterix/bad/test/BADRecoveryTest.java
File asterix-bad/src/test/java/org/apache/asterix/bad/test/BADRecoveryTest.java:

Line 86: if (testNumber == 1) {
> Sorry for the 2nd comment. What does this mean?
The cluster gets stopped after the 0th test (line 95) and started again before
the 1st test (line 88). Other than that we don't need to keep restarting the
cluster.

https://asterix-gerrit.ics.uci.edu/#/c/2641/11/asterix-bad/src/test/resources/recoveryts/queries/recovery/check_channels/check_channels.3.sleep.sqlpp
File
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_channels/check_channels.3.sleep.sqlpp:

Line 25: 25000
> why sleep 25 seconds? (This will make the bad test longer for every patch r
I want to make absolutely certain that the channel is still executing. I know
the tests can run on slower machines. There are several issues that lead me to
make it this long:

1) I don't have confidence in making the channel period less than 5 seconds
(even on my local build there are rare instances taking longer than 1 second to
execute the tiny channel job.
2) The cluster deinit() has no precise timing, e.g. we don't know how long the
channel was running before the cluster stopped.
3) The cluster init() also has no precise timing, so there's no way to
guarantee how long the channel has been running before this sleep happens.

Essentially I start the comparison much higher than expected (< 3 when we
actually expect 0) to account for potential executions during the start and
stop, so I have to add 15 seconds worth of time to make sure that we get to at
least 3, and I also want to make sure that we increase, which is why I put 10
seconds more.

I think by giving this much wiggle room we make sure to prevent random failures
of the test.

--
To view, visit https://asterix-gerrit.ics.uci.edu/2641
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
Gerrit-PatchSet: 11
Gerrit-Project: asterixdb-bad
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs
Gerrit-Reviewer: Jenkins
Gerrit-Reviewer: Steven Jacobs
Gerrit-Reviewer: Xikui Wang
Gerrit-HasComments: Yes

Change in asterixdb-bad[master]: Redeploy channels and procedures during recovery

2018-05-17 Thread Xikui Wang (Code Review)

Xikui Wang has posted comments on this change.

Change subject: Redeploy channels and procedures during recovery
..


Patch Set 11:

(1 comment)

https://asterix-gerrit.ics.uci.edu/#/c/2641/11/asterix-bad/src/test/java/org/apache/asterix/bad/test/BADRecoveryTest.java
File asterix-bad/src/test/java/org/apache/asterix/bad/test/BADRecoveryTest.java:

Line 86: if (testNumber == 1) {
Sorry for the 2nd comment. What does this mean?


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2641
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
Gerrit-PatchSet: 11
Gerrit-Project: asterixdb-bad
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: Yes

Change in asterixdb-bad[master]: Redeploy channels and procedures during recovery

2018-05-17 Thread Xikui Wang (Code Review)

Xikui Wang has posted comments on this change.

Change subject: Redeploy channels and procedures during recovery
..

Patch Set 11:

(4 comments)

Several more follow-ups... We should be able to push it in in next round...

https://asterix-gerrit.ics.uci.edu/#/c/2641/5/asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADGlobalRecoveryManager.java
File
asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADGlobalRecoveryManager.java:

PS5, Line 92: lusterStateManager().refreshState();
: } catch (Exception e) {
: throw HyracksDataException.create(e);
: }
: }
:
: private void deployJobs(ICcApplicationContext appCtx, Lis
> Done
sorry that I didn't explain my comment very well. What I meant is that it seems
to me that the getChannels and getAllPredures can be done within the previous
txn. Is the 2nd txn necessary?

Line 125: metadataProvider = new MetadataProvider(appCtx,
MetadataBuiltinEntities.DEFAULT_DATAVERSE);
> yes
the followup question is the system now is in suspend state right? Would it be
a problem for the later execution? (It's never resumed. My observation from
reading codes the job run will overwrite the state immediately...)

PS5, Line 161: LOGGER.log(Level.SEVERE, entityId.getExtensionName()
+ " " + entityId.ge
> Sure. We can talk about it more in future, but the main issue is:
In other word, we are not storing the period of the repetitive procedures? I
guess for the non-repetitive case, deploy the job spec should be good enough I
guess...

Line 25: 25000
why sleep 25 seconds? (This will make the bad test longer for every patch
right?...)

--
To view, visit https://asterix-gerrit.ics.uci.edu/2641
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Change in asterixdb-bad[master]: Redeploy channels and procedures during recovery

Jenkins has posted comments on this change.

Change subject: Redeploy channels and procedures during recovery
..


Patch Set 11:

Build Started https://asterix-jenkins.ics.uci.edu/job/asterixbad-gerrit/290/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2641
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
Gerrit-PatchSet: 11
Gerrit-Project: asterixdb-bad
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: No

Change in asterixdb-bad[master]: Redeploy channels and procedures during recovery

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/2641

to look at the new patch set (#11).

Change subject: Redeploy channels and procedures during recovery
..

Redeploy channels and procedures during recovery

Use the GlobalRecoveryManager extension to redeploy channels/procedures
Restart execution of channels during recovery
Some code cleanup

Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
---
M asterix-bad/pom.xml
M asterix-bad/src/main/java/org/apache/asterix/bad/BADConstants.java
M asterix-bad/src/main/java/org/apache/asterix/bad/BADJobService.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADStatementExecutor.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/CreateChannelStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/CreateProcedureStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/ExecuteProcedureStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/BADMetadataRecordTypes.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/DeployedJobSpecEventListener.java
M asterix-bad/src/main/java/org/apache/asterix/bad/metadata/Procedure.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/ProcedureTupleTranslator.java
A 
asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADGlobalRecoveryManager.java
A 
asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADRecoveryExtension.java
M asterix-bad/src/main/resources/cc.conf
M 
asterix-bad/src/test/java/org/apache/asterix/bad/test/BADAsterixHyracksIntegrationUtil.java
M asterix-bad/src/test/java/org/apache/asterix/bad/test/BADListenerTest.java
A asterix-bad/src/test/java/org/apache/asterix/bad/test/BADRecoveryTest.java
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_channels/check_channels.1.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_channels/check_channels.2.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_channels/check_channels.3.sleep.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_channels/check_channels.4.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_channels/check_channels.5.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_channels/check_channels.6.ddl.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_procedures/check_procedures.1.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_procedures/check_procedures.2.update.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_procedures/check_procedures.3.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_procedures/check_procedures.4.update.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_procedures/check_procedures.5.query.sqlpp
A asterix-bad/src/test/resources/recoveryts/queries/recovery/ddl/ddl.1.ddl.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/ddl/ddl.2.update.sqlpp
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/check_channels/check_channels.1.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/check_channels/check_channels.2.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/check_channels/check_channels.4.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/check_channels/check_channels.5.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/check_procedures/check_procedures.1.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/check_procedures/check_procedures.3.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/check_procedures/check_procedures.5.adm
A asterix-bad/src/test/resources/recoveryts/testsuite.xml
R 
asterix-bad/src/test/resources/runtimets/queries/procedure/query_procedure/query_procedure.3.query.sqlpp
R 
asterix-bad/src/test/resources/runtimets/queries/procedure/query_procedure/query_procedure.5.query.sqlpp
R 
asterix-bad/src/test/resources/runtimets/queries/procedure/query_procedure_with_parameters/query_procedure_with_parameters.2.query.sqlpp
R 
asterix-bad/src/test/resources/runtimets/queries/procedure/query_procedure_with_parameters/query_procedure_with_parameters.3.query.sqlpp
M 
asterix-bad/src/test/resources/runtimets/results/procedure/create_procedure_check_metadata/create_procedure_check_metadata.1.adm
M 
asterix-bad/src/test/resources/runtimets/results/procedure/query_procedure/query_procedure.3.adm
M 
asterix-bad/src/test/resources/runtimets/results/procedure/query_procedure/query_procedure.5.adm
M 
asterix-bad/src/test/resources/runtimets/results/procedure/query_procedure_with_parameters/query_procedure_with_parameters.2.adm
46 files changed, 627 insertions(+), 197

Change in asterixdb-bad[master]: Redeploy channels and procedures during recovery

Jenkins has posted comments on this change.

Change subject: Redeploy channels and procedures during recovery
..


Patch Set 10:

Build Started https://asterix-jenkins.ics.uci.edu/job/asterixbad-gerrit/289/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2641
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
Gerrit-PatchSet: 10
Gerrit-Project: asterixdb-bad
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: No

Change in asterixdb-bad[master]: Redeploy channels and procedures during recovery

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/2641

to look at the new patch set (#10).

Change subject: Redeploy channels and procedures during recovery
..

Redeploy channels and procedures during recovery

Use the GlobalRecoveryManager extension to redeploy channels/procedures
Restart execution of channels during recovery
Some code cleanup

Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
---
M asterix-bad/pom.xml
M asterix-bad/src/main/java/org/apache/asterix/bad/BADConstants.java
M asterix-bad/src/main/java/org/apache/asterix/bad/BADJobService.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADStatementExecutor.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/CreateChannelStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/CreateProcedureStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/ExecuteProcedureStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/BADMetadataRecordTypes.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/DeployedJobSpecEventListener.java
M asterix-bad/src/main/java/org/apache/asterix/bad/metadata/Procedure.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/ProcedureTupleTranslator.java
A 
asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADGlobalRecoveryManager.java
A 
asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADRecoveryExtension.java
M asterix-bad/src/main/resources/cc.conf
M 
asterix-bad/src/test/java/org/apache/asterix/bad/test/BADAsterixHyracksIntegrationUtil.java
M asterix-bad/src/test/java/org/apache/asterix/bad/test/BADListenerTest.java
A asterix-bad/src/test/java/org/apache/asterix/bad/test/BADRecoveryTest.java
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_channels/check_channels.1.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_channels/check_channels.2.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_channels/check_channels.3.sleep.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_channels/check_channels.4.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_channels/check_channels.5.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_channels/check_channels.6.ddl.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_procedures/check_procedures.1.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_procedures/check_procedures.2.update.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_procedures/check_procedures.3.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_procedures/check_procedures.4.update.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_procedures/check_procedures.5.query.sqlpp
A asterix-bad/src/test/resources/recoveryts/queries/recovery/ddl/ddl.1.ddl.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/ddl/ddl.2.update.sqlpp
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/check_channels/check_channels.1.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/check_channels/check_channels.2.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/check_channels/check_channels.4.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/check_channels/check_channels.5.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/check_procedures/check_procedures.1.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/check_procedures/check_procedures.3.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/check_procedures/check_procedures.5.adm
R 
asterix-bad/src/test/resources/runtimets/queries/procedure/query_procedure/query_procedure.3.query.sqlpp
R 
asterix-bad/src/test/resources/runtimets/queries/procedure/query_procedure/query_procedure.5.query.sqlpp
R 
asterix-bad/src/test/resources/runtimets/queries/procedure/query_procedure_with_parameters/query_procedure_with_parameters.2.query.sqlpp
R 
asterix-bad/src/test/resources/runtimets/queries/procedure/query_procedure_with_parameters/query_procedure_with_parameters.3.query.sqlpp
M 
asterix-bad/src/test/resources/runtimets/results/procedure/create_procedure_check_metadata/create_procedure_check_metadata.1.adm
M 
asterix-bad/src/test/resources/runtimets/results/procedure/query_procedure/query_procedure.3.adm
M 
asterix-bad/src/test/resources/runtimets/results/procedure/query_procedure/query_procedure.5.adm
M 
asterix-bad/src/test/resources/runtimets/results/procedure/query_procedure_with_parameters/query_procedure_with_parameters.2.adm
45 files changed, 587 insertions(+), 197 deletions(-)


  git pull

Change in asterixdb-bad[master]: Redeploy channels and procedures during recovery

Jenkins has posted comments on this change.

Change subject: Redeploy channels and procedures during recovery
..


Patch Set 9:

Build Started https://asterix-jenkins.ics.uci.edu/job/asterixbad-gerrit/288/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2641
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
Gerrit-PatchSet: 9
Gerrit-Project: asterixdb-bad
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: No

Change in asterixdb-bad[master]: Redeploy channels and procedures during recovery

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/2641

to look at the new patch set (#9).

Change subject: Redeploy channels and procedures during recovery
..

Redeploy channels and procedures during recovery

Use the GlobalRecoveryManager extension to redeploy channels/procedures
Restart execution of channels during recovery
Some code cleanup

Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
---
M asterix-bad/pom.xml
M asterix-bad/src/main/java/org/apache/asterix/bad/BADConstants.java
M asterix-bad/src/main/java/org/apache/asterix/bad/BADJobService.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADStatementExecutor.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/CreateChannelStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/CreateProcedureStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/ExecuteProcedureStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/BADMetadataRecordTypes.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/DeployedJobSpecEventListener.java
M asterix-bad/src/main/java/org/apache/asterix/bad/metadata/Procedure.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/ProcedureTupleTranslator.java
A 
asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADGlobalRecoveryManager.java
A 
asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADRecoveryExtension.java
M asterix-bad/src/main/resources/cc.conf
M 
asterix-bad/src/test/java/org/apache/asterix/bad/test/BADAsterixHyracksIntegrationUtil.java
M asterix-bad/src/test/java/org/apache/asterix/bad/test/BADListenerTest.java
A asterix-bad/src/test/java/org/apache/asterix/bad/test/BADRecoveryTest.java
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_channels/check_channels.1.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_channels/check_channels.2.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_channels/check_channels.3.sleep.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_channels/check_channels.4.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_channels/check_channels.5.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_channels/check_channels.6.ddl.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_procedures/check_procedures.1.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_procedures/check_procedures.2.update.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_procedures/check_procedures.3.query.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_procedures/check_procedures.4.update.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/check_procedures/check_procedures.5.query.sqlpp
A asterix-bad/src/test/resources/recoveryts/queries/recovery/ddl/ddl.1.ddl.sqlpp
C 
asterix-bad/src/test/resources/recoveryts/queries/recovery/ddl/ddl.2.update.sqlpp
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/check_channels/check_channels.1.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/check_channels/check_channels.2.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/check_channels/check_channels.4.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/check_channels/check_channels.5.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/check_procedures/check_procedures.1.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/check_procedures/check_procedures.3.adm
A 
asterix-bad/src/test/resources/recoveryts/results/recovery/check_procedures/check_procedures.5.adm
R 
asterix-bad/src/test/resources/runtimets/queries/procedure/query_procedure/query_procedure.3.query.sqlpp
R 
asterix-bad/src/test/resources/runtimets/queries/procedure/query_procedure/query_procedure.5.query.sqlpp
R 
asterix-bad/src/test/resources/runtimets/queries/procedure/query_procedure_with_parameters/query_procedure_with_parameters.2.query.sqlpp
R 
asterix-bad/src/test/resources/runtimets/queries/procedure/query_procedure_with_parameters/query_procedure_with_parameters.3.query.sqlpp
M 
asterix-bad/src/test/resources/runtimets/results/procedure/create_procedure_check_metadata/create_procedure_check_metadata.1.adm
M 
asterix-bad/src/test/resources/runtimets/results/procedure/query_procedure/query_procedure.3.adm
M 
asterix-bad/src/test/resources/runtimets/results/procedure/query_procedure/query_procedure.5.adm
M 
asterix-bad/src/test/resources/runtimets/results/procedure/query_procedure_with_parameters/query_procedure_with_parameters.2.adm
45 files changed, 594 insertions(+), 204 deletions(-)


  git pull

Change in asterixdb-bad[master]: Redeploy channels and procedures during recovery

2018-05-16 Thread Jenkins (Code Review)

Jenkins has posted comments on this change.

Change subject: Redeploy channels and procedures during recovery
..


Patch Set 8:

Build Started https://asterix-jenkins.ics.uci.edu/job/asterixbad-gerrit/287/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2641
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
Gerrit-PatchSet: 8
Gerrit-Project: asterixdb-bad
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: No

Change in asterixdb-bad[master]: Redeploy channels and procedures during recovery

2018-05-16 Thread Steven Jacobs (Code Review)

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/2641

to look at the new patch set (#8).

Change subject: Redeploy channels and procedures during recovery
..

Redeploy channels and procedures during recovery

Use the GlobalRecoveryManager extension to redeploy channels/procedures
Restart execution of channels during recovery
Some code cleanup

Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
---
M asterix-bad/pom.xml
M asterix-bad/src/main/java/org/apache/asterix/bad/BADConstants.java
M asterix-bad/src/main/java/org/apache/asterix/bad/BADJobService.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADStatementExecutor.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/CreateChannelStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/CreateProcedureStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/ExecuteProcedureStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/BADMetadataRecordTypes.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/DeployedJobSpecEventListener.java
M asterix-bad/src/main/java/org/apache/asterix/bad/metadata/Procedure.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/ProcedureTupleTranslator.java
A 
asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADGlobalRecoveryManager.java
A 
asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADRecoveryExtension.java
M asterix-bad/src/main/resources/cc.conf
M asterix-bad/src/test/java/org/apache/asterix/bad/test/BADListenerTest.java
M 
asterix-bad/src/test/resources/runtimets/results/procedure/create_procedure_check_metadata/create_procedure_check_metadata.1.adm
16 files changed, 352 insertions(+), 143 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb-bad 
refs/changes/41/2641/8
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2641
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
Gerrit-PatchSet: 8
Gerrit-Project: asterixdb-bad
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Xikui Wang

Change in asterixdb-bad[master]: Redeploy channels and procedures during recovery

2018-05-16 Thread Jenkins (Code Review)

Jenkins has posted comments on this change.

Change subject: Redeploy channels and procedures during recovery
..


Patch Set 7:

Build Started https://asterix-jenkins.ics.uci.edu/job/asterixbad-gerrit/286/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2641
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
Gerrit-PatchSet: 7
Gerrit-Project: asterixdb-bad
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: No

Change in asterixdb-bad[master]: Redeploy channels and procedures during recovery

2018-05-16 Thread Steven Jacobs (Code Review)

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/2641

to look at the new patch set (#7).

Change subject: Redeploy channels and procedures during recovery
..

Redeploy channels and procedures during recovery

Use the GlobalRecoveryManager extension to redeploy channels/procedures
Restart execution of channels during recovery
Some code cleanup

Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
---
M asterix-bad/pom.xml
M asterix-bad/src/main/java/org/apache/asterix/bad/BADConstants.java
M asterix-bad/src/main/java/org/apache/asterix/bad/BADJobService.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADStatementExecutor.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/CreateChannelStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/CreateProcedureStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/ExecuteProcedureStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/BADMetadataRecordTypes.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/DeployedJobSpecEventListener.java
M asterix-bad/src/main/java/org/apache/asterix/bad/metadata/Procedure.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/ProcedureTupleTranslator.java
A 
asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADGlobalRecoveryManager.java
A 
asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADRecoveryExtension.java
M asterix-bad/src/main/resources/cc.conf
M asterix-bad/src/test/java/org/apache/asterix/bad/test/BADListenerTest.java
M 
asterix-bad/src/test/resources/runtimets/results/procedure/create_procedure_check_metadata/create_procedure_check_metadata.1.adm
16 files changed, 352 insertions(+), 143 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb-bad 
refs/changes/41/2641/7
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2641
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
Gerrit-PatchSet: 7
Gerrit-Project: asterixdb-bad
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Xikui Wang

Change in asterixdb-bad[master]: Redeploy channels and procedures during recovery

2018-05-16 Thread Steven Jacobs (Code Review)

Steven Jacobs has posted comments on this change.

Change subject: Redeploy channels and procedures during recovery
..


Patch Set 5:

(11 comments)

https://asterix-gerrit.ics.uci.edu/#/c/2641/5/asterix-bad/src/main/java/org/apache/asterix/bad/BADJobService.java
File asterix-bad/src/main/java/org/apache/asterix/bad/BADJobService.java:

Line 142: ResultReader resultReader = new ResultReader(hdc, jobId, 
new ResultSetId(0));
> is this always reading the 1st result set? (I know there are some issues in
Procedures and Channels only allow one statement (so one result set) so this is 
a simplification for them. Maybe other deployed jobs will have potential issues,


Line 249: if (useNewId) {
> I'm comfused... What's the difference between deploy and redeploy + newId
When the cluster is restarted, we just deploy the jobs as new jobs (the old 
deployed jobs would be lost anyway).

When an index is added, we use the redeploy to replace the existing job spec


https://asterix-gerrit.ics.uci.edu/#/c/2641/5/asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/CreateChannelStatement.java
File 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/CreateChannelStatement.java:

Line 273: resultsTableName = !push ? channelName + 
BADConstants.resultsEnding : "";
> push : "" : constant
Done


https://asterix-gerrit.ics.uci.edu/#/c/2641/5/asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/CreateProcedureStatement.java
File 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/CreateProcedureStatement.java:

PS5, Line 274: LANGUAGE_AQL
> it's AQL?
Done


https://asterix-gerrit.ics.uci.edu/#/c/2641/5/asterix-bad/src/main/java/org/apache/asterix/bad/metadata/DeployedJobSpecEventListener.java
File 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/DeployedJobSpecEventListener.java:

Line 105: public void storeDeployedJobSpecId(DeployedJobSpecId 
deployedJobSpecId) {
> -> setDeployedJobSpecId ?
Done


Line 109: public void storeExecutorService(ScheduledExecutorService ses) {
> same
Done


https://asterix-gerrit.ics.uci.edu/#/c/2641/5/asterix-bad/src/main/java/org/apache/asterix/bad/metadata/Procedure.java
File asterix-bad/src/main/java/org/apache/asterix/bad/metadata/Procedure.java:

Line 49: public Procedure(String dataverseName, String functionName, int 
arity, List params, String type,
> Is it no longer the return type?
The return type was left over from functions, we don't use such a thing in 
procedures. I needed to add a Procedure type, so I figured I should remove the 
unused field while adding the new one.


https://asterix-gerrit.ics.uci.edu/#/c/2641/5/asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADGlobalRecoveryManager.java
File 
asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADGlobalRecoveryManager.java:

PS5, Line 92: tionContext mdTxnCtx = 
MetadataManager.INSTANCE.beginTransaction();
: metadataProvider.setMetadataTxnContext(mdTxnCtx);
: 
: List channels = 
BADLangExtension.getAllChannels(mdTxnCtx);
: List procedures = 
BADLangExtension.getAllProcedures(mdTxnCtx);
: 
: MetadataManager.INSTANCE.commitTransaction(mdTxnCtx);
> why not do this in recover after doRecovery()?
Done


Line 112: for (IActiveEntityEventsListener listener : 
activeEventHandler.getEventListeners()) {
> do you want to check & stop EventListeners' executorService before you remo
Done


Line 125: listener.suspend();
> this is to make sure if there is any dangling deployed job running instance
yes


PS5, Line 161: //Issue: need to store in metadata the information 
for running instances
> I don't understand this issue here... maybe we can talk about this offline.
Sure. We can talk about it more in future, but the main issue is:

1) A repetitive procedure can exist without running (no one has called "execute"
2) At cluster restart, the listener is gone, so the only information we have is 
the metadata
3) For every channel we automatically know that there should be one running 
instance, so we just start it again. We have no such guarantee for a procedure, 
and the metadata doesn't tell us anything about if/how many were running.


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2641
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
Gerrit-PatchSet: 5
Gerrit-Project: asterixdb-bad
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Steven Jacobs 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: Yes

Change in asterixdb-bad[master]: Redeploy channels and procedures during recovery

2018-05-15 Thread Xikui Wang (Code Review)

Xikui Wang has posted comments on this change.

Change subject: Redeploy channels and procedures during recovery
..


Patch Set 5:

(11 comments)

Testcases would be very much appreciated...

https://asterix-gerrit.ics.uci.edu/#/c/2641/5/asterix-bad/src/main/java/org/apache/asterix/bad/BADJobService.java
File asterix-bad/src/main/java/org/apache/asterix/bad/BADJobService.java:

Line 142: ResultReader resultReader = new ResultReader(hdc, jobId, 
new ResultSetId(0));
is this always reading the 1st result set? (I know there are some issues in a 
deployed job with multiple results. is this the one?)


Line 249: if (useNewId) {
I'm comfused... What's the difference between deploy and redeploy + newId


https://asterix-gerrit.ics.uci.edu/#/c/2641/5/asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/CreateChannelStatement.java
File 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/CreateChannelStatement.java:

Line 273: resultsTableName = !push ? channelName + 
BADConstants.resultsEnding : "";
push : "" : constant


https://asterix-gerrit.ics.uci.edu/#/c/2641/5/asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/CreateProcedureStatement.java
File 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/CreateProcedureStatement.java:

PS5, Line 274: LANGUAGE_AQL
it's AQL?


https://asterix-gerrit.ics.uci.edu/#/c/2641/5/asterix-bad/src/main/java/org/apache/asterix/bad/metadata/DeployedJobSpecEventListener.java
File 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/DeployedJobSpecEventListener.java:

Line 105: public void storeDeployedJobSpecId(DeployedJobSpecId 
deployedJobSpecId) {
-> setDeployedJobSpecId ?


Line 109: public void storeExecutorService(ScheduledExecutorService ses) {
same


https://asterix-gerrit.ics.uci.edu/#/c/2641/5/asterix-bad/src/main/java/org/apache/asterix/bad/metadata/Procedure.java
File asterix-bad/src/main/java/org/apache/asterix/bad/metadata/Procedure.java:

Line 49: public Procedure(String dataverseName, String functionName, int 
arity, List params, String type,
Is it no longer the return type?


https://asterix-gerrit.ics.uci.edu/#/c/2641/5/asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADGlobalRecoveryManager.java
File 
asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADGlobalRecoveryManager.java:

PS5, Line 92: tionContext mdTxnCtx = 
MetadataManager.INSTANCE.beginTransaction();
: metadataProvider.setMetadataTxnContext(mdTxnCtx);
: 
: List channels = 
BADLangExtension.getAllChannels(mdTxnCtx);
: List procedures = 
BADLangExtension.getAllProcedures(mdTxnCtx);
: 
: MetadataManager.INSTANCE.commitTransaction(mdTxnCtx);
why not do this in recover after doRecovery()?


Line 112: for (IActiveEntityEventsListener listener : 
activeEventHandler.getEventListeners()) {
do you want to check & stop EventListeners' executorService before you remove 
them?


Line 125: listener.suspend();
this is to make sure if there is any dangling deployed job running instance, 
they have to stop first?


PS5, Line 161: //Issue: need to store in metadata the information 
for running instances
I don't understand this issue here... maybe we can talk about this offline...


-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2641
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
Gerrit-PatchSet: 5
Gerrit-Project: asterixdb-bad
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: Yes

Change in asterixdb-bad[master]: Redeploy channels and procedures during recovery

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/2641

to look at the new patch set (#6).

Change subject: Redeploy channels and procedures during recovery
..

Redeploy channels and procedures during recovery

Use the GlobalRecoveryManager extension to redeploy channels/procedures
Restart execution of channels during recovery
Some code cleanup

Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
---
M asterix-bad/pom.xml
M asterix-bad/src/main/java/org/apache/asterix/bad/BADConstants.java
M asterix-bad/src/main/java/org/apache/asterix/bad/BADJobService.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADStatementExecutor.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/CreateChannelStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/CreateProcedureStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/ExecuteProcedureStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/BADMetadataRecordTypes.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/DeployedJobSpecEventListener.java
M asterix-bad/src/main/java/org/apache/asterix/bad/metadata/Procedure.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/ProcedureTupleTranslator.java
A 
asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADGlobalRecoveryManager.java
A 
asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADRecoveryExtension.java
M asterix-bad/src/main/resources/cc.conf
M asterix-bad/src/test/java/org/apache/asterix/bad/test/BADListenerTest.java
M 
asterix-bad/src/test/resources/runtimets/results/procedure/create_procedure_check_metadata/create_procedure_check_metadata.1.adm
16 files changed, 348 insertions(+), 143 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb-bad 
refs/changes/41/2641/6
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2641
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
Gerrit-PatchSet: 6
Gerrit-Project: asterixdb-bad
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang

Change in asterixdb-bad[master]: Redeploy channels and procedures during recovery

Jenkins has posted comments on this change.

Change subject: Redeploy channels and procedures during recovery
..


Patch Set 6:

Build Started https://asterix-jenkins.ics.uci.edu/job/asterixbad-gerrit/285/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2641
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
Gerrit-PatchSet: 6
Gerrit-Project: asterixdb-bad
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Till Westmann 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: No

Change in asterixdb-bad[master]: Redeploy channels and procedures during recovery

Jenkins has posted comments on this change.

Change subject: Redeploy channels and procedures during recovery
..


Patch Set 5:

Build Started https://asterix-jenkins.ics.uci.edu/job/asterixbad-gerrit/282/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2641
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
Gerrit-PatchSet: 5
Gerrit-Project: asterixdb-bad
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: No

Change in asterixdb-bad[master]: Redeploy channels and procedures during recovery

Jenkins has posted comments on this change.

Change subject: Redeploy channels and procedures during recovery
..


Patch Set 5:

Build Started https://asterix-jenkins.ics.uci.edu/job/asterixbad-gerrit/281/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2641
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
Gerrit-PatchSet: 5
Gerrit-Project: asterixdb-bad
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: No

Change in asterixdb-bad[master]: Redeploy channels and procedures during recovery

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/2641

to look at the new patch set (#5).

Change subject: Redeploy channels and procedures during recovery
..

Redeploy channels and procedures during recovery

Use the GlobalRecoveryManager extension to redeploy channels/procedures
Restart execution of channels during recovery
Some code cleanup

Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
---
M asterix-bad/pom.xml
M asterix-bad/src/main/java/org/apache/asterix/bad/BADConstants.java
M asterix-bad/src/main/java/org/apache/asterix/bad/BADJobService.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADStatementExecutor.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/CreateChannelStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/CreateProcedureStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/ExecuteProcedureStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/BADMetadataRecordTypes.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/DeployedJobSpecEventListener.java
M asterix-bad/src/main/java/org/apache/asterix/bad/metadata/Procedure.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/ProcedureTupleTranslator.java
A 
asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADGlobalRecoveryManager.java
A 
asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADRecoveryExtension.java
M asterix-bad/src/main/resources/cc.conf
M asterix-bad/src/test/java/org/apache/asterix/bad/test/BADListenerTest.java
M 
asterix-bad/src/test/resources/runtimets/results/procedure/create_procedure_check_metadata/create_procedure_check_metadata.1.adm
16 files changed, 348 insertions(+), 143 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb-bad 
refs/changes/41/2641/5
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2641
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
Gerrit-PatchSet: 5
Gerrit-Project: asterixdb-bad
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Xikui Wang

Change in asterixdb-bad[master]: Redeploy channels and procedures during recovery

Jenkins has posted comments on this change.

Change subject: Redeploy channels and procedures during recovery
..


Patch Set 4:

Build Started https://asterix-jenkins.ics.uci.edu/job/asterixbad-gerrit/280/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2641
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb-bad
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: No

Change in asterixdb-bad[master]: Redeploy channels and procedures during recovery

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/2641

to look at the new patch set (#4).

Change subject: Redeploy channels and procedures during recovery
..

Redeploy channels and procedures during recovery

Use the GlobalRecoveryManager extension to redeploy channels/procedures
Restart execution of channels during recovery
Some code cleanup

Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
---
M asterix-bad/pom.xml
M asterix-bad/src/main/java/org/apache/asterix/bad/BADConstants.java
M asterix-bad/src/main/java/org/apache/asterix/bad/BADJobService.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/BADStatementExecutor.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/CreateChannelStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/CreateProcedureStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/lang/statement/ExecuteProcedureStatement.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/BADMetadataRecordTypes.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/DeployedJobSpecEventListener.java
M asterix-bad/src/main/java/org/apache/asterix/bad/metadata/Procedure.java
M 
asterix-bad/src/main/java/org/apache/asterix/bad/metadata/ProcedureTupleTranslator.java
A 
asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADGlobalRecoveryManager.java
A 
asterix-bad/src/main/java/org/apache/asterix/bad/recovery/BADRecoveryExtension.java
M asterix-bad/src/main/resources/cc.conf
M asterix-bad/src/test/java/org/apache/asterix/bad/test/BADListenerTest.java
M 
asterix-bad/src/test/resources/runtimets/results/procedure/create_procedure_check_metadata/create_procedure_check_metadata.1.adm
16 files changed, 348 insertions(+), 143 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb-bad 
refs/changes/41/2641/4
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2641
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
Gerrit-PatchSet: 4
Gerrit-Project: asterixdb-bad
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Xikui Wang

Change in asterixdb-bad[master]: Redeploy channels and procedures during recovery

Jenkins has posted comments on this change.

Change subject: Redeploy channels and procedures during recovery
..


Patch Set 3:

Build Started https://asterix-jenkins.ics.uci.edu/job/asterixbad-gerrit/279/

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2641
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I6897ccf9cddb9ec8d10256e252ee893afe6db145
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb-bad
Gerrit-Branch: master
Gerrit-Owner: Steven Jacobs 
Gerrit-Reviewer: Jenkins 
Gerrit-Reviewer: Xikui Wang 
Gerrit-HasComments: No

Change in asterixdb-bad[master]: Redeploy channels and procedures during recovery