Change in asterixdb[master]: [NO ISSUE][STO] Adapt Structure Structure To Rebalance

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: [NO ISSUE][STO] Adapt Structure Structure To Rebalance
..


Patch Set 3:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/7240/ (5/11)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2181
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0f968b9f493bf5aa2d49f503afe21f0d438bb7f0
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE][STO] Adapt Structure Structure To Rebalance

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: [NO ISSUE][STO] Adapt Structure Structure To Rebalance
..


Patch Set 3:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-cancellation-test/1700/ 
(6/11)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2181
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0f968b9f493bf5aa2d49f503afe21f0d438bb7f0
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE][STO] Adapt Structure Structure To Rebalance

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: [NO ISSUE][STO] Adapt Structure Structure To Rebalance
..


Patch Set 3:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-asterix-app/2564/ (9/11)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2181
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0f968b9f493bf5aa2d49f503afe21f0d438bb7f0
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE][STO] Adapt Structure Structure To Rebalance

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: [NO ISSUE][STO] Adapt Structure Structure To Rebalance
..


Patch Set 3:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-rebalance-cancellation/889/
 (2/11)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2181
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0f968b9f493bf5aa2d49f503afe21f0d438bb7f0
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE][STO] Adapt Structure Structure To Rebalance

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: [NO ISSUE][STO] Adapt Structure Structure To Rebalance
..


Patch Set 3:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-sonar/5740/ (11/11)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2181
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0f968b9f493bf5aa2d49f503afe21f0d438bb7f0
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE][STO] Adapt Structure Structure To Rebalance

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: [NO ISSUE][STO] Adapt Structure Structure To Rebalance
..


Patch Set 3:

Build Started https://asterix-jenkins.ics.uci.edu/job/hyracks-gerrit/1688/ 
(4/11)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2181
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0f968b9f493bf5aa2d49f503afe21f0d438bb7f0
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE][STO] Adapt Structure Structure To Rebalance

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: [NO ISSUE][STO] Adapt Structure Structure To Rebalance
..


Patch Set 3:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-verify-storage/2316/ (3/11)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2181
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0f968b9f493bf5aa2d49f503afe21f0d438bb7f0
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE][STO] Adapt Structure Structure To Rebalance

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: [NO ISSUE][STO] Adapt Structure Structure To Rebalance
..


Patch Set 3:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-verify-no-installer-app/2353/
 (10/11)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2181
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0f968b9f493bf5aa2d49f503afe21f0d438bb7f0
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE][STO] Adapt Structure Structure To Rebalance

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: [NO ISSUE][STO] Adapt Structure Structure To Rebalance
..


Patch Set 3:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-verify-asterix-app/2374/ 
(7/11)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2181
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0f968b9f493bf5aa2d49f503afe21f0d438bb7f0
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE][STO] Adapt Structure Structure To Rebalance

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: [NO ISSUE][STO] Adapt Structure Structure To Rebalance
..


Patch Set 3:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-source-assemblies/2019/ 
(8/11)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2181
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0f968b9f493bf5aa2d49f503afe21f0d438bb7f0
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE][STO] Adapt Structure Structure To Rebalance

[Murtadha Hubail (Code Review)]

Hello Jenkins,

I'd like you to reexamine a change.  Please visit

https://asterix-gerrit.ics.uci.edu/2181

to look at the new patch set (#3).

Change subject: [NO ISSUE][STO] Adapt Structure Structure To Rebalance
..

[NO ISSUE][STO] Adapt Structure Structure To Rebalance

- user model changes: no
- storage format changes: no
- interface changes: yes
-- Added IResource#setPath to use for the resource
   storage migration.

Details:
- Unify storage structure to support dataset rebalance:
  Old format:
  ./storage/partition_#/dataverse/datasetName_idx_indexName
  New format:
  ./storage/partition_#/dataverse/datasetName/rebalanaceNum/indexName
- Adapt recovery and replication to new storage structure.
- Add old structure -> new structure NC migration task.
- Add CompatibilityUtil to ensure NC can be upgraded during
  NC startup.
- Centralize the logic for parsing file path to its components in
  ResourceReference/DatasetResourceReference.
- Add storage structure migration test case.
- Add test case for recovery after rebalance.

Change-Id: I0f968b9f493bf5aa2d49f503afe21f0d438bb7f0
---
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/RecoveryManager.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/TransactionSubsystem.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/task/MigrateStorageResourcesTask.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplication.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/CompatibilityUtil.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/api/http/servlet/ConnectorApiServletTest.java
A 
asterixdb/asterix-app/src/test/java/org/apache/asterix/common/TestDataUtil.java
A 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/storage/MigrateStorageResourcesTaskTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/txn/RecoveryManagerTest.java
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/all_datasets/all_datasets.11.adm
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/all_datasets/all_datasets.12.adm
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/all_datasets/all_datasets.5.adm
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/all_datasets/all_datasets.6.adm
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataset/single_dataset.5.adm
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataset/single_dataset.9.adm
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataset_with_index/single_dataset_with_index.10.adm
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataset_with_index/single_dataset_with_index.5.adm
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataverse/single_dataverse.11.adm
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataverse/single_dataverse.12.adm
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataverse/single_dataverse.5.adm
M 
asterixdb/asterix-app/src/test/resources/runtimets/results/rebalance/single_dataverse/single_dataverse.6.adm
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/dataflow/DatasetLocalResource.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/replication/IReplicaResourcesManager.java
A 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/storage/DatasetResourceReference.java
D 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/storage/IndexFileProperties.java
A 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/storage/IndexPathElements.java
A 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/storage/ResourceReference.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/utils/StorageConstants.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/utils/StoragePathUtil.java
M asterixdb/asterix-replication/pom.xml
M 
asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/management/ReplicationChannel.java
M 
asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/management/ReplicationManager.java
M 
asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/storage/LSMComponentProperties.java
M 
asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/storage/LSMIndexFileProperties.java
M 
asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/storage/ReplicaResourcesManager.java
M 
asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/resource/PersistentLocalResourceRepository.java
M 

Change in asterixdb[master]: [NO ISSUE][STO] Adapt Structure Structure To Rebalance

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: [NO ISSUE][STO] Adapt Structure Structure To Rebalance
..


Patch Set 3:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-asterix-app-sql-execution/1709/
 (1/11)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2181
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0f968b9f493bf5aa2d49f503afe21f0d438bb7f0
Gerrit-PatchSet: 3
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE][STO] Adapt Structure Structure To Rebalance

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: [NO ISSUE][STO] Adapt Structure Structure To Rebalance
..


Patch Set 1:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-source-assemblies/2017/ 
(10/11)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2181
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0f968b9f493bf5aa2d49f503afe21f0d438bb7f0
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE][STO] Adapt Structure Structure To Rebalance

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: [NO ISSUE][STO] Adapt Structure Structure To Rebalance
..


Patch Set 1:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-notopic/7238/ (5/11)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2181
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0f968b9f493bf5aa2d49f503afe21f0d438bb7f0
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE][STO] Adapt Structure Structure To Rebalance

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: [NO ISSUE][STO] Adapt Structure Structure To Rebalance
..


Patch Set 1:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-verify-storage/2314/ (2/11)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2181
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0f968b9f493bf5aa2d49f503afe21f0d438bb7f0
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE][STO] Adapt Structure Structure To Rebalance

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: [NO ISSUE][STO] Adapt Structure Structure To Rebalance
..


Patch Set 1:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-rebalance-cancellation/887/
 (8/11)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2181
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0f968b9f493bf5aa2d49f503afe21f0d438bb7f0
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE][STO] Adapt Structure Structure To Rebalance

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: [NO ISSUE][STO] Adapt Structure Structure To Rebalance
..


Patch Set 1:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-cancellation-test/1698/ 
(7/11)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2181
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0f968b9f493bf5aa2d49f503afe21f0d438bb7f0
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE][STO] Adapt Structure Structure To Rebalance

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: [NO ISSUE][STO] Adapt Structure Structure To Rebalance
..


Patch Set 1:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-verify-asterix-app/2372/ 
(9/11)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2181
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0f968b9f493bf5aa2d49f503afe21f0d438bb7f0
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE][STO] Adapt Structure Structure To Rebalance

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: [NO ISSUE][STO] Adapt Structure Structure To Rebalance
..


Patch Set 1:

Build Started https://asterix-jenkins.ics.uci.edu/job/hyracks-gerrit/1686/ 
(3/11)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2181
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0f968b9f493bf5aa2d49f503afe21f0d438bb7f0
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE][STO] Adapt Structure Structure To Rebalance

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: [NO ISSUE][STO] Adapt Structure Structure To Rebalance
..


Patch Set 1:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-sonar/5738/ (11/11)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2181
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0f968b9f493bf5aa2d49f503afe21f0d438bb7f0
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE][STO] Adapt Structure Structure To Rebalance

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: [NO ISSUE][STO] Adapt Structure Structure To Rebalance
..


Patch Set 1:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-asterix-app/2562/ (4/11)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2181
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0f968b9f493bf5aa2d49f503afe21f0d438bb7f0
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE][STO] Adapt Structure Structure To Rebalance

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: [NO ISSUE][STO] Adapt Structure Structure To Rebalance
..


Patch Set 1:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-verify-no-installer-app/2351/
 (6/11)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2181
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0f968b9f493bf5aa2d49f503afe21f0d438bb7f0
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No

Change in asterixdb[master]: [NO ISSUE][STO] Adapt Structure Structure To Rebalance

[Murtadha Hubail (Code Review)]

Murtadha Hubail has uploaded a new change for review.

  https://asterix-gerrit.ics.uci.edu/2181

Change subject: [NO ISSUE][STO] Adapt Structure Structure To Rebalance
..

[NO ISSUE][STO] Adapt Structure Structure To Rebalance

- user model changes: no
- storage format changes: no
- interface changes: yes
-- Added IResource#setPath to use for the resource
   storage migration.

Details:
- Unify storage structure to support dataset rebalance:
  Old format:
  ./storage/partition_#/dataverse/datasetName_idx_indexName
  New format:
  ./storage/partition_#/dataverse/datasetName/rebalanaceNum/indexName
- Adapt recovery and replication to new storage structure.
- Add old structure -> new structure NC migration task.
- Add CompatibilityUtil to ensure NC can be upgraded during
  NC startup.
- Centralize the logic for parsing file path to its components in
  ResourceReference/DatasetResourceReference.
- Add storage structure migration test case.
- Add test case for recovery after rebalance.

Change-Id: I0f968b9f493bf5aa2d49f503afe21f0d438bb7f0
---
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/RecoveryManager.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/TransactionSubsystem.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/task/MigrateStorageResourcesTask.java
M 
asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplication.java
A 
asterixdb/asterix-app/src/main/java/org/apache/asterix/utils/CompatibilityUtil.java
A 
asterixdb/asterix-app/src/test/java/org/apache/asterix/common/TestDataUtil.java
A 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/storage/MigrateStorageResourcesTaskTest.java
M 
asterixdb/asterix-app/src/test/java/org/apache/asterix/test/txn/RecoveryManagerTest.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/dataflow/DatasetLocalResource.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/replication/IReplicaResourcesManager.java
A 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/storage/DatasetResourceReference.java
D 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/storage/IndexFileProperties.java
A 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/storage/IndexPathElements.java
A 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/storage/ResourceReference.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/utils/StorageConstants.java
M 
asterixdb/asterix-common/src/main/java/org/apache/asterix/common/utils/StoragePathUtil.java
M asterixdb/asterix-replication/pom.xml
M 
asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/management/ReplicationChannel.java
M 
asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/management/ReplicationManager.java
M 
asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/storage/LSMComponentProperties.java
M 
asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/storage/LSMIndexFileProperties.java
M 
asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/storage/ReplicaResourcesManager.java
M 
asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/resource/PersistentLocalResourceRepository.java
M 
asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/service/recovery/ReplicationCheckpointManager.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/dataflow/BTreeResource.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/dataflow/LsmResource.java
M 
hyracks-fullstack/hyracks/hyracks-storage-am-rtree/src/main/java/org/apache/hyracks/storage/am/rtree/dataflow/RTreeResource.java
M 
hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/IResource.java
28 files changed, 882 insertions(+), 422 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb 
refs/changes/81/2181/1

diff --git 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/RecoveryManager.java
 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/RecoveryManager.java
index 19966fe..e29e3fe 100644
--- 
a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/RecoveryManager.java
+++ 
b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/RecoveryManager.java
@@ -443,7 +443,7 @@
 return minFirstLSN;
 }
 
-private long getRemoteMinFirstLSN() {
+private long getRemoteMinFirstLSN() throws HyracksDataException {
 IReplicaResourcesManager remoteResourcesManager =
 
txnSubsystem.getAsterixAppRuntimeContextProvider().getAppContext().getReplicaResourcesManager();
 return 

Change in asterixdb[master]: [NO ISSUE][STO] Adapt Structure Structure To Rebalance

[Jenkins (Code Review)]

Jenkins has posted comments on this change.

Change subject: [NO ISSUE][STO] Adapt Structure Structure To Rebalance
..


Patch Set 1:

Build Started 
https://asterix-jenkins.ics.uci.edu/job/asterix-gerrit-asterix-app-sql-execution/1707/
 (1/11)

-- 
To view, visit https://asterix-gerrit.ics.uci.edu/2181
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I0f968b9f493bf5aa2d49f503afe21f0d438bb7f0
Gerrit-PatchSet: 1
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail 
Gerrit-Reviewer: Jenkins 
Gerrit-HasComments: No