Separate account service will solve the data sync problem. But that would be a 
very big change and can be probably considered for a later release.

Accounts/Users/Domains(Resources) need to be in sync across Regions. 
Each resource has a source region (The region in which the resource is created).
All modify/delete API calls will be propagated to source region of the resource.

It is the responsibility of the source region to ensure a change is propagated 
to all regions.
All this works well when all the regions are up. I'm looking at options to 
ensure data is propagated reliably when a regions comes back up after going 
down.

Take account creation for example:

1. Rollback on failure: Create account in source region. Source region makes 
create account API calls to all regions. If create fails is any region, abort 
the operation and roll back account creation in regions where it is created 
successfully. Account creation will fail even if one region is down. Rollback 
of account creation is also not straight forward. 

2. Retry using region_sync table: Create account in source region. Source 
region makes create API calls to all regions. If create fails is any region, 
source region makes an entry in region_sync table with following info 
(resoucre_id, api_call, failed_region_id). Periodically retry the API call for 
failed regions and clear the entry once its successful.

3. Use 3rd party NOSQL db: Create account in source region. Create an entry is 
region_sync table (resoucre_id, api_call, source_region_id). Additionally use a 
eventually consistent distributed nosql db (like Apache Cassandra) in each 
region. Insert entries in region_sync table to NOSQL db and this data will be 
available in all the regions. Each region will run unprocessed APIs locally and 
mark them as processed. This approach will offload the data sync problem to a 
proven solution like Cassandra.

Please let me know your thoughts on the above mentioned approaches.

________________________________________
From: Anthony Xu [xuefei...@citrix.com]
Sent: Saturday, January 19, 2013 5:30 AM
To: CloudStack DeveloperList
Subject: RE: Questions related to Regions Feature

It is not easy to sync/maintain the account information in several regions.
Is it possible to have a separate account service, cloudstack management 
servers in multiple region can consult ?

Anthony

> -----Original Message-----
> From: Chiradeep Vittal [mailto:chiradeep.vit...@citrix.com]
> Sent: Friday, January 18, 2013 3:41 PM
> To: CloudStack DeveloperList
> Subject: Re: Questions related to Regions Feature
>
> I think that replication of data to each region (even if it is only for
> account data) is fraught with complexity and should be left to an
> external
> service.
> Can we simplify by assuming that accounts are synced "somehow"?
>

Reply via email to