Migration between zones is NOT possible in any shape or form, so this is a route you should, IMO, abandon (you can always export VMs in this way or another, but this is not feasible in production)
I understand you have 2 DCs and you want VMs to, eventually, become alive in 2nd DC, if the plane crashes on 1st DC? (well, your data is there, unless CEPH is stretched/distributed across 2 DCs and could survive the whole DC1 going down) If you are insisting on that HA level - then you could do it in 2 ways, that cross my mind right now. (CEPH as distributed storage, zone wide, some nodes DC1, some DC2 - make sure your CEPH setup survives whole DC going down (this requires that CRUSH maps correctly configured etc) (1) DC1 = Pod1 (1/2/3) and DC2 = Pod2 (or Pod 4/5/6 etc) - i.e. multiple Pods per DC - they all will be using zone-wide Ceph storage - your VMs are on your storage, that is the crucial part to not lose data. -- you can't really migrate VMs between Pods, only within cluster (and in some cases between clusters in the same Pod, staring from 4.16) -- this is OK if you have not-low-enough-latency between DC1 and DC2 (but then CEPH will also suffer from that higher latency) (2) A very untipical, not recommends, but technically possible setup - DC1+DC2 = one large DC = 1 POD = 1 cluster (or more clusters if needed) - still using CEPH as before -- requires ultra-low latency between DC1 and DC2 - and if plane crashes on DC1 (taking this example, as I've been to some Zurich DCs next to the airport...) - you can still start VM on hosts in DC2 in case it was a single cluster. In case you had multiple clusters - then it get's more complicated (minor DB hacks) etc. In both cases you still have to sort out Secondary Storage NFS HA.... In general, you can't achieve what you want that easy nor you should be stretching the possibilities (that I just explained, as I would, probably, never use them in production) I guess I didn't help - but there you go. Andrija On Tue, 20 Jul 2021 at 16:28, Vladimir Dombrovski < vladimir.dombrov...@bso.co> wrote: > Hello, > > We're trying to draw a multisite architecture where any VM could be > relocated to the secondary site whenever the primary site fails > (primary/backup for disaster recovery purposes). We don't require live > migration, and we are okay with shutting down machines in order to relocate > them. > > We are using Cloudstack 4.15 on Ubuntu Focal. In our current setup, each > datacenter has a Cloudstack management node, as well as a few hypervisors > running KVM and a Cloudstack agent. We're using Ceph as our primary > storage, and NFS as our secondary storage on each site. > > To ensure metadata resiliency, we've replicated the MySQL database across > both sites, much like described following this guide: > > > https://docs.cloudstack.apache.org/projects/cloudstack-installation/en/4.11/choosing_deployment_architecture.html#multi-site-deployment > > We tried setting up multiple zones, one for each datacenter, each one > having its own primary storage, but we are faced with the issue where we > are not able to migrate VMs across zones (only Pod/Cluster/Host level is > available via the GUI and the Cloudmonkey CLI). > > Are we using the right level of abstraction for our case? If so, how can > we migrate a VM (compute + storage) from one zone to another? If not, what > is the right level to use that allows us to use two separate primary > storage endpoints and ensures that only the primary site gets used for > compute resource allocation in normal conditions? > > Also, we would like to know whether there is some documentation already > touching on the subject of best practices when performing these "more > advanced" deployments. > > Kind regards, > > Vladimir DOMBROVSKI > -- Andrija Panić
