Hello CloudStack community,

I would like to share an overview of the **CloudStack-based KVM HCI** we 
developed at TRTEK. This is not only a lab experiment: our goal was a 
**practical, widely usable solution in Turkey**, built on **Apache 
CloudStack**, that helps organizations move away from rising VMware licensing 
costs and from hesitation around less familiar hypervisors.

First, **thank you to everyone on the Apache CloudStack team** who has 
contributed. This product would not exist without CloudStack’s solid 
orchestration layer. Our work has been to complete it for Turkish public and 
private institutions—especially around **storage, installation, migration, 
replication, and console** behavior.

## Why we built this

TRTEK works with roughly **100 public and private institutions in Turkey**. 
What we keep seeing in the field:

- **VMware license and renewal costs** put real pressure on budgets; 
sustainability of licensing is often under discussion.
- Alternatives such as **Proxmox** come up, but in government and 
compliance-sensitive environments concerns about support, roadmap, 
institutional acceptance, and operational familiarity (what we hear as 
**Proxmox hesitation**) slow or block adoption.
- What was needed: a familiar cloud orchestrator (**CloudStack**), an open 
hypervisor (**KVM**), **single-image install**, thin storage, migration, and 
cross-site continuity—an **HCI shaped for our customers and ourselves**.

So we built a vertical appliance / HCI on Rocky Linux + **Apache CloudStack 
4.23 (KVM)**, aimed at **increasing CloudStack adoption in Turkey**. Current 
install image: **trtek-cs-setup59**.

**The ISO is fully public.** Any organization or individual can download it 
free of charge:

https://cloudstack.trtek.tr

This is not a closed “only-for-us” package. We chose **open distribution** so 
the CloudStack ecosystem can be more visible in Turkey.

## Product intent

- Small–medium **HCI**: no Ceph/OVS requirement; a simple KVM node model.
- **Thin** storage for space efficiency; reclaim via discard/UNMAP.
- Migrate existing **VMware / Hyper-V** workloads into CloudStack.
- Continuous **CS→CS replication + promote** across two sites (DR / warm 
standby).
- Console, live migrate, shared disk (CLVM), and HA that hold up in production.
- Install and updates managed from **one channel** (public ISO + update panel).

In short: a **CloudStack HCI usable in Turkey**—vertical integration from disk 
to console, migration to replication.

## Architecture (short)

| Layer | What |
|-------|------|
| Install | Rocky kickstart + GUI wizard + appliance installer |
| Hypervisor | KVM / libvirt, cloudbr0 |
| Orchestration | CloudStack 4.23 management + agent |
| Storage | LVM thin VG **trtekthin** (+ optional shared **CLVM / trtekvg**) |
| Ops panel | :8091 — disks, migration, repl, updates (TR/EN) |
| Distribution | Public ISO (trtek-cs-setupNN) + update CDs |

Roles: **management (host/cloud)** and **extra KVM (client)**; clients join via 
a join code.

## How we use storage

- Limited system area (root + swap); the large remainder is for VMs.
- VM disks live under **trtekthin** thin pool (`data`) instead of fat LVs / 
ballooning qcow.
- Thin volume **export** → XFS `/export` (templates, ISOs, helpers); VG 
activation on boot.
- `issue_discards=1` — guest UNMAP reclaims thin space.
- Thin orphan scan (visibility; does not auto-delete).
- Shared disk via CLVM + sanlock; VG lock / LVM wrappers after hard power-off.
- Surviving node can take an **exclusive** activation on the fallen host’s 
volume (with our CLVM patch).

Model: **thin-first HCI**—overcommit, discard, shared disk, and single-node 
local in one language.

## What we changed on QEMU / libvirt

Not an upstream fork—targeted production fixes:

1. **Console VNC** — console proxy speaks plain VNC; host VNC TLS off 
(otherwise the console sticks on “Connecting”).
2. **iothread poll hook** — I/O polling after guest start.
3. **VirtIO CD hook** — Windows migration / driver ISO paths.
4. **Sparse copy** — migration/repl/delta; BLKDISCARD when punch is 
unavailable; sparse path for live migrate.
5. **Optional trtek-qemu RPM** — special cases (e.g. LSI SAS); not required by 
the install ISO; BIOS/IDE migration uses stock Rocky qemu-kvm.
6. Simple KVM node sysctl/modprobe (IOMMU, aio limits, etc.).

## CloudStack patches (4.23.0.0) and why

| Patch | Why |
|-------|-----|
| **Preallocation → falloc** | “Full” zero-fills the disk in qemu and hurts 
thin; value becomes `falloc`, XFS allocates without zero-fill. |
| **Discard / UNMAP (virtio bus)** | UNMAP was keyed to virtio-blk; root bus 
stays `virtio` so discard never applied → thin reclaim. |
| **ClvmStorageAdaptor** | After host loss, exclusive `lvchange -aey` without 
useless pool refresh. |
| **MigrateKVMAsync** | Sparse migrate on live/panel move. |
| **VirtualMachineMO** | vCenter HTTP 500 on not-yet-attached VMDK broke disk 
attach during migration; flow continues. |
| **Console proxy multiviewer** | Second console must not drop the first. |

Appliance complements: VMware/Hyper-V migration, CS→CS repl + promote, 
keep/import, two-node HA, 8080→8443 redirect, panel TLS upload proxy, public 
update channel.

## Migration and replication

- **Migration:** VMware / Hyper-V → thin LV → KVM → CloudStack import (VirtIO / 
IDE as needed).
- **CS→CS repl:** seed + dirty-bitmap delta; peer status Flowing / Lag / In 
sync.
- **Promote:** planned or emergency; `start --paused` before import (OS never 
boots), then CloudStack start → correct VNC/XML and the real first boot (so 
console works).

## Outcome

For institutions in Turkey facing VMware cost and alternative-hypervisor 
hesitation, we shipped a **CloudStack-centered HCI appliance** with thin 
storage, migration, and cross-site replication. The ISO is **public** at 
**https://cloudstack.trtek.tr**.

Again, thank you to the CloudStack community and to every developer who 
contributed. This is **not a CloudStack fork**; it is a vertical product on 
**4.23**—storage and QEMU behavior, targeted patches, and open distribution for 
the Turkish field. We welcome feedback and collaboration.

Best regards,  
Ümit Eyigün  
TRTEK — Systems  
[email protected]  
https://cloudstack.trtek.tr  
https://www.trtekyazilim.com

Reply via email to