Hi Ümit,

Thanks for sharing such a detailed write-up. It's great to see Apache
CloudStack 4.23 packaged as an HCI appliance for public and private
institutions in Turkey, and thank you for making the ISO freely available
to everyone.

I'll download the iso  and try it out in my homelab over the coming days

Thanks again for the contribution to the ecosystem.

Regards
Kiran Chavala

On Sat, Nov 7, 2026 at 10:39 PM Ümit Eyigün <[email protected]> wrote:

> Hello CloudStack community,
>
> I would like to share an overview of the **CloudStack-based KVM HCI** we
> developed at TRTEK. This is not only a lab experiment: our goal was a
> **practical, widely usable solution in Turkey**, built on **Apache
> CloudStack**, that helps organizations move away from rising VMware
> licensing costs and from hesitation around less familiar hypervisors.
>
> First, **thank you to everyone on the Apache CloudStack team** who has
> contributed. This product would not exist without CloudStack’s solid
> orchestration layer. Our work has been to complete it for Turkish public
> and private institutions—especially around **storage, installation,
> migration, replication, and console** behavior.
>
> ## Why we built this
>
> TRTEK works with roughly **100 public and private institutions in
> Turkey**. What we keep seeing in the field:
>
> - **VMware license and renewal costs** put real pressure on budgets;
> sustainability of licensing is often under discussion.
> - Alternatives such as **Proxmox** come up, but in government and
> compliance-sensitive environments concerns about support, roadmap,
> institutional acceptance, and operational familiarity (what we hear as
> **Proxmox hesitation**) slow or block adoption.
> - What was needed: a familiar cloud orchestrator (**CloudStack**), an open
> hypervisor (**KVM**), **single-image install**, thin storage, migration,
> and cross-site continuity—an **HCI shaped for our customers and ourselves**.
>
> So we built a vertical appliance / HCI on Rocky Linux + **Apache
> CloudStack 4.23 (KVM)**, aimed at **increasing CloudStack adoption in
> Turkey**. Current install image: **trtek-cs-setup59**.
>
> **The ISO is fully public.** Any organization or individual can download
> it free of charge:
>
> https://cloudstack.trtek.tr
>
> This is not a closed “only-for-us” package. We chose **open distribution**
> so the CloudStack ecosystem can be more visible in Turkey.
>
> ## Product intent
>
> - Small–medium **HCI**: no Ceph/OVS requirement; a simple KVM node model.
> - **Thin** storage for space efficiency; reclaim via discard/UNMAP.
> - Migrate existing **VMware / Hyper-V** workloads into CloudStack.
> - Continuous **CS→CS replication + promote** across two sites (DR / warm
> standby).
> - Console, live migrate, shared disk (CLVM), and HA that hold up in
> production.
> - Install and updates managed from **one channel** (public ISO + update
> panel).
>
> In short: a **CloudStack HCI usable in Turkey**—vertical integration from
> disk to console, migration to replication.
>
> ## Architecture (short)
>
> | Layer | What |
> |-------|------|
> | Install | Rocky kickstart + GUI wizard + appliance installer |
> | Hypervisor | KVM / libvirt, cloudbr0 |
> | Orchestration | CloudStack 4.23 management + agent |
> | Storage | LVM thin VG **trtekthin** (+ optional shared **CLVM /
> trtekvg**) |
> | Ops panel | :8091 — disks, migration, repl, updates (TR/EN) |
> | Distribution | Public ISO (trtek-cs-setupNN) + update CDs |
>
> Roles: **management (host/cloud)** and **extra KVM (client)**; clients
> join via a join code.
>
> ## How we use storage
>
> - Limited system area (root + swap); the large remainder is for VMs.
> - VM disks live under **trtekthin** thin pool (`data`) instead of fat LVs
> / ballooning qcow.
> - Thin volume **export** → XFS `/export` (templates, ISOs, helpers); VG
> activation on boot.
> - `issue_discards=1` — guest UNMAP reclaims thin space.
> - Thin orphan scan (visibility; does not auto-delete).
> - Shared disk via CLVM + sanlock; VG lock / LVM wrappers after hard
> power-off.
> - Surviving node can take an **exclusive** activation on the fallen host’s
> volume (with our CLVM patch).
>
> Model: **thin-first HCI**—overcommit, discard, shared disk, and
> single-node local in one language.
>
> ## What we changed on QEMU / libvirt
>
> Not an upstream fork—targeted production fixes:
>
> 1. **Console VNC** — console proxy speaks plain VNC; host VNC TLS off
> (otherwise the console sticks on “Connecting”).
> 2. **iothread poll hook** — I/O polling after guest start.
> 3. **VirtIO CD hook** — Windows migration / driver ISO paths.
> 4. **Sparse copy** — migration/repl/delta; BLKDISCARD when punch is
> unavailable; sparse path for live migrate.
> 5. **Optional trtek-qemu RPM** — special cases (e.g. LSI SAS); not
> required by the install ISO; BIOS/IDE migration uses stock Rocky qemu-kvm.
> 6. Simple KVM node sysctl/modprobe (IOMMU, aio limits, etc.).
>
> ## CloudStack patches (4.23.0.0) and why
>
> | Patch | Why |
> |-------|-----|
> | **Preallocation → falloc** | “Full” zero-fills the disk in qemu and
> hurts thin; value becomes `falloc`, XFS allocates without zero-fill. |
> | **Discard / UNMAP (virtio bus)** | UNMAP was keyed to virtio-blk; root
> bus stays `virtio` so discard never applied → thin reclaim. |
> | **ClvmStorageAdaptor** | After host loss, exclusive `lvchange -aey`
> without useless pool refresh. |
> | **MigrateKVMAsync** | Sparse migrate on live/panel move. |
> | **VirtualMachineMO** | vCenter HTTP 500 on not-yet-attached VMDK broke
> disk attach during migration; flow continues. |
> | **Console proxy multiviewer** | Second console must not drop the first. |
>
> Appliance complements: VMware/Hyper-V migration, CS→CS repl + promote,
> keep/import, two-node HA, 8080→8443 redirect, panel TLS upload proxy,
> public update channel.
>
> ## Migration and replication
>
> - **Migration:** VMware / Hyper-V → thin LV → KVM → CloudStack import
> (VirtIO / IDE as needed).
> - **CS→CS repl:** seed + dirty-bitmap delta; peer status Flowing / Lag /
> In sync.
> - **Promote:** planned or emergency; `start --paused` before import (OS
> never boots), then CloudStack start → correct VNC/XML and the real first
> boot (so console works).
>
> ## Outcome
>
> For institutions in Turkey facing VMware cost and alternative-hypervisor
> hesitation, we shipped a **CloudStack-centered HCI appliance** with thin
> storage, migration, and cross-site replication. The ISO is **public** at **
> https://cloudstack.trtek.tr**.
>
> Again, thank you to the CloudStack community and to every developer who
> contributed. This is **not a CloudStack fork**; it is a vertical product on
> **4.23**—storage and QEMU behavior, targeted patches, and open distribution
> for the Turkish field. We welcome feedback and collaboration.
>
> Best regards,
> Ümit Eyigün
> TRTEK — Systems
> [email protected]
> https://cloudstack.trtek.tr
> https://www.trtekyazilim.com
>

Reply via email to