On 2025-05-23 20:17, Randy Bush wrote:
There is no option to replicate a full ZFS pool to another.
not exactly what i want to do. my bad in saying "full replication."
what i meant was all vms are replicated. on other nodes.
i was thinking that each node could have one pool for primary vm images
and a second to receive replication from other nodes.
Ah okay. No, the way it works is that you have ZFS pools in both nodes
with the same name. Then when the replication is configured for a guest,
its disk images are replicated to the other node to the pool with the
same name.
So, you have a current pool with one raidz2 VDEV made up of 4x 2TB
disks.
yup
Because if you have another set of 4x 2TB disks, you can just expand
the pool with another raidz2 VDEV, without expanding the current VDEV
you have.
yup. what are the performance implications?
As usually with ZFS, the change will only affect newly written data. It
will be spread over both VDEVs, with likely a bias to the newer, much
emptier VDEV. So if you are already happy with the current performance,
you should see similar or better performance, depending if one or both
VDEVs are used to read/write data.
If you add another VDEV, the pool could have the following layout:
pool
raidz2-0
disk0
disk1
disk2
disk3
raidz2-1
disk4
disk5
disk6
disk7
yup
If you want to create a new pool, then things will be a bit more
complicated, as you would need to create a new storage config for it
as well, Move-Disk all the disks over to it. If you have a cluster and
use the VM replication feature, that new pool must be present on the
other nodes as well and you will have to remove the replication jobs
before you move the disks to the new pool and then re-create them once
all VM disks are on the new pool.
we would keep the nodes all symmetric, so that would not be an issue.
and it's just a few hours of ops pain to de-repl and re-repl. but what
i do not see is how to tell `/etc/pve/storage.cfg` that pool0 is for
images and pool1 is for incoming replication. maybe i am just trying to
do something too weird.
Yeah, see my reply at the beginning. I think you have a more complicated
view of the replication than it actually is.
If you are okay with the current performance, I would just add the
second VDEV to the pool with
`zpool add {pool} raidz2 /dev/disk/by-id/nvme-… /dev/disk/by-id/nvme-…`
Before you do it on a production system, you can test the procedure in a
(virtual) test machine to make sure you get the CLI command correct.
By extending the pool, you don't need to change anything in the storage
config or replication settings.
randy
_______________________________________________
pve-user mailing list
pve-user@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user
