Re: Building ZFS disk images
On 9/28/21 9:15 AM, Rodney W. Grimes wrote: On Tue, Sep 28, 2021 at 9:48 AM Rodney W. Grimes wrote: ^^^ re-guid the pool on first boot. Isnt the proper place to solve this lack of Unique UUID creation in the tool(s) that are creating the zfs pool in the first place. Fixing it "post boot" seems to be a far to late hack and doesnt fix any of the situations where one might import these pools between creation and first boot. No, because you might create a VM image once, then instantiate it dozens or thousands of times. The firstboot solution is great because it lets you reuse the same image file. I would continue to argue that the place to fix this is in the "instantiate tool". ESXI vmfs deals with this all the time when you clone a disk. And again the "fix at boot" does not deal with the problem in that if I "instatiate" 10 copies of a zpool for VM's and then try to mount 2 of them at once on the host this problem rares it head. Fix the problem as close to point of creation as possible for minimal issues in all operations for everyone. Define a special magic "change to something else" ID that will always do the right thing exactly once.
Re: Building ZFS disk images
‐‐snip--- *puts on cloud-init contributor hat* > > No, because you might create a VM image once, then instantiate it > > dozens or thousands of times. The firstboot solution is great because > > it lets you reuse the same image file. > > I would continue to argue that the place to fix this is in the > "instantiate tool". ESXI vmfs deals with this all the time > when you clone a disk. And again the "fix at boot" does not > deal with the problem in that if I "instatiate" 10 copies of > a zpool for VM's and then try to mount 2 of them at once on > the host this problem rares it head. Fix the problem as close > to point of creation as possible for minimal issues in all > operations for everyone. a lot of folks use cloud-init for provisioning different Unices onto different virtualisation (cloud) platforms. We could fix it there. We already extend paritions, filesystems and ZFS pools in cloud-init. now, again, one could argue that's the wrong place to do any of that, and we should just be using firstboot. But the problem seems to be that a lot folks out there got into the habit of creating and publishing (FreeBSD) images, seem to have forgotten or never knew about firstboot, and don't to set it. Mina Galić Web: https://igalic.co/ PkgBase: https://alpha.pkgbase.live/
Re: Building ZFS disk images
On Tue, Sep 28, 2021 at 10:15 AM Rodney W. Grimes wrote: > > > On Tue, Sep 28, 2021 at 9:48 AM Rodney W. Grimes > > wrote: > > > > > > > On Mon, Sep 27, 2021 at 1:54 PM Mark Johnston wrote: > > > > > > > > > > On Thu, Aug 05, 2021 at 10:54:19AM -0500, Alan Somers wrote: > > > > > > There's this: > > > > > > https://openzfs.github.io/openzfs-docs/man/8/zpool-reguid.8.html . > > > > > > I > > > > > > haven't used it myself. > > > > > > > > > > Would it be useful to have an rc.d script that can run this, probably > > > > > just on the root pool? It could be configured to run only upon the > > > > > first boot, like growfs already does. > > > > > > > > Absolutely! > > > > > > Eww! :-) > > > > > > > > > > > > > > On Thu, Aug 5, 2021, 9:29 AM David Chisnall > > > > > > wrote: > > > > > > > > > > > > > On 05/08/2021 13:53, Alan Somers wrote: > > > > > > > > I don't know of any way to do it using the official release > > > > > > > > scripts > > > > > > > > either. One problem is that every ZFS pool and file system is > > > > > > > > supposed > > > > > > > > to have a unique GUID. So any kind of ZFS release builder > > > > > > > > would need to > > > ^^^ > > > > > > > > re-guid the pool on first boot. > > > > > > Isnt the proper place to solve this lack of Unique UUID creation > > > in the tool(s) that are creating the zfs pool in the first place. > > > > > > Fixing it "post boot" seems to be a far to late hack and doesnt > > > fix any of the situations where one might import these pools > > > between creation and first boot. > > > > No, because you might create a VM image once, then instantiate it > > dozens or thousands of times. The firstboot solution is great because > > it lets you reuse the same image file. > > I would continue to argue that the place to fix this is in the > "instantiate tool". ESXI vmfs deals with this all the time > when you clone a disk. And again the "fix at boot" does not > deal with the problem in that if I "instatiate" 10 copies of > a zpool for VM's and then try to mount 2 of them at once on > the host this problem rares it head. Fix the problem as close > to point of creation as possible for minimal issues in all > operations for everyone. But that requires ESXI, or whatever VM system you're using, to know about ZFS and GPT, and to know to look for a zpool on the 3rd partition, right? That seems like a lot to ask, especially since the logic would have to be duplicated for ESXI, vm-bhyve, OpenNebula, etc etc. > > > > > > > > > > > > > > > > > > > > Is there a tool / command to do this? I've hit this problem in > > > > > > > the > > > > > > > past: I have multiple FreeBSD VMs that are all created from the > > > > > > > same > > > > > > > template and if one dies I can't import its zpool into another > > > > > > > because > > > > > > > they have the same UUID. > > > > > > > > > > > > > > It doesn't matter for modern deployments where the VM is > > > > > > > stateless and > > > > > > > reimaged periodically but it's annoying for classic deployments > > > > > > > where I > > > > > > > have things I care about on the VM. > > > > > > > > > > > > > > David > > > > > > > > > > > > > > -- > > > Rod Grimes > > > rgri...@freebsd.org > > > > -- > Rod Grimes rgri...@freebsd.org
Re: Building ZFS disk images
> On Tue, Sep 28, 2021 at 9:48 AM Rodney W. Grimes > wrote: > > > > > On Mon, Sep 27, 2021 at 1:54 PM Mark Johnston wrote: > > > > > > > > On Thu, Aug 05, 2021 at 10:54:19AM -0500, Alan Somers wrote: > > > > > There's this: > > > > > https://openzfs.github.io/openzfs-docs/man/8/zpool-reguid.8.html . I > > > > > haven't used it myself. > > > > > > > > Would it be useful to have an rc.d script that can run this, probably > > > > just on the root pool? It could be configured to run only upon the > > > > first boot, like growfs already does. > > > > > > Absolutely! > > > > Eww! :-) > > > > > > > > > > > On Thu, Aug 5, 2021, 9:29 AM David Chisnall > > > > > wrote: > > > > > > > > > > > On 05/08/2021 13:53, Alan Somers wrote: > > > > > > > I don't know of any way to do it using the official release > > > > > > > scripts > > > > > > > either. One problem is that every ZFS pool and file system is > > > > > > > supposed > > > > > > > to have a unique GUID. So any kind of ZFS release builder would > > > > > > > need to > > ^^^ > > > > > > > re-guid the pool on first boot. > > > > Isnt the proper place to solve this lack of Unique UUID creation > > in the tool(s) that are creating the zfs pool in the first place. > > > > Fixing it "post boot" seems to be a far to late hack and doesnt > > fix any of the situations where one might import these pools > > between creation and first boot. > > No, because you might create a VM image once, then instantiate it > dozens or thousands of times. The firstboot solution is great because > it lets you reuse the same image file. I would continue to argue that the place to fix this is in the "instantiate tool". ESXI vmfs deals with this all the time when you clone a disk. And again the "fix at boot" does not deal with the problem in that if I "instatiate" 10 copies of a zpool for VM's and then try to mount 2 of them at once on the host this problem rares it head. Fix the problem as close to point of creation as possible for minimal issues in all operations for everyone. > > > > > > > > > > > > > > > Is there a tool / command to do this? I've hit this problem in the > > > > > > past: I have multiple FreeBSD VMs that are all created from the same > > > > > > template and if one dies I can't import its zpool into another > > > > > > because > > > > > > they have the same UUID. > > > > > > > > > > > > It doesn't matter for modern deployments where the VM is stateless > > > > > > and > > > > > > reimaged periodically but it's annoying for classic deployments > > > > > > where I > > > > > > have things I care about on the VM. > > > > > > > > > > > > David > > > > > > > > > > -- > > Rod Grimes > > rgri...@freebsd.org > -- Rod Grimes rgri...@freebsd.org
Re: Building ZFS disk images
On Tue, Sep 28, 2021 at 9:48 AM Rodney W. Grimes wrote: > > > On Mon, Sep 27, 2021 at 1:54 PM Mark Johnston wrote: > > > > > > On Thu, Aug 05, 2021 at 10:54:19AM -0500, Alan Somers wrote: > > > > There's this: > > > > https://openzfs.github.io/openzfs-docs/man/8/zpool-reguid.8.html . I > > > > haven't used it myself. > > > > > > Would it be useful to have an rc.d script that can run this, probably > > > just on the root pool? It could be configured to run only upon the > > > first boot, like growfs already does. > > > > Absolutely! > > Eww! :-) > > > > > > > > On Thu, Aug 5, 2021, 9:29 AM David Chisnall > > > > wrote: > > > > > > > > > On 05/08/2021 13:53, Alan Somers wrote: > > > > > > I don't know of any way to do it using the official release scripts > > > > > > either. One problem is that every ZFS pool and file system is > > > > > > supposed > > > > > > to have a unique GUID. So any kind of ZFS release builder would > > > > > > need to > ^^^ > > > > > > re-guid the pool on first boot. > > Isnt the proper place to solve this lack of Unique UUID creation > in the tool(s) that are creating the zfs pool in the first place. > > Fixing it "post boot" seems to be a far to late hack and doesnt > fix any of the situations where one might import these pools > between creation and first boot. No, because you might create a VM image once, then instantiate it dozens or thousands of times. The firstboot solution is great because it lets you reuse the same image file. > > > > > > > > > > > Is there a tool / command to do this? I've hit this problem in the > > > > > past: I have multiple FreeBSD VMs that are all created from the same > > > > > template and if one dies I can't import its zpool into another because > > > > > they have the same UUID. > > > > > > > > > > It doesn't matter for modern deployments where the VM is stateless and > > > > > reimaged periodically but it's annoying for classic deployments where > > > > > I > > > > > have things I care about on the VM. > > > > > > > > > > David > > > > > > -- > Rod Grimes rgri...@freebsd.org
Re: Building ZFS disk images
> On Mon, Sep 27, 2021 at 1:54 PM Mark Johnston wrote: > > > > On Thu, Aug 05, 2021 at 10:54:19AM -0500, Alan Somers wrote: > > > There's this: > > > https://openzfs.github.io/openzfs-docs/man/8/zpool-reguid.8.html . I > > > haven't used it myself. > > > > Would it be useful to have an rc.d script that can run this, probably > > just on the root pool? It could be configured to run only upon the > > first boot, like growfs already does. > > Absolutely! Eww! :-) > > > > > On Thu, Aug 5, 2021, 9:29 AM David Chisnall wrote: > > > > > > > On 05/08/2021 13:53, Alan Somers wrote: > > > > > I don't know of any way to do it using the official release scripts > > > > > either. One problem is that every ZFS pool and file system is supposed > > > > > to have a unique GUID. So any kind of ZFS release builder would need > > > > > to ^^^ > > > > > re-guid the pool on first boot. Isnt the proper place to solve this lack of Unique UUID creation in the tool(s) that are creating the zfs pool in the first place. Fixing it "post boot" seems to be a far to late hack and doesnt fix any of the situations where one might import these pools between creation and first boot. > > > > > > > > Is there a tool / command to do this? I've hit this problem in the > > > > past: I have multiple FreeBSD VMs that are all created from the same > > > > template and if one dies I can't import its zpool into another because > > > > they have the same UUID. > > > > > > > > It doesn't matter for modern deployments where the VM is stateless and > > > > reimaged periodically but it's annoying for classic deployments where I > > > > have things I care about on the VM. > > > > > > > > David > > -- Rod Grimes rgri...@freebsd.org
Re: Building ZFS disk images
On Mon, Sep 27, 2021 at 1:54 PM Mark Johnston wrote: > > On Thu, Aug 05, 2021 at 10:54:19AM -0500, Alan Somers wrote: > > There's this: > > https://openzfs.github.io/openzfs-docs/man/8/zpool-reguid.8.html . I > > haven't used it myself. > > Would it be useful to have an rc.d script that can run this, probably > just on the root pool? It could be configured to run only upon the > first boot, like growfs already does. Absolutely! > > > On Thu, Aug 5, 2021, 9:29 AM David Chisnall wrote: > > > > > On 05/08/2021 13:53, Alan Somers wrote: > > > > I don't know of any way to do it using the official release scripts > > > > either. One problem is that every ZFS pool and file system is supposed > > > > to have a unique GUID. So any kind of ZFS release builder would need to > > > > re-guid the pool on first boot. > > > > > > Is there a tool / command to do this? I've hit this problem in the > > > past: I have multiple FreeBSD VMs that are all created from the same > > > template and if one dies I can't import its zpool into another because > > > they have the same UUID. > > > > > > It doesn't matter for modern deployments where the VM is stateless and > > > reimaged periodically but it's annoying for classic deployments where I > > > have things I care about on the VM. > > > > > > David
Re: Building ZFS disk images
On Thu, Aug 05, 2021 at 10:54:19AM -0500, Alan Somers wrote: > There's this: > https://openzfs.github.io/openzfs-docs/man/8/zpool-reguid.8.html . I > haven't used it myself. Would it be useful to have an rc.d script that can run this, probably just on the root pool? It could be configured to run only upon the first boot, like growfs already does. > On Thu, Aug 5, 2021, 9:29 AM David Chisnall wrote: > > > On 05/08/2021 13:53, Alan Somers wrote: > > > I don't know of any way to do it using the official release scripts > > > either. One problem is that every ZFS pool and file system is supposed > > > to have a unique GUID. So any kind of ZFS release builder would need to > > > re-guid the pool on first boot. > > > > Is there a tool / command to do this? I've hit this problem in the > > past: I have multiple FreeBSD VMs that are all created from the same > > template and if one dies I can't import its zpool into another because > > they have the same UUID. > > > > It doesn't matter for modern deployments where the VM is stateless and > > reimaged periodically but it's annoying for classic deployments where I > > have things I care about on the VM. > > > > David
Re: Building ZFS disk images
On 05/08/2021 15:06, David Chisnall wrote: Would poudriere work for you? man poudriere-image Wow, there's a lot of stuff I didn't know poudriere could do! It looks as if it can produce a GPT partition table with all of the bootable bits, or it can produce a ZFS disk image. I guess it wouldn't be too difficult to teach it to do both? FYI: I have raised a PR[1] that allows me to create a ZFS disk image. This allows me to create a ZFS root image that I can boot as a Gen1 or Gen2 Hyper-V VM. I have not yet tried it in Azure, but it should work. David [1] https://github.com/freebsd/poudriere/pull/921
Re: Building ZFS disk images
There's this: https://openzfs.github.io/openzfs-docs/man/8/zpool-reguid.8.html . I haven't used it myself. On Thu, Aug 5, 2021, 9:29 AM David Chisnall wrote: > On 05/08/2021 13:53, Alan Somers wrote: > > I don't know of any way to do it using the official release scripts > > either. One problem is that every ZFS pool and file system is supposed > > to have a unique GUID. So any kind of ZFS release builder would need to > > re-guid the pool on first boot. > > Is there a tool / command to do this? I've hit this problem in the > past: I have multiple FreeBSD VMs that are all created from the same > template and if one dies I can't import its zpool into another because > they have the same UUID. > > It doesn't matter for modern deployments where the VM is stateless and > reimaged periodically but it's annoying for classic deployments where I > have things I care about on the VM. > > David
Re: Building ZFS disk images
On 05/08/2021 13:53, Alan Somers wrote: I don't know of any way to do it using the official release scripts either. One problem is that every ZFS pool and file system is supposed to have a unique GUID. So any kind of ZFS release builder would need to re-guid the pool on first boot. Is there a tool / command to do this? I've hit this problem in the past: I have multiple FreeBSD VMs that are all created from the same template and if one dies I can't import its zpool into another because they have the same UUID. It doesn't matter for modern deployments where the VM is stateless and reimaged periodically but it's annoying for classic deployments where I have things I care about on the VM. David
Re: Building ZFS disk images
On 05/08/2021 14:01, Juraj Lutter wrote: On 5 Aug 2021, at 14:53, Alan Somers wrote: I don't know of any way to do it using the official release scripts either. One problem is that every ZFS pool and file system is supposed to have a unique GUID. So any kind of ZFS release builder would need to re-guid the pool on first boot. On Thu, Aug 5, 2021, 6:41 AM David Chisnall wrote: Hi, Does anyone know how to build ZFS disk images from any existing tooling? I haven't used UFS for over a decade now and the official cloud images are all UFS, so I end up doing an install from the CD ISO into Hyper-V locally and then exporting the VHD, but that can't be the most efficient way of getting a FreeBSD VHD with ZFS. I haven't been able to find any documentation and reading the release scripts they seem to hard-code UFS. Would poudriere work for you? man poudriere-image Wow, there's a lot of stuff I didn't know poudriere could do! It looks as if it can produce a GPT partition table with all of the bootable bits, or it can produce a ZFS disk image. I guess it wouldn't be too difficult to teach it to do both? David
Re: Building ZFS disk images
I don't know of any way to do it using the official release scripts either. One problem is that every ZFS pool and file system is supposed to have a unique GUID. So any kind of ZFS release builder would need to re-guid the pool on first boot. On Thu, Aug 5, 2021, 6:41 AM David Chisnall wrote: > Hi, > > Does anyone know how to build ZFS disk images from any existing tooling? > > I haven't used UFS for over a decade now and the official cloud images > are all UFS, so I end up doing an install from the CD ISO into Hyper-V > locally and then exporting the VHD, but that can't be the most efficient > way of getting a FreeBSD VHD with ZFS. > > I haven't been able to find any documentation and reading the release > scripts they seem to hard-code UFS. > > David > >
Building ZFS disk images
Hi, Does anyone know how to build ZFS disk images from any existing tooling? I haven't used UFS for over a decade now and the official cloud images are all UFS, so I end up doing an install from the CD ISO into Hyper-V locally and then exporting the VHD, but that can't be the most efficient way of getting a FreeBSD VHD with ZFS. I haven't been able to find any documentation and reading the release scripts they seem to hard-code UFS. David
