Re: Packaging ZFS
On 10/08/23 22:36, Thibaut wrote: Le 10 août 2023 à 22:25, Philip Prindeville a écrit : On Aug 10, 2023, at 11:49 AM, Torbjörn Jansson wrote: On 2023-08-06 21:39, Philip Prindeville wrote: I don't know... I have a Xeon D-1548 based 1U Supermicro server with a 4TB NVMe stick that would make a decent file server/NAS... On Aug 6, 2023, at 11:46 AM, Paul D wrote: Pretty sure not. I'm receptive to ZFS and have used it in a few projects. Openwrt tends to focus on (devices with) smaller flash drives. Other FS better suited to such env. No ZFS is in available software packages today, in any case. On 2023-08-06 00:53, Philip Prindeville wrote: Has anyone tried to package ZFS (more correctly, OpenZFS) for OpenWRT? Is there any interest in doing so? https://github.com/openzfs/zfs you could always run openwrt as a vm under a hypervisor, for example proxmox. then you can keep openwrt without any extra packages like zfs and create extra vms as needed, proxmox already supports zfs if im not mistaken. if your lucky with the iommu groups you might even be able to pass thru one or more physical network interfaces to the openwrt vm directly. I can't assume that the underlying hardware supports virtualization or does so in a meaningful way. Some of the platforms I'm looking at are resource lean. I threw out the Xeon-D as an example as my prototyping hardware, but I'm not going to assume that everyone has comparable hardware. ZFS is anything *but* resource lean, though. It's actually more lean than most think. Lots of outdated information from the old Solaris/Illumos or BSD's version of ZFS is still preached as gospel in some influential circles (*cough*TrueNAS forums*cough*). For example the most common rule of thumb aka "1GB for 1TB of storage" is outdated since OpenZFS on Linux 0.8 (released in 2019). Also, while by default ZFS is occupying half the available ram as cache, that's just a default setting that can (and in most cases should) be changed depending on your needs. For example this person ran a 2TB ZFS with the RAM cache limited to 64MB (the minimum possible) on a tiny RiscV board with 512MB RAM. And in his benchmark it still ran slightly better than ext4 for non-encrypted volumes. https://andreas.welcomes-you.com/zfs-risc-v-512mb-lichee-rv/ (archived version on Internet Archive) http://web.archive.org/web/20230604205911/https://andreas.welcomes-you.com/zfs-risc-v-512mb-lichee-rv/ I have also ran some resource-contrained NAS PCs where I had limited ZFS to use only 512MB of ram and it is still running fine for 8TB arrays. -Alberto ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: Packaging ZFS
On 2023-08-10 22:36, Thibaut wrote: Le 10 août 2023 à 22:25, Philip Prindeville a écrit : On Aug 10, 2023, at 11:49 AM, Torbjörn Jansson wrote: On 2023-08-06 21:39, Philip Prindeville wrote: I don't know... I have a Xeon D-1548 based 1U Supermicro server with a 4TB NVMe stick that would make a decent file server/NAS... On Aug 6, 2023, at 11:46 AM, Paul D wrote: Pretty sure not. I'm receptive to ZFS and have used it in a few projects. Openwrt tends to focus on (devices with) smaller flash drives. Other FS better suited to such env. No ZFS is in available software packages today, in any case. On 2023-08-06 00:53, Philip Prindeville wrote: Has anyone tried to package ZFS (more correctly, OpenZFS) for OpenWRT? Is there any interest in doing so? https://github.com/openzfs/zfs you could always run openwrt as a vm under a hypervisor, for example proxmox. then you can keep openwrt without any extra packages like zfs and create extra vms as needed, proxmox already supports zfs if im not mistaken. if your lucky with the iommu groups you might even be able to pass thru one or more physical network interfaces to the openwrt vm directly. I can't assume that the underlying hardware supports virtualization or does so in a meaningful way. Some of the platforms I'm looking at are resource lean. I threw out the Xeon-D as an example as my prototyping hardware, but I'm not going to assume that everyone has comparable hardware. ZFS is anything *but* resource lean, though. in my experience, most regular computers (not counting home broadband "routers" with tiny flash and cpu) do support virtualization. even my old gen 6 intel cpu works fine and likely some older gen cpus work too. you don't really need to forward a physical network card (which would require some better hardware), a set of virtual nics provided by the hypervisor will work most of the time, yes the thru-put may be lower but if the bandwidth requirement is not gbit speeds then it probably works fine anyway. BUT, there is reason for having your router on a separate box. having your main router as a vm with other vms on the same box can create some interesting startup dependencies. but most regular computers, even old ones, have plenty of resources left for other things than just openwrt. ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: Packaging ZFS
> Le 10 août 2023 à 22:25, Philip Prindeville > a écrit : > > > >> On Aug 10, 2023, at 11:49 AM, Torbjörn Jansson wrote: >> >> On 2023-08-06 21:39, Philip Prindeville wrote: >>> I don't know... I have a Xeon D-1548 based 1U Supermicro server with a 4TB >>> NVMe stick that would make a decent file server/NAS... On Aug 6, 2023, at 11:46 AM, Paul D wrote: Pretty sure not. I'm receptive to ZFS and have used it in a few projects. Openwrt tends to focus on (devices with) smaller flash drives. Other FS better suited to such env. No ZFS is in available software packages today, in any case. On 2023-08-06 00:53, Philip Prindeville wrote: > Has anyone tried to package ZFS (more correctly, OpenZFS) for OpenWRT? > Is there any interest in doing so? > > https://github.com/openzfs/zfs > > > >> >> you could always run openwrt as a vm under a hypervisor, for example proxmox. >> then you can keep openwrt without any extra packages like zfs and create >> extra vms as needed, proxmox already supports zfs if im not mistaken. >> >> if your lucky with the iommu groups you might even be able to pass thru one >> or more physical network interfaces to the openwrt vm directly. > > > > I can't assume that the underlying hardware supports virtualization or does > so in a meaningful way. Some of the platforms I'm looking at are resource > lean. I threw out the Xeon-D as an example as my prototyping hardware, but > I'm not going to assume that everyone has comparable hardware. ZFS is anything *but* resource lean, though. ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: Packaging ZFS
> On Aug 10, 2023, at 11:49 AM, Torbjörn Jansson wrote: > > On 2023-08-06 21:39, Philip Prindeville wrote: >> I don't know... I have a Xeon D-1548 based 1U Supermicro server with a 4TB >> NVMe stick that would make a decent file server/NAS... >>> On Aug 6, 2023, at 11:46 AM, Paul D wrote: >>> >>> Pretty sure not. I'm receptive to ZFS and have used it in a few projects. >>> Openwrt tends to focus on (devices with) smaller flash drives. Other FS >>> better suited to such env. >>> >>> No ZFS is in available software packages today, in any case. >>> >>> >>> On 2023-08-06 00:53, Philip Prindeville wrote: Has anyone tried to package ZFS (more correctly, OpenZFS) for OpenWRT? Is there any interest in doing so? https://github.com/openzfs/zfs > > you could always run openwrt as a vm under a hypervisor, for example proxmox. > then you can keep openwrt without any extra packages like zfs and create > extra vms as needed, proxmox already supports zfs if im not mistaken. > > if your lucky with the iommu groups you might even be able to pass thru one > or more physical network interfaces to the openwrt vm directly. I can't assume that the underlying hardware supports virtualization or does so in a meaningful way. Some of the platforms I'm looking at are resource lean. I threw out the Xeon-D as an example as my prototyping hardware, but I'm not going to assume that everyone has comparable hardware. -Philip ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: Packaging ZFS
On 2023-08-06 21:39, Philip Prindeville wrote: I don't know... I have a Xeon D-1548 based 1U Supermicro server with a 4TB NVMe stick that would make a decent file server/NAS... On Aug 6, 2023, at 11:46 AM, Paul D wrote: Pretty sure not. I'm receptive to ZFS and have used it in a few projects. Openwrt tends to focus on (devices with) smaller flash drives. Other FS better suited to such env. No ZFS is in available software packages today, in any case. On 2023-08-06 00:53, Philip Prindeville wrote: Has anyone tried to package ZFS (more correctly, OpenZFS) for OpenWRT? Is there any interest in doing so? https://github.com/openzfs/zfs you could always run openwrt as a vm under a hypervisor, for example proxmox. then you can keep openwrt without any extra packages like zfs and create extra vms as needed, proxmox already supports zfs if im not mistaken. if your lucky with the iommu groups you might even be able to pass thru one or more physical network interfaces to the openwrt vm directly. ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: Packaging ZFS
> On Aug 8, 2023, at 3:14 PM, Luiz Angelo Daros de Luca > wrote: > >> Thanks, Alberto. I'm wondering how much work making it cross-build packages >> is going to be... >> >> Digging into it now... >> > > You should take a look at ksmbd package. It does compile an out-of-tree > module. > > Maintaining a package like that might be challenging for a stable > version. The kernel in the stable branch might receive updates during > its life but they are only published when a dot version (yy.mm.x+1) is > actually released. > However, packages are rebuilt as they are changed using the current > kernel in the stable branch. So, to build your module with a kernel > that will actually be published. you can only update a package with a > kernel module when you know the kernel in the stable branch is the one > that will be used by the next dot release. This "window of > opportunity" happens a little before or after the dot release is > built. Once the kernel is updated again, that window is closed. And > that updated kernel module will only be compatible with that specific > dot version, and not with any other dot version before or after it. > > Regards, > > Luiz There are a lot of modules that are built out-of-tree (like DPDK). How do they manage it? ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: Packaging ZFS
> Thanks, Alberto. I'm wondering how much work making it cross-build packages > is going to be... > > Digging into it now... > You should take a look at ksmbd package. It does compile an out-of-tree module. Maintaining a package like that might be challenging for a stable version. The kernel in the stable branch might receive updates during its life but they are only published when a dot version (yy.mm.x+1) is actually released. However, packages are rebuilt as they are changed using the current kernel in the stable branch. So, to build your module with a kernel that will actually be published. you can only update a package with a kernel module when you know the kernel in the stable branch is the one that will be used by the next dot release. This "window of opportunity" happens a little before or after the dot release is built. Once the kernel is updated again, that window is closed. And that updated kernel module will only be compatible with that specific dot version, and not with any other dot version before or after it. Regards, Luiz ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: Packaging ZFS
Thanks, Alberto. I'm wondering how much work making it cross-build packages is going to be... Digging into it now... > On Aug 7, 2023, at 3:23 AM, Alberto Bursi wrote: > > ZFS would be useful for any device with a few GB of RAM that has data drives > (a NAS for example). I've used ZFS extensively on x86 systems with other > Linux distros (Debian/Proxmox and OpenSUSE). > I think ZFS support is a good thing. > > Booting from ZFS is probably not necessary for OpenWrt but zfs snapshots are > used by some BSD distros (TrueNAS Core) and maybe Ubuntu as a way to version > the rootfs and revert to an older OS version in case of problems with updates > (similar to what the Turris Omnia and OpenSUSE does with btrfs afaik). > > Afaik the ZFS project does support the two "major" archs aka x86_64 and > ARM64, and maybe Power. People have been using ZFS on Raspberry Pis and on > some Rockchip boards (in Debian/Ubuntu/Armbian/RaspberryOS) for years at this > point. > > -Alberto > > On 06/08/23 21:39, Philip Prindeville wrote: >> I don't know... I have a Xeon D-1548 based 1U Supermicro server with a 4TB >> NVMe stick that would make a decent file server/NAS... >>> On Aug 6, 2023, at 11:46 AM, Paul D wrote: >>> >>> Pretty sure not. I'm receptive to ZFS and have used it in a few projects. >>> Openwrt tends to focus on (devices with) smaller flash drives. Other FS >>> better suited to such env. >>> >>> No ZFS is in available software packages today, in any case. >>> >>> >>> On 2023-08-06 00:53, Philip Prindeville wrote: Has anyone tried to package ZFS (more correctly, OpenZFS) for OpenWRT? Is there any interest in doing so? https://github.com/openzfs/zfs ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: Packaging ZFS
ZFS would be useful for any device with a few GB of RAM that has data drives (a NAS for example). I've used ZFS extensively on x86 systems with other Linux distros (Debian/Proxmox and OpenSUSE). I think ZFS support is a good thing. Booting from ZFS is probably not necessary for OpenWrt but zfs snapshots are used by some BSD distros (TrueNAS Core) and maybe Ubuntu as a way to version the rootfs and revert to an older OS version in case of problems with updates (similar to what the Turris Omnia and OpenSUSE does with btrfs afaik). Afaik the ZFS project does support the two "major" archs aka x86_64 and ARM64, and maybe Power. People have been using ZFS on Raspberry Pis and on some Rockchip boards (in Debian/Ubuntu/Armbian/RaspberryOS) for years at this point. -Alberto On 06/08/23 21:39, Philip Prindeville wrote: I don't know... I have a Xeon D-1548 based 1U Supermicro server with a 4TB NVMe stick that would make a decent file server/NAS... On Aug 6, 2023, at 11:46 AM, Paul D wrote: Pretty sure not. I'm receptive to ZFS and have used it in a few projects. Openwrt tends to focus on (devices with) smaller flash drives. Other FS better suited to such env. No ZFS is in available software packages today, in any case. On 2023-08-06 00:53, Philip Prindeville wrote: Has anyone tried to package ZFS (more correctly, OpenZFS) for OpenWRT? Is there any interest in doing so? https://github.com/openzfs/zfs ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: Packaging ZFS
I don't know... I have a Xeon D-1548 based 1U Supermicro server with a 4TB NVMe stick that would make a decent file server/NAS... > On Aug 6, 2023, at 11:46 AM, Paul D wrote: > > Pretty sure not. I'm receptive to ZFS and have used it in a few projects. > Openwrt tends to focus on (devices with) smaller flash drives. Other FS > better suited to such env. > > No ZFS is in available software packages today, in any case. > > > On 2023-08-06 00:53, Philip Prindeville wrote: > > Has anyone tried to package ZFS (more correctly, OpenZFS) for OpenWRT? Is > > there any interest in doing so? > > > > https://github.com/openzfs/zfs > > > > > > > > ___ > openwrt-devel mailing list > openwrt-devel@lists.openwrt.org > https://lists.openwrt.org/mailman/listinfo/openwrt-devel ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Re: Packaging ZFS
Pretty sure not. I'm receptive to ZFS and have used it in a few projects. Openwrt tends to focus on (devices with) smaller flash drives. Other FS better suited to such env. No ZFS is in available software packages today, in any case. On 2023-08-06 00:53, Philip Prindeville wrote: > Has anyone tried to package ZFS (more correctly, OpenZFS) for OpenWRT? Is there any interest in doing so? > > https://github.com/openzfs/zfs > > > ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
Packaging ZFS
Has anyone tried to package ZFS (more correctly, OpenZFS) for OpenWRT? Is there any interest in doing so? https://github.com/openzfs/zfs ___ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
