Re: security/clamav: /var/run on TMPFS renders the port broken by design
In message <20220829082514.63926...@thor.intern.walstatt.dynvpn.de>, FreeBSD Us er writes: > Am Sun, 28 Aug 2022 06:11:20 -0700 > Cy Schubert schrieb: > > > In message <20220828130107.1a76d54a.gre...@freebsd.org>, Michael Gmelin > > writes: > > > > > > > > > > > > On Sun, 28 Aug 2022 03:21:24 -0700 > > > Cy Schubert wrote: > > > > > > > In message <16b4-76a1-4e46-b7c3-60492d379...@freebsd.org>, > > > > Michael Gmelin w > > > > rites: > > > > > > > > > > > > > > > > > > > > > On 28. Aug 2022, at 10:42, free...@oldach.net wrote: > > > > > >=20 > > > > > > =EF=BB=BFCy Schubert wrote on Sat, 27 Aug 2022 17:26:38 +0200 > > > > > > (CEST): > > > > > >> As stated before in this thread, replacing /var/run with tmpfs > > > > > >> is not a supported configuration. > > > > > >=20 > > > > > > Not supported? What is the purpose of /etc/rc.d/var then? That > > > > > > creates a t= > > > > > mpfs backed /var, populates it through mtree, and makes a proper > > > > > /var/run av= ailable. > > > > > >=20 > > > > > > However it doesn't (yet) create /var/run/clamav of course. > > > > > >=20 > > > > > > It would be fairly easy to extend /etc/rc.d/var by a logic that > > > > > > walks thro= > > > > > ugh /usr/local/etc/mtree/* and runs mtree on each of the files > > > > > found as need= ed. All that the security/clamav port would need to > > > > > do then is to drop an ap= propriate small mtree file as > > > > > /usr/local/etc/mtree/clamav. =46rom a port's p= erspective that is > > > > > the same logic as dropping service scripts as /usr/local/= > > > > > etc/rc.d/clamav-*. > > > > > > > > > > =46rom a user's perspective, it would be preferable to have this > > > > > happen at s= ervice start though, as (unlike in the setup > > > > > described) reboots don't happen= that frequently, but files in > > > > > /var/run might get deleted manually. Maybe so= me rc framework > > > > > based solution would make sense, e.g., a variable `mtree_fil= es`, > > > > > which, if set, is applied in the default start_precmd. Besides > > > > > being mo= re resilient, this would also have the advantage that all > > > > > required file syst= ems should be available at that point and the > > > > > separation between system and p = orts would be more clear. Another > > > > > advantage would be that directories are on= ly created for services > > > > > that are actually enabled/started. > > > > > > > > Unfortunately this requires all ports to include an mtree file. > > > > Relying on port maintainers who are human to ensure that these files > > > > are created and updated when ports are created and maintained will > > > > result in more human error. I've learned over my long career to rely > > > > more on automation than human beings. Automation [should] never fail > > > > and when it does it does temporarily until the bug is found and > > > > fixed. Human beings inconsistently fail. > > > > > > > > If it were an auto-discovery script that created an mtree file as > > > > part of the packaging process, it would be another matter. But this > > > > optional solution path should be discussed on ports@, not here. > > > > > > > > > > > > > > I don't have much skin in the game, but I created a little proof of > > > concept to allow further discussion (which is not ports-specific, as it > > > works for all service scripts): > > > > > > https://reviews.freebsd.org/D36385 > > > > I've been toying with the idea for a few months but was never bothered to > > create a review or even a script for that matter. > > > > > > > > This basically allows both system admins and port maintainers to > > > create mtree files in /usr/local/etc/mtree (or /etc/mtree, as it's > > > always relative to the service script called) which are automatically > > > applied on service start. It's non-intrusive and doesn't require any > > > sweeping changes to existing ports/services. > > > > Understood that this is a manual process. > > > > > > > > In this specific case, the requester could create > > > /usr/local/etc/mtree/clamav-clamd with the required content (or > > > persuade the port maintainer to include that file). > > > > > > You could of course add some construct to the ports framework that > > > picks up certain directories from the package list automatically and > > > places them into an mtree file as part of the build or installation > > > process. But that would be an additional feature on top of this change. > > > > Someone could. Personally, I think that's a lot of work compared to simply > > saving the state of /var/run at shutdown and restoring it at boot. I can't > > speak for the ports management though. > > > > > > > > This is meant to inspire more discussions, I'm not trying to force > > > anything in. ;) > > > > Agreed. > > > > I cobbled something up yesterday that saves the directory tree state of > > /var/run prior to shutdown (or manually) and restores it at boot. > > > >
Re: security/clamav: /var/run on TMPFS renders the port broken by design
On Mon, 29 Aug 2022 10:31:28 +0200
Franco Fichtner wrote:
> Hi,
>
> > On 29. Aug 2022, at 8:24 AM, FreeBSD User wrote:
> >
> > Checking today NanoBSD based projects, i.e. XigmaNAS, which also let /var
> > reside on a memory
> > disk and the way NanoBSD handles /var, contradicts some claims that is is
> > 'unsupported' to put
> > /var on a volatile memory infrastructure.
>
> I fully agree with the situation that at least NanoBSD has always been a valid
> use case for this and don't see the need to "recycle" contents in /var/run and
> let alone assume software that state inside /var/run is not volatile.
>
> Milage varies for other /var related subdirectories of course, but people
> using
> a /var MFS type setup have managed the situation for many years anyway with
> all
> of its ups and downs.
>
> The situation is a bit sloppy on the ClamAV end and has been for a couple of
> releases assuming this and that and failing on tmpfs nodes, not bootstrapping
> required things in the first place, but let's just assume that is the case for
> other software as well now or in the future.
Uncertain about current default, but at least, at some point,
varmfs="AUTO" on /etc/defaults/rc.conf forcibly create and mount
mfs on /var, overriding existing contents, unless varmfs="NO"
exists on /etc/rc.conf[.local].
If the behaviour is unchanged until now, no port SHALL assume
non-volatile /var, and use usually-non-volatile {LOCALBASE}/var instead.
At least, security/rkhunter uses there.
For use-case writes to non-volatile fs is prohibited and volatile
fs'es are mandated to be cleared daily, the admin should create
script(s) to
1. stop all jobs writing to (allowed) volatile fs,
2. clear volatile fs'es,
3. then restart stopped jobs
on {LOCALBASE}/etc/periodic/daily.
> > what (fatal?) implications does it have to create some folders by port's
> > rc-script in /var/run
> > or whatever folder is needed to be on a volatile memory device for FreeBSD
> > base system's mtree
> > infrastructure?
>
> So the obvious "separation of concerns" solution isn't something that was
> being
> discussed here seeing that this is a ports/packages related issue:
>
> The fix in all cases is to upgrade/reinstall the package in question, so
> the knowledge of these required directories is buried inside the +POST_INSTALL
> script but you cannot easily re-run these scripts since there is no command
> option for it. Obviously this beats having a copy of the package lying around
> just to reinstall it on a reboot...
>
> In the past you were able to extract them from "pkg info --raw $packagename",
> and run them in the shell but that workaround is no longer feasible for LUA
> scripting was added since pkg uses internal definitions in the ports tree
> provided scripts.
>
> Whether or not someone will have to script something to rerun the
> +POST_INSTALL
> on reboot shouldn't stop from adding a pkg script run option to enable the
> former.
>
> Solutions not involving the actual ports scripts seem to be over-engineering
> when trying to record "unknown state" for a "reproducible outcome". ;)
>
>
> Cheers,
> Franco
>
--
Tomoaki AOKI
Re: security/clamav: /var/run on TMPFS renders the port broken by design
Hi, > On 29. Aug 2022, at 8:24 AM, FreeBSD User wrote: > > Checking today NanoBSD based projects, i.e. XigmaNAS, which also let /var > reside on a memory > disk and the way NanoBSD handles /var, contradicts some claims that is is > 'unsupported' to put > /var on a volatile memory infrastructure. I fully agree with the situation that at least NanoBSD has always been a valid use case for this and don't see the need to "recycle" contents in /var/run and let alone assume software that state inside /var/run is not volatile. Milage varies for other /var related subdirectories of course, but people using a /var MFS type setup have managed the situation for many years anyway with all of its ups and downs. The situation is a bit sloppy on the ClamAV end and has been for a couple of releases assuming this and that and failing on tmpfs nodes, not bootstrapping required things in the first place, but let's just assume that is the case for other software as well now or in the future. > what (fatal?) implications does it have to create some folders by port's > rc-script in /var/run > or whatever folder is needed to be on a volatile memory device for FreeBSD > base system's mtree > infrastructure? So the obvious "separation of concerns" solution isn't something that was being discussed here seeing that this is a ports/packages related issue: The fix in all cases is to upgrade/reinstall the package in question, so the knowledge of these required directories is buried inside the +POST_INSTALL script but you cannot easily re-run these scripts since there is no command option for it. Obviously this beats having a copy of the package lying around just to reinstall it on a reboot... In the past you were able to extract them from "pkg info --raw $packagename", and run them in the shell but that workaround is no longer feasible for LUA scripting was added since pkg uses internal definitions in the ports tree provided scripts. Whether or not someone will have to script something to rerun the +POST_INSTALL on reboot shouldn't stop from adding a pkg script run option to enable the former. Solutions not involving the actual ports scripts seem to be over-engineering when trying to record "unknown state" for a "reproducible outcome". ;) Cheers, Franco
Re: security/clamav: /var/run on TMPFS renders the port broken by design
Am Sun, 28 Aug 2022 06:11:20 -0700 Cy Schubert schrieb: > In message <20220828130107.1a76d54a.gre...@freebsd.org>, Michael Gmelin > writes: > > > > > > > > On Sun, 28 Aug 2022 03:21:24 -0700 > > Cy Schubert wrote: > > > > > In message <16b4-76a1-4e46-b7c3-60492d379...@freebsd.org>, > > > Michael Gmelin w > > > rites: > > > > > > > > > > > > > > > > > On 28. Aug 2022, at 10:42, free...@oldach.net wrote: > > > > >=20 > > > > > =EF=BB=BFCy Schubert wrote on Sat, 27 Aug 2022 17:26:38 +0200 > > > > > (CEST): > > > > >> As stated before in this thread, replacing /var/run with tmpfs > > > > >> is not a supported configuration. > > > > >=20 > > > > > Not supported? What is the purpose of /etc/rc.d/var then? That > > > > > creates a t= > > > > mpfs backed /var, populates it through mtree, and makes a proper > > > > /var/run av= ailable. > > > > >=20 > > > > > However it doesn't (yet) create /var/run/clamav of course. > > > > >=20 > > > > > It would be fairly easy to extend /etc/rc.d/var by a logic that > > > > > walks thro= > > > > ugh /usr/local/etc/mtree/* and runs mtree on each of the files > > > > found as need= ed. All that the security/clamav port would need to > > > > do then is to drop an ap= propriate small mtree file as > > > > /usr/local/etc/mtree/clamav. =46rom a port's p= erspective that is > > > > the same logic as dropping service scripts as /usr/local/= > > > > etc/rc.d/clamav-*. > > > > > > > > =46rom a user's perspective, it would be preferable to have this > > > > happen at s= ervice start though, as (unlike in the setup > > > > described) reboots don't happen= that frequently, but files in > > > > /var/run might get deleted manually. Maybe so= me rc framework > > > > based solution would make sense, e.g., a variable `mtree_fil= es`, > > > > which, if set, is applied in the default start_precmd. Besides > > > > being mo= re resilient, this would also have the advantage that all > > > > required file syst= ems should be available at that point and the > > > > separation between system and p = orts would be more clear. Another > > > > advantage would be that directories are on= ly created for services > > > > that are actually enabled/started. > > > > > > Unfortunately this requires all ports to include an mtree file. > > > Relying on port maintainers who are human to ensure that these files > > > are created and updated when ports are created and maintained will > > > result in more human error. I've learned over my long career to rely > > > more on automation than human beings. Automation [should] never fail > > > and when it does it does temporarily until the bug is found and > > > fixed. Human beings inconsistently fail. > > > > > > If it were an auto-discovery script that created an mtree file as > > > part of the packaging process, it would be another matter. But this > > > optional solution path should be discussed on ports@, not here. > > > > > > > > > > I don't have much skin in the game, but I created a little proof of > > concept to allow further discussion (which is not ports-specific, as it > > works for all service scripts): > > > > https://reviews.freebsd.org/D36385 > > I've been toying with the idea for a few months but was never bothered to > create a review or even a script for that matter. > > > > > This basically allows both system admins and port maintainers to > > create mtree files in /usr/local/etc/mtree (or /etc/mtree, as it's > > always relative to the service script called) which are automatically > > applied on service start. It's non-intrusive and doesn't require any > > sweeping changes to existing ports/services. > > Understood that this is a manual process. > > > > > In this specific case, the requester could create > > /usr/local/etc/mtree/clamav-clamd with the required content (or > > persuade the port maintainer to include that file). > > > > You could of course add some construct to the ports framework that > > picks up certain directories from the package list automatically and > > places them into an mtree file as part of the build or installation > > process. But that would be an additional feature on top of this change. > > Someone could. Personally, I think that's a lot of work compared to simply > saving the state of /var/run at shutdown and restoring it at boot. I can't > speak for the ports management though. > > > > > This is meant to inspire more discussions, I'm not trying to force > > anything in. ;) > > Agreed. > > I cobbled something up yesterday that saves the directory tree state of > /var/run prior to shutdown (or manually) and restores it at boot. > > https://reviews.freebsd.org/D36386 > > People can try it out if they want. If there's enough interest I'd be > willing to commit it. > > We have a few options on the table and probably more. The ports > infrastructure option is probably the most work. Adding functionality to > all the ports that use /var/run is also a lot of
