Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
On 2021-03-27 23:40, Joshua Kinard wrote: The MIPS machine has functioning local disk drives, and currently, it boots fine by just pulling a kernel off my TFTP server and booting from the local drive, no initramfs needed because I compiled everything into it. Out of curiosity, if your kernel images already come from the TFTP servers why not simply put separate initramfs files there too? I wonder if there's a small C program out there that can call whatever the kernel functions are to mount a disk partition that could be embedded into a tiny initramfs, then pivot_root to $REAL_ROOT and run /bin/init? You might be interested in this FOSDEM 2020 talk: https://archive.fosdem.org/2020/schedule/event/ema_boot_linux_only/ Not exactly what you have asked for but the problem they are trying to solve is the same as yours - boot Linux on a system whose first-stage bootloader impose considerable size constraints. And since it uses kexec at its core, it's essentially what Rich has suggested - except it's already been (at least partially) done :-) -- MS OpenPGP_signature Description: OpenPGP digital signature
Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
On 2021-03-29 10:06, James Le Cuirot wrote: Have you seen CONFIG_CMDLINE? It lets you bake command line args into the kernel image itself. And since 5.6, there is also bootconfig: https://www.kernel.org/doc/html/latest/admin-guide/bootconfig.html -- MS OpenPGP_signature Description: OpenPGP digital signature
Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
On Sun, 28 Mar 2021 19:46:32 -0400 Joshua Kinard wrote: > I've kinda come to this conclusion myself. I could probably host an > extracted, micro-initramfs on my NFS server that would be loaded by the > kernel to jump to $REAL_ROOT. Not *too* much of an issue on the Octane, > because I can store the kernel command line args in a PROM variable so that > all I have to do is type "$foo" at the PROM prompt to load something. But, > SGI did their best to be inconsistent, and IP27 systems cannot permanently > save variables to NVRAM. Once you power cycle, all user-defined PROM vars > are lost. And Linux's NFS Root command args are somewhat complicated from > what I remember. Upshot is I boot the IP27 over serial console, so I can > just save bootp() args in a text file on my desktop and paste it into the > console for those instances. Have you seen CONFIG_CMDLINE? It lets you bake command line args into the kernel image itself. -- James Le Cuirot (chewi) Gentoo Linux Developer pgpjPj3Fv0z6w.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
This is getting really long, so I'm going to remove more things I'm not answering directly. On Sun, Mar 28, 2021 at 07:31:02PM -0400, Joshua Kinard wrote: > > The problem is, there's a chicken-and-egg problem in the scenario where > > / and /usr are on separate partitions, and this is why a number of linux > > distros have moved to requiring an initramfs in this situation. > > I'm linking systemd's description here, only because it is the best > > writeup of the issue I've found [1]. > > Anything that is needed in the early boot process requires all of its > > libraries, > > dependent libraries, binaries, data files, etc to be on /, and this has > > become a moving target. > > Yeah, I've read systemd's explanation, and generally disagree with it. They > created the problem in the first place, then invented their own solution for > it, and now everyone acts like they're the wise men on the mountain for it. http://lists.busybox.net/pipermail/busybox/2010-December/074114.html Rob Landley had a lot to say about how linux should be using initramfs to handle early boot long before systemd came along, so this has been an issue a lot longer than systemd; the systemd guys just amplified it. > I still don't see the connection to the static *.a libs and whether they're > on /lib or /usr/lib, though. Unless we're implying that where the *.a's go, > so too do the *.so's go, then THAT makes sense because *.so's ARE needed at > program runtime, whereas *.a's are not. It is a linker behavior issue we discovered and worked around. In the past, Gentoo has gone an extra mile to make some things useable by people who have separate /usr without an initramfs by moving only shared libraries to /lib*. We leave everything else that would be in libdir in the default location which is /usr/lib*. This includes things like static libs, pkgconfig files etc. When we started doing this we found that the linker favors libraries in /usr/lib* over libraries in /lib*, so if you are linking to libfoo and there's a static version in /usr/lib* and a shared version in /lib* you will link to the static version. That's what bug 4411 is about. gen_usr_ldscript gets around this by creating a linker script at the original location of the shared library in /usr/lib* when it moves the shared library to /lib*. See /usr/lib64/libbz2.so for an example of the generated linker script. Getting rid of gen_usr_ldscript would move the shared libraries back to their upstream location (/usr/lib*) and remove the linker scripts. This would also allow the removal of the usr-ldscript eclass and the gen_usr_ldscript function from eutils.eclass. The up side of this is that it allows us to get rid of some of our custom code in ebuilds, and it is completely transparent to most of our user base. The down side is that if you are using separate /usr with no initramfs, libraries that you were accessing in /lib* during early boot will moved to /usr/lib* and not be available before /usr is mounted. This would cause breakage until you reconfigured your system to boot with an initramfs and mount / and /usr before jumping into the real system. > I wonder if we couldn't shovel all static libs off to a dedicated folder > somewhere, like '/usr/lib/static//*.a', similar to the way debug files > are now consolidated under '/usr/lib/debug'. Since they're only needed > during a specific kind of compilation that we don't support out-of-the-box > that happens long after the system is fully booted, stuffing them off > somewhere unimportant would make some sense. Most modern software should be > using shared libs by default, and if it ain't, that's either a bug or that > software is for a very specific function (like a bootloader). If you started creating separate static libs folders you would need one per abi, so it would end up being pretty ugly. We don't build that many static libraries right now, so I'm not sure it is worth moving them to some other folder. If we put the shared libs back in the upstream expected location (/usr/lib*) we would eliminate the linker issue. *snip* > > The way I see it, when we start to remove the gen_usr_ldscript calls, > > people using a sep-usr mount without an initramfs will run into one or > > both of these issues: > > > > - they might have to increase the size of their root partition depending > > on what gets added to /lib* > > - if one package in that list drops gen_usr_ldscript without installing > > libraries in /lib*, it will mean they need an initramfs. > > I tend to make my root partitions ~4GB, which has often been plenty of room > for well over 15 years. But again, location of the *.a static libs is > irrelevant during system boot. They are not needed nor referenced when a > program executes. A statically-compiled program has all of its dependencies > lumped inside of it, so you could put it pretty much anywhere on the > filesystem and run it (ignoring for a moment 'noexec' potentially being > set). Or even
Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
On 3/28/2021 05:25, James Le Cuirot wrote:> On Sat, 27 Mar 2021 18:40:52 -0400 Joshua Kinard > wrote: > >> On 3/27/2021 18:16, James Le Cuirot wrote: >>> On Sat, 27 Mar 2021 17:43:34 -0400 Joshua Kinard >>> wrote: >>> I kinda wish the Linux kernel had an ability to partially boot, init the networking subsystem, then fetch an initramfs image over TFTP like it can do with NFS Root. That would solve the problem on my MIPS system(s) (and make install netboots better). I've dug around, but this does not seem to be a capability currently in the kernel, unless I have over looked something. Otherwise in the future, I may just have to setup an initramfs into an NFS Root and teach the SGI's to somehow deal with it. Which all still seems unnecessarily complicated because some other distro thinks it knows what's best for everyone else (but I digress...). >>> >>> NBD may be a slightly simpler alternative and a bit more like an >>> initramfs. nbdkit can do all sorts of weird things. I thought it >>> might have a plugin for cpio archives, allowing you to use a regular >>> initramfs generated by Dracut or similar directly. It doesn't >>> appear to but plugins are quite easy to write. Alternatively you >>> could just extract the initramfs and use nbdkit-linuxdisk-plugin. >> >> Can NBD be used like I described? Never played with it before. The >> MIPS machine has functioning local disk drives, and currently, it >> boots fine by just pulling a kernel off my TFTP server and booting from >> the local drive, no initramfs needed because I compiled everything into >> it. If we ever force sep-usr to end, then I'll need a way to teach it >> to mount /usr before dropping into /bin/init (and nothing else). > > Apologies, I went to experiment with this idea but I'd forgotten that > booting over NBD is not a built-in kernel feature and requires... an > initramfs. NFS looks like the only option with this general approach. I've kinda come to this conclusion myself. I could probably host an extracted, micro-initramfs on my NFS server that would be loaded by the kernel to jump to $REAL_ROOT. Not *too* much of an issue on the Octane, because I can store the kernel command line args in a PROM variable so that all I have to do is type "$foo" at the PROM prompt to load something. But, SGI did their best to be inconsistent, and IP27 systems cannot permanently save variables to NVRAM. Once you power cycle, all user-defined PROM vars are lost. And Linux's NFS Root command args are somewhat complicated from what I remember. Upshot is I boot the IP27 over serial console, so I can just save bootp() args in a text file on my desktop and paste it into the console for those instances. In any event, I kinda wish there was some way to just boot the kernel to a point, have it copy an initramfs image via TFTP, load it, run it, jump to $REAL_ROOT. If I had the time, maybe I could put something together like that myself, but time is not something I have much of right now. > Rich is right though, the initramfs can be tiny. Dracut images are heavy > because they are very flexible and pack a lot of features but none of > that is required. > > The kexec idea would be a nice bonus for you, if it works. Limiting what > kernel features you can enable must be frustrating. My primary kernel image for daily use on the Octane isn't terribly limited. The final vmlinux output is ~39MB, due to loading debugging symbols in. But the cutoff in size for Octane seems to be somewhere around 43-45MB, before the PROM errors out due to no available contiguous FreeMemory() areas (the mem region defined by the ARCS PROM where stuff can be loaded and executed). Netboot kernels for installs can creep up there if you go too crazy on kernel features and pack the netboot with supporting packages (mainly filesystem tools). This already impacts older systems, like SGI Indy, which cannot boot current MIPS netboot images because they have smaller FreeMemory() areas. I thought switching to musl would make the netboots smaller, but that oddly didn't work and musl netboots are about ~2MB larger than the older uclibc-ng versions. And all of my uclibc-ng stages have become unusable for some unknown reason. As soon as you rebuild ncurses in one, all binaries linked to it throws SIGSEGVs (which kills portage). glibc-based netboots are too big and will not load at all, even on the Octane. > It's also worth bearing in mind that you could just generate an image > with Dracut and extract it to a disk partition. That may seem a little > pointless if you're not using something like LVM or encryption but I > don't see the point in separate /usr either. ;) I'm guessing you don't > want to change your partition layout at this point though. No, no LUKS, encryption, btrfs, etc. Just mdraid w/ 0.90 metadata for auto-assemble. Which is why I hate having to use an initramfs for something so simple. Unnecessary
Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
On 3/28/2021 16:05, William Hubbs wrote:
> On Sat, Mar 27, 2021 at 10:51:11PM -0400, Joshua Kinard wrote:
>> On 3/27/2021 20:32, William Hubbs wrote:
>>> On Sat, Mar 27, 2021 at 05:43:34PM -0400, Joshua Kinard wrote:
On 3/23/2021 07:31, Rich Freeman wrote:
> On Mon, Mar 22, 2021 at 6:54 PM Andreas K. Huettel
> wrote:
>>
Council decided years ago that we don't support separate /usr without
an initramfs, but we haven't completed that transition yet.
>>>
>>> Which doesn't imply that we deliberately break things.
>>
>> That's right. Though we should at some point start thinking about an end
>> of support for separate usr without initramfs.
>>
>
> Just to clarify - it is already unsupported at a distro level. It is
> just that some individual packages still work with it.
>
> The current Council decisions on the issue are (just providing for
> general reference):
>
> - "Since that particular setup may already be subtly broken today
> depending on the installed software, Council recommends using an
> early boot mount mechanism, e.g. initramfs, to mount /usr if /usr
> is on a separate partition."
> Accepted unanimously. [1]
>
> - "The intention is to eventually not require maintainers to support
> a separate /usr without an early boot mechanism once the Council
> agrees that the necessary docs/migration path is in place."
> Accepted with 4 yes votes, 1 no vote, 2 abstentions. [1]
>
> - "The Council agrees that all preparations for dropping support for
> separate /usr without an initramfs or similar boot mechanism are
> complete. A news item will be prepared, and users will be given one
> month to switch after the news item has been sent."
> Accepted with 5 yes votes, 1 no vote, 1 abstention. [2]
>
> Current policy documentation:
> Developers are not required to support using separate /usr filesystem
> without an initramfs. [3]
>
> 1 - https://projects.gentoo.org/council/meeting-logs/20130813-summary.txt
> 2 - https://projects.gentoo.org/council/meeting-logs/20130924-summary.txt
> 3 - https://projects.gentoo.org/qa/policy-guide/filesystem.html#pg0202
Is there a list of software/ebuilds that currently do this "subtle"
handling
of separate /usr w/o initramfs?
I've got just my MIPS systems left that use a separate /usr and do not boot
with initramfs because I build fully monolithic kernels and that makes the
resulting vmlinux images run up against hard size limits in the SGI PROM
(firmware, BIOS, etc) on these machines if I try to pack too large of an
initramfs in. I can check for any software that may be switched over soon
to a hard initramfs requirement and look at my options.
>>>
>>> One group of packages involved in this is any package that calls
>>> gen_usr_ldscript. We have this function for a couple of reasons.
>>>
>>> Gentoo has a policy that bans *.a static libraries from being
>>> installed in /lib*. I think this policy is unique to Gentoo,
>>> because Most build systems I've seen do not
>>> have separate paths for static vs dynamic libraries, so all libraries
>>> from a package are installed in /usr/lib* or /lib*. This policy was
>>> originally hard coded into portage some time back (I can find the commit
>>> if you want it) before it was made a formal policy by the qa team.
>>> It has since become a formal policy.
>>>
>>> Because of this policy, we have to tell upstream build systems to
>>> install libraries in ${ED}/usr/$(get_libdir). This is fine unless you
>>> have separate usr with no initramfs. In that case, you have binaries on
>>> / that link to shared libraries on /usr. When we saw this happening, we
>>> started manually moving shared libraries from /usr/lib* to /lib* if
>>> someone needed the package at boot time. Then we hit bug 4411 [1]
>>> where the linker was prioritizing static libraries in /usr/lib* over shared
>>> libraries in /lib*.
>>>
>>> I don't know if we tried to address that upstream or not, but ultimately
>>> gen_usr_ldscript came out of it.
>>>
>>> Several folks have wanted to get rid of this function for years [2].
>>>
>>> I have looked into it before, and there are several things that can be done.
>>>
>>> We can have packages that currently build static libraries and
>>> use gen_usr_ldscript stop building static libraries and use the
>>> appropriate setting in their build systems to install libraries in
>>> /$(get_libdir). This is the path OpenRC is taking per request of the qa
>>> lead. The next release will not support the static-libs use flag. This
>>> will actively break anyone who is expecting this use flag to exist for
>>> OpenRC.
>>>
>>> If a package does not build static libraries in the first place, there is
>>> really no reason to call gen_usr_ldscript; the ebuild should be using
>>> the upstream build
Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
On Sat, Mar 27, 2021 at 10:51:11PM -0400, Joshua Kinard wrote:
> On 3/27/2021 20:32, William Hubbs wrote:
> > On Sat, Mar 27, 2021 at 05:43:34PM -0400, Joshua Kinard wrote:
> >> On 3/23/2021 07:31, Rich Freeman wrote:
> >>> On Mon, Mar 22, 2021 at 6:54 PM Andreas K. Huettel
> >>> wrote:
>
> >> Council decided years ago that we don't support separate /usr without
> >> an initramfs, but we haven't completed that transition yet.
> >
> > Which doesn't imply that we deliberately break things.
>
> That's right. Though we should at some point start thinking about an end
> of support for separate usr without initramfs.
>
> >>>
> >>> Just to clarify - it is already unsupported at a distro level. It is
> >>> just that some individual packages still work with it.
> >>>
> >>> The current Council decisions on the issue are (just providing for
> >>> general reference):
> >>>
> >>> - "Since that particular setup may already be subtly broken today
> >>> depending on the installed software, Council recommends using an
> >>> early boot mount mechanism, e.g. initramfs, to mount /usr if /usr
> >>> is on a separate partition."
> >>> Accepted unanimously. [1]
> >>>
> >>> - "The intention is to eventually not require maintainers to support
> >>> a separate /usr without an early boot mechanism once the Council
> >>> agrees that the necessary docs/migration path is in place."
> >>> Accepted with 4 yes votes, 1 no vote, 2 abstentions. [1]
> >>>
> >>> - "The Council agrees that all preparations for dropping support for
> >>> separate /usr without an initramfs or similar boot mechanism are
> >>> complete. A news item will be prepared, and users will be given one
> >>> month to switch after the news item has been sent."
> >>> Accepted with 5 yes votes, 1 no vote, 1 abstention. [2]
> >>>
> >>> Current policy documentation:
> >>> Developers are not required to support using separate /usr filesystem
> >>> without an initramfs. [3]
> >>>
> >>> 1 - https://projects.gentoo.org/council/meeting-logs/20130813-summary.txt
> >>> 2 - https://projects.gentoo.org/council/meeting-logs/20130924-summary.txt
> >>> 3 - https://projects.gentoo.org/qa/policy-guide/filesystem.html#pg0202
> >>
> >> Is there a list of software/ebuilds that currently do this "subtle"
> >> handling
> >> of separate /usr w/o initramfs?
> >>
> >> I've got just my MIPS systems left that use a separate /usr and do not boot
> >> with initramfs because I build fully monolithic kernels and that makes the
> >> resulting vmlinux images run up against hard size limits in the SGI PROM
> >> (firmware, BIOS, etc) on these machines if I try to pack too large of an
> >> initramfs in. I can check for any software that may be switched over soon
> >> to a hard initramfs requirement and look at my options.
> >
> > One group of packages involved in this is any package that calls
> > gen_usr_ldscript. We have this function for a couple of reasons.
> >
> > Gentoo has a policy that bans *.a static libraries from being
> > installed in /lib*. I think this policy is unique to Gentoo,
> > because Most build systems I've seen do not
> > have separate paths for static vs dynamic libraries, so all libraries
> > from a package are installed in /usr/lib* or /lib*. This policy was
> > originally hard coded into portage some time back (I can find the commit
> > if you want it) before it was made a formal policy by the qa team.
> > It has since become a formal policy.
> >
> > Because of this policy, we have to tell upstream build systems to
> > install libraries in ${ED}/usr/$(get_libdir). This is fine unless you
> > have separate usr with no initramfs. In that case, you have binaries on
> > / that link to shared libraries on /usr. When we saw this happening, we
> > started manually moving shared libraries from /usr/lib* to /lib* if
> > someone needed the package at boot time. Then we hit bug 4411 [1]
> > where the linker was prioritizing static libraries in /usr/lib* over shared
> > libraries in /lib*.
> >
> > I don't know if we tried to address that upstream or not, but ultimately
> > gen_usr_ldscript came out of it.
> >
> > Several folks have wanted to get rid of this function for years [2].
> >
> > I have looked into it before, and there are several things that can be done.
> >
> > We can have packages that currently build static libraries and
> > use gen_usr_ldscript stop building static libraries and use the
> > appropriate setting in their build systems to install libraries in
> > /$(get_libdir). This is the path OpenRC is taking per request of the qa
> > lead. The next release will not support the static-libs use flag. This
> > will actively break anyone who is expecting this use flag to exist for
> > OpenRC.
> >
> > If a package does not build static libraries in the first place, there is
> > really no reason to call gen_usr_ldscript; the ebuild should be using
> > the upstream build system to put the libraries where
Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
On Sat, 27 Mar 2021 18:40:52 -0400 Joshua Kinard wrote: > On 3/27/2021 18:16, James Le Cuirot wrote: > > On Sat, 27 Mar 2021 17:43:34 -0400 > > Joshua Kinard wrote: > > > >> I kinda wish the Linux kernel had an ability to partially boot, init the > >> networking subsystem, then fetch an initramfs image over TFTP like it can > >> do > >> with NFS Root. That would solve the problem on my MIPS system(s) (and make > >> install netboots better). I've dug around, but this does not seem to be a > >> capability currently in the kernel, unless I have over looked something. > >> > >> Otherwise in the future, I may just have to setup an initramfs into an NFS > >> Root and teach the SGI's to somehow deal with it. Which all still seems > >> unnecessarily complicated because some other distro thinks it knows what's > >> best for everyone else (but I digress...). > > > > NBD may be a slightly simpler alternative and a bit more like an > > initramfs. nbdkit can do all sorts of weird things. I thought it might > > have a plugin for cpio archives, allowing you to use a regular > > initramfs generated by Dracut or similar directly. It doesn't appear to > > but plugins are quite easy to write. Alternatively you could just > > extract the initramfs and use nbdkit-linuxdisk-plugin. > > Can NBD be used like I described? Never played with it before. The MIPS > machine has functioning local disk drives, and currently, it boots fine by > just pulling a kernel off my TFTP server and booting from the local drive, > no initramfs needed because I compiled everything into it. If we ever force > sep-usr to end, then I'll need a way to teach it to mount /usr before > dropping into /bin/init (and nothing else). Apologies, I went to experiment with this idea but I'd forgotten that booting over NBD is not a built-in kernel feature and requires... an initramfs. NFS looks like the only option with this general approach. Rich is right though, the initramfs can be tiny. Dracut images are heavy because they are very flexible and pack a lot of features but none of that is required. The kexec idea would be a nice bonus for you, if it works. Limiting what kernel features you can enable must be frustrating. It's also worth bearing in mind that you could just generate an image with Dracut and extract it to a disk partition. That may seem a little pointless if you're not using something like LVM or encryption but I don't see the point in separate /usr either. ;) I'm guessing you don't want to change your partition layout at this point though. -- James Le Cuirot (chewi) Gentoo Linux Developer pgpAb6gauIo0N.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
On 3/27/2021 20:32, William Hubbs wrote:
> On Sat, Mar 27, 2021 at 05:43:34PM -0400, Joshua Kinard wrote:
>> On 3/23/2021 07:31, Rich Freeman wrote:
>>> On Mon, Mar 22, 2021 at 6:54 PM Andreas K. Huettel
>>> wrote:
>> Council decided years ago that we don't support separate /usr without
>> an initramfs, but we haven't completed that transition yet.
>
> Which doesn't imply that we deliberately break things.
That's right. Though we should at some point start thinking about an end
of support for separate usr without initramfs.
>>>
>>> Just to clarify - it is already unsupported at a distro level. It is
>>> just that some individual packages still work with it.
>>>
>>> The current Council decisions on the issue are (just providing for
>>> general reference):
>>>
>>> - "Since that particular setup may already be subtly broken today
>>> depending on the installed software, Council recommends using an
>>> early boot mount mechanism, e.g. initramfs, to mount /usr if /usr
>>> is on a separate partition."
>>> Accepted unanimously. [1]
>>>
>>> - "The intention is to eventually not require maintainers to support
>>> a separate /usr without an early boot mechanism once the Council
>>> agrees that the necessary docs/migration path is in place."
>>> Accepted with 4 yes votes, 1 no vote, 2 abstentions. [1]
>>>
>>> - "The Council agrees that all preparations for dropping support for
>>> separate /usr without an initramfs or similar boot mechanism are
>>> complete. A news item will be prepared, and users will be given one
>>> month to switch after the news item has been sent."
>>> Accepted with 5 yes votes, 1 no vote, 1 abstention. [2]
>>>
>>> Current policy documentation:
>>> Developers are not required to support using separate /usr filesystem
>>> without an initramfs. [3]
>>>
>>> 1 - https://projects.gentoo.org/council/meeting-logs/20130813-summary.txt
>>> 2 - https://projects.gentoo.org/council/meeting-logs/20130924-summary.txt
>>> 3 - https://projects.gentoo.org/qa/policy-guide/filesystem.html#pg0202
>>
>> Is there a list of software/ebuilds that currently do this "subtle" handling
>> of separate /usr w/o initramfs?
>>
>> I've got just my MIPS systems left that use a separate /usr and do not boot
>> with initramfs because I build fully monolithic kernels and that makes the
>> resulting vmlinux images run up against hard size limits in the SGI PROM
>> (firmware, BIOS, etc) on these machines if I try to pack too large of an
>> initramfs in. I can check for any software that may be switched over soon
>> to a hard initramfs requirement and look at my options.
>
> One group of packages involved in this is any package that calls
> gen_usr_ldscript. We have this function for a couple of reasons.
>
> Gentoo has a policy that bans *.a static libraries from being
> installed in /lib*. I think this policy is unique to Gentoo,
> because Most build systems I've seen do not
> have separate paths for static vs dynamic libraries, so all libraries
> from a package are installed in /usr/lib* or /lib*. This policy was
> originally hard coded into portage some time back (I can find the commit
> if you want it) before it was made a formal policy by the qa team.
> It has since become a formal policy.
>
> Because of this policy, we have to tell upstream build systems to
> install libraries in ${ED}/usr/$(get_libdir). This is fine unless you
> have separate usr with no initramfs. In that case, you have binaries on
> / that link to shared libraries on /usr. When we saw this happening, we
> started manually moving shared libraries from /usr/lib* to /lib* if
> someone needed the package at boot time. Then we hit bug 4411 [1]
> where the linker was prioritizing static libraries in /usr/lib* over shared
> libraries in /lib*.
>
> I don't know if we tried to address that upstream or not, but ultimately
> gen_usr_ldscript came out of it.
>
> Several folks have wanted to get rid of this function for years [2].
>
> I have looked into it before, and there are several things that can be done.
>
> We can have packages that currently build static libraries and
> use gen_usr_ldscript stop building static libraries and use the
> appropriate setting in their build systems to install libraries in
> /$(get_libdir). This is the path OpenRC is taking per request of the qa
> lead. The next release will not support the static-libs use flag. This
> will actively break anyone who is expecting this use flag to exist for OpenRC.
>
> If a package does not build static libraries in the first place, there is
> really no reason to call gen_usr_ldscript; the ebuild should be using
> the upstream build system to put the libraries where they need to be.
The number of packages calling gen_usr_ldscript looks to be fairly small.
Creating a TRACKER bug and sub-bugs for checking or removing the need for
gen_usr_ldscript might be a way to go to pair the list down and reduce the
footprint:
Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
On 3/27/2021 21:00, Rich Freeman wrote: > On Sat, Mar 27, 2021 at 5:43 PM Joshua Kinard wrote: >> >> I kinda wish the Linux kernel had an ability to partially boot, init the >> networking subsystem, then fetch an initramfs image over TFTP like it can do >> with NFS Root. That would solve the problem on my MIPS system(s) (and make >> install netboots better). I've dug around, but this does not seem to be a >> capability currently in the kernel, unless I have over looked something. > > Support exists, in the form of an initramfs. Bear with me for a second. > > An initramfs does not need to be large. It is just software. Sure, > they're often implemented in bash with all sorts of userspace tools in > the image, and that uses lots of space. However, an initramfs could > be nothing but a small compiled binary and nothing else - a few > kilobytes even. > > I'm not aware if anything like this has already been created, but you > could do something reminiscent of coreboot, and it would be > distro-agnostic. Just build a minimal kernel with only support for > whatever you need to read a kernel and initramfs (from disk, network, > whatever). Then build an initramfs that minimally fetches that kernel > - it could just be a tiny binary, or it could use tools like curl/etc. > Then you'd kexec the new kernel/initramfs which would do the full > boot, and this wouldn't have any size limitation. > > An initramfs is just a way to automate the kernel boot process using > userspace code, vs needing to do a lot of fancy stuff in kernel space. > They can get a bit bloated when you make them full-featured/generic, > but if you just want a secondary bootloader you can shrink both the > kernel and the initramfs considerably, and use that to boot another > kernel. > > If I were to look anywhere for something I could use out-of-the-box it > would be with coreboot. That is a linux-based kernel/initramfs > designed to be a bootloader really. I believe kexec support is fairly new on MIPS, and I am not 100% certain it works with the older SGI hardware. Each SGI machine is not entirely dissimilar from a unique sub-architecture, with their own memory map layouts that are enforced at the PROM level. I believe that kexec is picky about where it stores or jumps into the newer kernel, and on the SGI machines, that's going to be tricky, since there's only one or two FreeMemory() areas available to load things into. And it may not even work at all on the IP27 platform that is ccNUMA-aware and can have memory scattered across multiple physical machine nodes (and I lack the hardware and 240V plugs to even test this). As such, this might be an endeavor that has too much overhead to attempt. As it stands, I currently only have 5.4 LTS kernels working on Octane (IP30) and Origin/Onyx (IP27). There were massive changes from 5.5 through 5.7 that deal with the mainlining of the SGI Octane code (and cleanups in the IP27 PCI core) that I haven't had time to deal with yet, so I am quite a ways behind on things. Hope to take a crack at porting to 5.10 in the next few months (which is why I am sticking to LTS kernels now -- mainline kernels move too fast these days). -- Joshua Kinard Gentoo/MIPS ku...@gentoo.org rsa6144/5C63F4E3F5C6C943 2015-04-27 177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943 "The past tempts us, the present confuses us, the future frightens us. And our lives slip away, moment by moment, lost in that vast, terrible in-between." --Emperor Turhan, Centauri Republic
Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
On Sat, Mar 27, 2021 at 5:43 PM Joshua Kinard wrote: > > I kinda wish the Linux kernel had an ability to partially boot, init the > networking subsystem, then fetch an initramfs image over TFTP like it can do > with NFS Root. That would solve the problem on my MIPS system(s) (and make > install netboots better). I've dug around, but this does not seem to be a > capability currently in the kernel, unless I have over looked something. Support exists, in the form of an initramfs. Bear with me for a second. An initramfs does not need to be large. It is just software. Sure, they're often implemented in bash with all sorts of userspace tools in the image, and that uses lots of space. However, an initramfs could be nothing but a small compiled binary and nothing else - a few kilobytes even. I'm not aware if anything like this has already been created, but you could do something reminiscent of coreboot, and it would be distro-agnostic. Just build a minimal kernel with only support for whatever you need to read a kernel and initramfs (from disk, network, whatever). Then build an initramfs that minimally fetches that kernel - it could just be a tiny binary, or it could use tools like curl/etc. Then you'd kexec the new kernel/initramfs which would do the full boot, and this wouldn't have any size limitation. An initramfs is just a way to automate the kernel boot process using userspace code, vs needing to do a lot of fancy stuff in kernel space. They can get a bit bloated when you make them full-featured/generic, but if you just want a secondary bootloader you can shrink both the kernel and the initramfs considerably, and use that to boot another kernel. If I were to look anywhere for something I could use out-of-the-box it would be with coreboot. That is a linux-based kernel/initramfs designed to be a bootloader really. -- Rich
Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
On Sat, Mar 27, 2021 at 05:43:34PM -0400, Joshua Kinard wrote:
> On 3/23/2021 07:31, Rich Freeman wrote:
> > On Mon, Mar 22, 2021 at 6:54 PM Andreas K. Huettel
> > wrote:
> >>
> Council decided years ago that we don't support separate /usr without
> an initramfs, but we haven't completed that transition yet.
> >>>
> >>> Which doesn't imply that we deliberately break things.
> >>
> >> That's right. Though we should at some point start thinking about an end
> >> of support for separate usr without initramfs.
> >>
> >
> > Just to clarify - it is already unsupported at a distro level. It is
> > just that some individual packages still work with it.
> >
> > The current Council decisions on the issue are (just providing for
> > general reference):
> >
> > - "Since that particular setup may already be subtly broken today
> > depending on the installed software, Council recommends using an
> > early boot mount mechanism, e.g. initramfs, to mount /usr if /usr
> > is on a separate partition."
> > Accepted unanimously. [1]
> >
> > - "The intention is to eventually not require maintainers to support
> > a separate /usr without an early boot mechanism once the Council
> > agrees that the necessary docs/migration path is in place."
> > Accepted with 4 yes votes, 1 no vote, 2 abstentions. [1]
> >
> > - "The Council agrees that all preparations for dropping support for
> > separate /usr without an initramfs or similar boot mechanism are
> > complete. A news item will be prepared, and users will be given one
> > month to switch after the news item has been sent."
> > Accepted with 5 yes votes, 1 no vote, 1 abstention. [2]
> >
> > Current policy documentation:
> > Developers are not required to support using separate /usr filesystem
> > without an initramfs. [3]
> >
> > 1 - https://projects.gentoo.org/council/meeting-logs/20130813-summary.txt
> > 2 - https://projects.gentoo.org/council/meeting-logs/20130924-summary.txt
> > 3 - https://projects.gentoo.org/qa/policy-guide/filesystem.html#pg0202
>
> Is there a list of software/ebuilds that currently do this "subtle" handling
> of separate /usr w/o initramfs?
>
> I've got just my MIPS systems left that use a separate /usr and do not boot
> with initramfs because I build fully monolithic kernels and that makes the
> resulting vmlinux images run up against hard size limits in the SGI PROM
> (firmware, BIOS, etc) on these machines if I try to pack too large of an
> initramfs in. I can check for any software that may be switched over soon
> to a hard initramfs requirement and look at my options.
One group of packages involved in this is any package that calls
gen_usr_ldscript. We have this function for a couple of reasons.
Gentoo has a policy that bans *.a static libraries from being
installed in /lib*. I think this policy is unique to Gentoo,
because Most build systems I've seen do not
have separate paths for static vs dynamic libraries, so all libraries
from a package are installed in /usr/lib* or /lib*. This policy was
originally hard coded into portage some time back (I can find the commit
if you want it) before it was made a formal policy by the qa team.
It has since become a formal policy.
Because of this policy, we have to tell upstream build systems to
install libraries in ${ED}/usr/$(get_libdir). This is fine unless you
have separate usr with no initramfs. In that case, you have binaries on
/ that link to shared libraries on /usr. When we saw this happening, we
started manually moving shared libraries from /usr/lib* to /lib* if
someone needed the package at boot time. Then we hit bug 4411 [1]
where the linker was prioritizing static libraries in /usr/lib* over shared
libraries in /lib*.
I don't know if we tried to address that upstream or not, but ultimately
gen_usr_ldscript came out of it.
Several folks have wanted to get rid of this function for years [2].
I have looked into it before, and there are several things that can be done.
We can have packages that currently build static libraries and
use gen_usr_ldscript stop building static libraries and use the
appropriate setting in their build systems to install libraries in
/$(get_libdir). This is the path OpenRC is taking per request of the qa
lead. The next release will not support the static-libs use flag. This
will actively break anyone who is expecting this use flag to exist for OpenRC.
If a package does not build static libraries in the first place, there is
really no reason to call gen_usr_ldscript; the ebuild should be using
the upstream build system to put the libraries where they need to be.
If static libraries are needed for a package that is using
gen_usr_ldscript to move shared libraries to /lib*, the only option you
have is to drop the gen_usr_ldscript call. If you do that, all of the
libraries will stay in /usr/lib*, but as soon as one package takes this
path, there will be active breakage for someone who is using a separate
/usr without an
Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
On 3/27/2021 18:16, James Le Cuirot wrote: > On Sat, 27 Mar 2021 17:43:34 -0400 > Joshua Kinard wrote: > >> I kinda wish the Linux kernel had an ability to partially boot, init the >> networking subsystem, then fetch an initramfs image over TFTP like it can do >> with NFS Root. That would solve the problem on my MIPS system(s) (and make >> install netboots better). I've dug around, but this does not seem to be a >> capability currently in the kernel, unless I have over looked something. >> >> Otherwise in the future, I may just have to setup an initramfs into an NFS >> Root and teach the SGI's to somehow deal with it. Which all still seems >> unnecessarily complicated because some other distro thinks it knows what's >> best for everyone else (but I digress...). > > NBD may be a slightly simpler alternative and a bit more like an > initramfs. nbdkit can do all sorts of weird things. I thought it might > have a plugin for cpio archives, allowing you to use a regular > initramfs generated by Dracut or similar directly. It doesn't appear to > but plugins are quite easy to write. Alternatively you could just > extract the initramfs and use nbdkit-linuxdisk-plugin. Can NBD be used like I described? Never played with it before. The MIPS machine has functioning local disk drives, and currently, it boots fine by just pulling a kernel off my TFTP server and booting from the local drive, no initramfs needed because I compiled everything into it. If we ever force sep-usr to end, then I'll need a way to teach it to mount /usr before dropping into /bin/init (and nothing else). I wonder if there's a small C program out there that can call whatever the kernel functions are to mount a disk partition that could be embedded into a tiny initramfs, then pivot_root to $REAL_ROOT and run /bin/init? -- Joshua Kinard Gentoo/MIPS ku...@gentoo.org rsa6144/5C63F4E3F5C6C943 2015-04-27 177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943 "The past tempts us, the present confuses us, the future frightens us. And our lives slip away, moment by moment, lost in that vast, terrible in-between." --Emperor Turhan, Centauri Republic
Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
On Sat, 27 Mar 2021 17:43:34 -0400 Joshua Kinard wrote: > I kinda wish the Linux kernel had an ability to partially boot, init the > networking subsystem, then fetch an initramfs image over TFTP like it can do > with NFS Root. That would solve the problem on my MIPS system(s) (and make > install netboots better). I've dug around, but this does not seem to be a > capability currently in the kernel, unless I have over looked something. > > Otherwise in the future, I may just have to setup an initramfs into an NFS > Root and teach the SGI's to somehow deal with it. Which all still seems > unnecessarily complicated because some other distro thinks it knows what's > best for everyone else (but I digress...). NBD may be a slightly simpler alternative and a bit more like an initramfs. nbdkit can do all sorts of weird things. I thought it might have a plugin for cpio archives, allowing you to use a regular initramfs generated by Dracut or similar directly. It doesn't appear to but plugins are quite easy to write. Alternatively you could just extract the initramfs and use nbdkit-linuxdisk-plugin. -- James Le Cuirot (chewi) Gentoo Linux Developer pgpGX8oMTiAmr.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
On 3/23/2021 07:31, Rich Freeman wrote: > On Mon, Mar 22, 2021 at 6:54 PM Andreas K. Huettel > wrote: >> Council decided years ago that we don't support separate /usr without an initramfs, but we haven't completed that transition yet. >>> >>> Which doesn't imply that we deliberately break things. >> >> That's right. Though we should at some point start thinking about an end of >> support for separate usr without initramfs. >> > > Just to clarify - it is already unsupported at a distro level. It is > just that some individual packages still work with it. > > The current Council decisions on the issue are (just providing for > general reference): > > - "Since that particular setup may already be subtly broken today > depending on the installed software, Council recommends using an > early boot mount mechanism, e.g. initramfs, to mount /usr if /usr > is on a separate partition." > Accepted unanimously. [1] > > - "The intention is to eventually not require maintainers to support > a separate /usr without an early boot mechanism once the Council > agrees that the necessary docs/migration path is in place." > Accepted with 4 yes votes, 1 no vote, 2 abstentions. [1] > > - "The Council agrees that all preparations for dropping support for > separate /usr without an initramfs or similar boot mechanism are > complete. A news item will be prepared, and users will be given one > month to switch after the news item has been sent." > Accepted with 5 yes votes, 1 no vote, 1 abstention. [2] > > Current policy documentation: > Developers are not required to support using separate /usr filesystem > without an initramfs. [3] > > 1 - https://projects.gentoo.org/council/meeting-logs/20130813-summary.txt > 2 - https://projects.gentoo.org/council/meeting-logs/20130924-summary.txt > 3 - https://projects.gentoo.org/qa/policy-guide/filesystem.html#pg0202 Is there a list of software/ebuilds that currently do this "subtle" handling of separate /usr w/o initramfs? I've got just my MIPS systems left that use a separate /usr and do not boot with initramfs because I build fully monolithic kernels and that makes the resulting vmlinux images run up against hard size limits in the SGI PROM (firmware, BIOS, etc) on these machines if I try to pack too large of an initramfs in. I can check for any software that may be switched over soon to a hard initramfs requirement and look at my options. I kinda wish the Linux kernel had an ability to partially boot, init the networking subsystem, then fetch an initramfs image over TFTP like it can do with NFS Root. That would solve the problem on my MIPS system(s) (and make install netboots better). I've dug around, but this does not seem to be a capability currently in the kernel, unless I have over looked something. Otherwise in the future, I may just have to setup an initramfs into an NFS Root and teach the SGI's to somehow deal with it. Which all still seems unnecessarily complicated because some other distro thinks it knows what's best for everyone else (but I digress...). -- Joshua Kinard Gentoo/MIPS ku...@gentoo.org rsa6144/5C63F4E3F5C6C943 2015-04-27 177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943 "The past tempts us, the present confuses us, the future frightens us. And our lives slip away, moment by moment, lost in that vast, terrible in-between." --Emperor Turhan, Centauri Republic
Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
On Mon, Mar 22, 2021 at 6:54 PM Andreas K. Huettel wrote: > > > > Council decided years ago that we don't support separate /usr without > > > an initramfs, but we haven't completed that transition yet. > > > > Which doesn't imply that we deliberately break things. > > That's right. Though we should at some point start thinking about an end of > support for separate usr without initramfs. > Just to clarify - it is already unsupported at a distro level. It is just that some individual packages still work with it. The current Council decisions on the issue are (just providing for general reference): - "Since that particular setup may already be subtly broken today depending on the installed software, Council recommends using an early boot mount mechanism, e.g. initramfs, to mount /usr if /usr is on a separate partition." Accepted unanimously. [1] - "The intention is to eventually not require maintainers to support a separate /usr without an early boot mechanism once the Council agrees that the necessary docs/migration path is in place." Accepted with 4 yes votes, 1 no vote, 2 abstentions. [1] - "The Council agrees that all preparations for dropping support for separate /usr without an initramfs or similar boot mechanism are complete. A news item will be prepared, and users will be given one month to switch after the news item has been sent." Accepted with 5 yes votes, 1 no vote, 1 abstention. [2] Current policy documentation: Developers are not required to support using separate /usr filesystem without an initramfs. [3] 1 - https://projects.gentoo.org/council/meeting-logs/20130813-summary.txt 2 - https://projects.gentoo.org/council/meeting-logs/20130924-summary.txt 3 - https://projects.gentoo.org/qa/policy-guide/filesystem.html#pg0202 -- Rich
Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
Hi Andreas, On 2021/03/23 00:54, Andreas K. Huettel wrote: >>> Council decided years ago that we don't support separate /usr without >>> an initramfs, but we haven't completed that transition yet. >> Which doesn't imply that we deliberately break things. > That's right. Though we should at some point start thinking about an end of > support for separate usr without initramfs. > > Why? Because the number of required hacks and complexity will only increase, > as will the number of uncooperative upstreams. It's called a strategic > retreat. :D > > My suggestion would be that the next profile version (21? 22?) declares > separate /usr a broken configuration, and explicitly encourages devs to > introduce all ebuild simplifications that are made possible by this. (Like > this symlink - no more conditional code.) No more discussions about "not > breaking things" at that point. Why was it ever copied in the first place? For as long as I can recall (been using Gentoo since 2003) I've always been using a symlink here, not to mention a separate /usr (which has only been in the last year or so mounted from initrd along with /lib/firmware - which was the trigger point when it became too big to be contained on our normal / partition, the other fix would have been to be more selective in which firmware to install but that would be a higher administrative overhead). To this day I still believe that / should contain a minimal viable bootable system (give me a shell and just enough to perform basic tasks like activating LVM, repairing and mounting filesystems, and ideally some or another editor such as busybox vi is good enough, mostly everything else can go to /usr even with it being "split"). I still don't see why a split /usr is a bad thing. In fact, there are a significant number cases where I've had FS corruption which I was able to recover without the need for additional bootable media (which when working remotely via IP KVMs can be difficult at best) due to "living in the past" as you say. There is no reason a symbolic link can't cross a filesystem boundary ... it's hard links that can't. Kind Regards, Jaco > (Or to put it another way, I think we should stop wasting time and effort > here just to be able to live in the past.) >
Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
On Monday, 22 March 2021 21:18:32 CET Lars Wendler wrote: > With enough motivation we can carry that revert for a very long time. I > know that because I still carry reverts in my udev packages from when > it was devoured by systemd. It is 11.2 KiB worth of patch that at least I know I won't take responsibility over. signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
> > Council decided years ago that we don't support separate /usr without > > an initramfs, but we haven't completed that transition yet. > > Which doesn't imply that we deliberately break things. That's right. Though we should at some point start thinking about an end of support for separate usr without initramfs. Why? Because the number of required hacks and complexity will only increase, as will the number of uncooperative upstreams. It's called a strategic retreat. :D My suggestion would be that the next profile version (21? 22?) declares separate /usr a broken configuration, and explicitly encourages devs to introduce all ebuild simplifications that are made possible by this. (Like this symlink - no more conditional code.) No more discussions about "not breaking things" at that point. (Or to put it another way, I think we should stop wasting time and effort here just to be able to live in the past.) -- Andreas K. Hüttel dilfri...@gentoo.org Gentoo Linux developer (council, toolchain, base-system, perl, libreoffice) signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
On Mon, 22 Mar 2021 09:59:11 +0100 Andreas Sturmlechner wrote: >On Sonntag, 21. März 2021 08:58:34 CET James Le Cuirot wrote: >> How about making emerge --config dynamic, copying if it's on a >> different partition and symlinking if it's not? You can't accurately >> determine the use of an initramfs but at least this is closer to what >> we want. > >We will not be able to carry the revert for Qt5Core forever. I suspect >it will be gone with Qt6, and then all we can do is ewarn users that >their clock will be broken if their localtime is not a symlink. > >Keeping it as a file would mean convincing upstream(s) that the >symlink is a systemd-ism, not a standard. > >Regards With enough motivation we can carry that revert for a very long time. I know that because I still carry reverts in my udev packages from when it was devoured by systemd. -- Lars Wendler Gentoo package maintainer GPG: 21CC CF02 4586 0A07 ED93 9F68 498F E765 960E 9B39 pgpAZvyBvjeAP.pgp Description: Digitale Signatur von OpenPGP
Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
On Sonntag, 21. März 2021 08:58:34 CET James Le Cuirot wrote: > How about making emerge --config dynamic, copying if it's on a > different partition and symlinking if it's not? You can't accurately > determine the use of an initramfs but at least this is closer to what > we want. We will not be able to carry the revert for Qt5Core forever. I suspect it will be gone with Qt6, and then all we can do is ewarn users that their clock will be broken if their localtime is not a symlink. Keeping it as a file would mean convincing upstream(s) that the symlink is a systemd-ism, not a standard. Regards signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
On Sun, Mar 21, 2021 at 11:06 PM Thomas Deutschmann wrote: > > On 2021-03-22 03:06, Mike Gilbert wrote: > > Based on that commit message, it looks systemd switched to looking at > > the symlink target instead of /etc/timezone well *after* some major > > distro started using a symlink for /etc/localtime. I suspect Kay > > Sievers noticed that the content of /etc/timezone and /etc/localtime > > were redundant on his development machine, and added a TODO entry to > > eliminate the redundant /etc/timezone file. > > > > In other words, this isn't a case of systemd forcing distros to > > symlink /etc/localtime; they were already doing that anyway. > > I just downloaded and tested some old distributions: > > Debian 9 was the first Debian release with systemd. Because of systemd, > /etc/localtime became a symlink. In Debian 8 or when you install Debian > 9 without systemd, it is a regular file. > > Ubuntu 12.04.5 is the same: No systemd, /etc/localtime is a regular > file. Once they moved to systemd it became a symlink. > > In Fedora 17, which is already using systemd but a version before linked > commit, /etc/localtime is also a regular file. But once Fedora upgraded > to >=systemd-190 it became a symlink. Thanks for looking into it. I wonder how Kay's system ended up that way then. Just a curiosity. > That's why from my P.O.V. this is clearly caused by systemd. But does > this matter? You're the one who brought it up; I'm not sure what the point of that was, other than to complain about systemd.
Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
PS: Even Debian is mentioning "to follow systemd" when they updated their tzdata package end of 2015, https://bugs.debian.org/803144. OpenPGP_signature Description: OpenPGP digital signature
Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
On 2021-03-22 03:06, Mike Gilbert wrote: Based on that commit message, it looks systemd switched to looking at the symlink target instead of /etc/timezone well *after* some major distro started using a symlink for /etc/localtime. I suspect Kay Sievers noticed that the content of /etc/timezone and /etc/localtime were redundant on his development machine, and added a TODO entry to eliminate the redundant /etc/timezone file. In other words, this isn't a case of systemd forcing distros to symlink /etc/localtime; they were already doing that anyway. I just downloaded and tested some old distributions: Debian 9 was the first Debian release with systemd. Because of systemd, /etc/localtime became a symlink. In Debian 8 or when you install Debian 9 without systemd, it is a regular file. Ubuntu 12.04.5 is the same: No systemd, /etc/localtime is a regular file. Once they moved to systemd it became a symlink. In Fedora 17, which is already using systemd but a version before linked commit, /etc/localtime is also a regular file. But once Fedora upgraded to >=systemd-190 it became a symlink. That's why from my P.O.V. this is clearly caused by systemd. But does this matter? I doubt that systemd will even think about removing what I believe to be a false warning when systemd detects that /etc/localtime is a regular file. So let's focus on dealing with the fallout... -- Regards, Thomas Deutschmann / Gentoo Linux Developer fpr: C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5 OpenPGP_signature Description: OpenPGP digital signature
Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
On Sat, Mar 20, 2021 at 11:37 AM Andreas K. Huettel wrote: > > Hi all, > > why do we *copy* the timezone file to /etc/localtime, instead of symlinking > it like everyone else? > > 1) Our handbook recommends: > > echo "Europe/Brussels" > /etc/timezone > emerge --config sys-libs/timezone-data > > which is effectively doing > > echo "Europe/Brussels" > /etc/timezone > cp -f /usr/share/zoneinfo/Europe/Brussels /etc/localtime > > 2) Most other distros seem to just do > > ln -sf /usr/share/zoneinfo/Europe/Brussels /etc/localtime > > and use the link content as timezone name (this is also what is required by > systemd). > > Does anyone remember the reason for 1) ? Or is that lost in history? I think the most obvious reason would be to support late-mounting of /usr, after the system clock has been reset with a UTC offset on boot. To make this work, the /etc/localtime file would need to be available when the hwclock init script runs. These days, I suspect there are not many users running systems with a hardware clock in local time, and a separate /usr filesystem, and no initramfs to mount /usr early. I created a PR to adjust the logic in timezone-data to promote symlinks for new installs, while keeping regular files in place for existing users. https://github.com/gentoo/gentoo/pull/20050 If we added some logic to detect if /usr is a separate filesystem, we could possibly be even more aggressive about this. But I'm not sure I want to rock the boat that much in one commit.
Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
On Sun, Mar 21, 2021 at 7:58 PM Thomas Deutschmann wrote: > > On 2021-03-20 16:37, Andreas K. Huettel wrote: > > 2) Most other distros seem to just do > > No, not most distros are doing that. systemd is forcing that downstream > (the result is the same)! > > It was added via > https://github.com/systemd/systemd/commit/92c4ef2d357baeef78b6f82f119b92f7ed12ac77 > without mentioning a reason. Based on that commit message, it looks systemd switched to looking at the symlink target instead of /etc/timezone well *after* some major distro started using a symlink for /etc/localtime. I suspect Kay Sievers noticed that the content of /etc/timezone and /etc/localtime were redundant on his development machine, and added a TODO entry to eliminate the redundant /etc/timezone file. In other words, this isn't a case of systemd forcing distros to symlink /etc/localtime; they were already doing that anyway.
Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
On 2021-03-20 16:37, Andreas K. Huettel wrote: 2) Most other distros seem to just do No, not most distros are doing that. systemd is forcing that downstream (the result is the same)! It was added via https://github.com/systemd/systemd/commit/92c4ef2d357baeef78b6f82f119b92f7ed12ac77 without mentioning a reason. -- Regards, Thomas Deutschmann / Gentoo Linux Developer fpr: C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5 OpenPGP_signature Description: OpenPGP digital signature
Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
On Sun, 21 Mar 2021 05:18:52 +0100 Ulrich Mueller wrote: > > On Sat, 20 Mar 2021, William Hubbs wrote: > > > /etc/localtime should definitely be a symlink to the proper file in > > /usr/share/zoneinfo. > > > This works fine if /usr is on a separate partition *and* you are using > > an initramfs. The only time it doesn't work is if /usr is separate > > without using an initramfs. > > > Council decided years ago that we don't support separate /usr without > > an initramfs, but we haven't completed that transition yet. > > Which doesn't imply that we deliberately break things. How about making emerge --config dynamic, copying if it's on a different partition and symlinking if it's not? You can't accurately determine the use of an initramfs but at least this is closer to what we want. -- James Le Cuirot (chewi) Gentoo Linux Developer pgppd9SmOxjcX.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
> On Sun, 21 Mar 2021, Alec Warner wrote: > https://bugs.gentoo.org/737914 seems to imply for some upstreams, it > being a file is not a valid option anymore? > (I'm ignoring the logic of that decision of course, but this was the > original reason this was raised.) Indeed, that's a strange design decision. It also violates the principle of least surprise, because nobody will expect a symlink to behave any different from a literal copy of the file (or even from a hardlink) when reading it. Ulrich signature.asc Description: PGP signature
Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
On Sat, Mar 20, 2021 at 10:27 PM Ulrich Mueller wrote: > > > On Sun, 21 Mar 2021, Alec Warner wrote: > > >> Which doesn't imply that we deliberately break things. > > > Not sure I follow.. how is updating the handbook breaking anything? > > Both configurations (regular file and symlink) work just fine, and > sys-libs/timezone-data supports them. I don't see a good reason why we > would suddenly declare the regular file to be an invalid option. https://bugs.gentoo.org/737914 seems to imply for some upstreams, it being a file is not a valid option anymore? (I'm ignoring the logic of that decision of course, but this was the original reason this was raised.) -A > > Ulrich
Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
> On Sun, 21 Mar 2021, Alec Warner wrote: >> Which doesn't imply that we deliberately break things. > Not sure I follow.. how is updating the handbook breaking anything? Both configurations (regular file and symlink) work just fine, and sys-libs/timezone-data supports them. I don't see a good reason why we would suddenly declare the regular file to be an invalid option. Ulrich signature.asc Description: PGP signature
Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
On Sat, Mar 20, 2021 at 9:19 PM Ulrich Mueller wrote: > > > On Sat, 20 Mar 2021, William Hubbs wrote: > > > /etc/localtime should definitely be a symlink to the proper file in > > /usr/share/zoneinfo. > > > This works fine if /usr is on a separate partition *and* you are using > > an initramfs. The only time it doesn't work is if /usr is separate > > without using an initramfs. > > > Council decided years ago that we don't support separate /usr without > > an initramfs, but we haven't completed that transition yet. > > Which doesn't imply that we deliberately break things. Not sure I follow.. how is updating the handbook breaking anything? -A > > Ulrich
Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
> On Sat, 20 Mar 2021, William Hubbs wrote: > /etc/localtime should definitely be a symlink to the proper file in > /usr/share/zoneinfo. > This works fine if /usr is on a separate partition *and* you are using > an initramfs. The only time it doesn't work is if /usr is separate > without using an initramfs. > Council decided years ago that we don't support separate /usr without > an initramfs, but we haven't completed that transition yet. Which doesn't imply that we deliberately break things. Ulrich signature.asc Description: PGP signature
Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
On Sat, Mar 20, 2021 at 05:04:06PM +0100, Nils Freydank wrote: > Hi Andreas, > > Am Samstag, den 20.03.2021 um 16:37:23 Uhr +0100 schrieb "Andreas K. Huettel" > : > > [...] > > Does anyone remember the reason for 1) ? Or is that lost in history? > > I just quote comment 3 from the linked bug https://bugs.gentoo.org/737914#c3: > "copying the zonefile to /etc/localtime is a good idea, as /usr could be on a > separate partition. How about creating the > > /etc/TZ -> /etc/timezone softlink by default?" > > In my opinion so many tools tend to expect an always-mounted /usr that the > symlink would not do any harm. FWIW I use symlinks on a handfull Gentoo > machines > and had no issues in years, but I'm pretty close to mainstream (amd64, glibc, > OpenRC). I'm not sure which symlink you are talking about, so I am adding my comments here. /etc/localtime should definitely be a symlink to the proper file in /usr/share/zoneinfo. This works fine if /usr is on a separate partition *and* you are using an initramfs. The only time it doesn't work is if /usr is separate without using an initramfs. Council decided years ago that we don't support separate /usr without an initramfs, but we haven't completed that transition yet. William signature.asc Description: PGP signature
Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?
Hi Andreas, Am Samstag, den 20.03.2021 um 16:37:23 Uhr +0100 schrieb "Andreas K. Huettel" : > [...] > Does anyone remember the reason for 1) ? Or is that lost in history? I just quote comment 3 from the linked bug https://bugs.gentoo.org/737914#c3: "copying the zonefile to /etc/localtime is a good idea, as /usr could be on a separate partition. How about creating the /etc/TZ -> /etc/timezone softlink by default?" In my opinion so many tools tend to expect an always-mounted /usr that the symlink would not do any harm. FWIW I use symlinks on a handfull Gentoo machines and had no issues in years, but I'm pretty close to mainstream (amd64, glibc, OpenRC). Best regards, Nils signature.asc Description: PGP signature
