Re: [gentoo-user] Handbook and question about manual network setup

[J. Roeleveld]

On Friday, 19 April 2024 16:05:47 CEST Dale wrote:
> Howdy,
> 
> I'm playing around with my NAS box again.  I ran into a network issue. 
> I sorta forgot I unplugged the network cable so obviously, it made it
> difficult to ssh into the thing from my main rig.  After hooking up a
> monitor and keyboard, I found the problem and plugged the network cable
> back in.  ROFLMBO  Told y'all I forget stuff. 
> 
> Anyway, while investigating this, I realized the network setup is not
> like on my old rig.  Heck, I couldn't even figure out how to restart it
> other than switching to the boot runlevel and back to default, or
> rebooting.  After a bit, I think I can restart DHCP and it restart the
> network.  I figured out the cable was unplugged before trying that.  I'm
> wanting to set up the NAS box network the same way as my main rig. 
> That's the old manual way.  I went back to the install handbook, that's
> what I followed when installing on my main rig.  Thing is, it has been
> updated and the old way isn't all there.  I followed what little bit is
> there but it defaults back to the new way.  I'm sure I'm missing some
> file I need to edit but I can't figure out which one it is.  So, is
> there a way to get the old instructions again?  The ones I followed
> several years ago for my main rig?  I tried searching but it seems they
> all gone.  Maybe there is a place I'm not aware of tho.  Basically, I
> want to be able to start/stop/restart enp3s0 as a service and have it in
> a runlevel. 
> 
> Also, I'd like to get the install handbook as one large page.  My
> intention is to save it locally for future reference as it is now.  I
> may even print a copy.  I looked at all the places that have different
> options but can't find the whole thing as one large page.  I looked
> under several drop down menus and such.  A long time ago, it was a
> option.  I just can't find it now.  May that option isn't available
> anymore.  I wish I had a copy of the one from several years ago.  Back
> when I installed on my main rig. 
> 
> Some network info.  Lines that are commented out are options I tried but
> didn't work.  It was worth a shot.  o_O 
> 
> 
> nas / # grep -r '!net' /etc/
> /etc/rc.conf:rc_hotplug="!net.*"
> nas / # grep -r 'enp3s0' /etc/
> /etc/resolv.conf:# Generated by dhcpcd from enp3s0.dhcp
> /etc/conf.d/net:config_enp3s0="dhcp"
> /etc/conf.d/net:dns_servers_enp3s0="8.8.8.8 8.8.4.4"
> /etc/conf.d/net:#config_enp3s0="10.0.0.5"
> nas / #nas / # ifconfig -s enp3s0
> Iface  MTURX-OK RX-ERR RX-DRP RX-OVRTX-OK TX-ERR TX-DRP
> TX-OVR Flg
> enp3s0   150016802  0  0 0 17196  0 
> 0  0 BMRU
> nas / #
> 
> 
> Thoughts?  If I had the old install info, I think I could get it to
> work.  I did last time.  ;-)

Yes, try:
config_enp3s0="10.0.0.5/24"
routes_enp3s0="default gw "

Changes to what I see:
1) You forgot the netmask ( /24 ) for the network
2) I don't see a default route

--
Joost

Re: [gentoo-user] How to find out all openrc dependencies?

[J. Roeleveld]

On Sunday, 14 April 2024 10:24:54 CEST Michael wrote:
> On Sunday, 14 April 2024 08:28:07 BST J. Roeleveld wrote:



> > > 
> > > Our systems are certainly different, but I noticed this dependency on my
> > > localmount which is missing on yours:
> > > 
> > > # /lib/rc/bin/rc-depend localmount
> > > sysfs devfs udev udev-trigger hwclock modules fsck root dmcrypt
> > > localmount
> > > 
> > >   ^^^
> > > 
> > > Have you compared your system services which has this problem, with
> > > other
> > > systems of yours which can startup properly?
> > 
> > Adding additional dependencies into the tree is more likely to cause
> > further issues. I am actually looking for how to quickly find out which
> > dependency is causing a circular dependency issue as the first time it
> > thinks it needs to start a service it fails. But the 2nd time it starts,
> > it goes correctly.
> > 
> > I removed hwclock from ALL VMs as they don't actually have a hwclock.
> > 
> > I did find out the actual cause of the problem through a lot of trial and
> > error,
> 
> Out of curiosity - what was the cause of this?  I have only come across
> hwclock on my installations (not VMs).

The issue is related to how openrc handles the boot process. It first starts 
everything in "boot" and then switches to whatever is set as default runlevel.

At some point, Gentoo devs found it useful to add multipath and zfs-import to 
the boot-runlevel. This was not originally the case.
I consider it best that the boot-runlevel is kept as empty as possible and 
have the user/admin add necessary stuff to it when needed.

I removed them from the boot runlevel and the issue has been resolved.

This was the last item on the list preventing correct parallel boot and my 
server now boots in 8 minutes instead of the original 20.
It's still 8 because I need to add some tricks to reduce some of the 
simultaneous stuff which will lead to timeout issues which actually make it 
worse if left to defaults.

> > but this is not really useful in actually quickly finding the
> > problem. Being able to "simulate" the startup sequence for how OpenRC
> > wants
> > to do things would have simplified and sped up the entire process.
> > 
> > --
> > Joost
> 
> I enable the rc log and check how the various services try to start up,
> however the information provided is not always useful.

The rc.log is actually less useful then one would expect. It only logs the 
same output I see on the terminal.
Unless there is an option to add additional information to the logs, like how 
it determines what to start next?

--
Joost

Re: [gentoo-user] How to find out all openrc dependencies?

[J. Roeleveld]

On Thursday, 11 April 2024 12:10:31 CEST Michael wrote:
> On Thursday, 11 April 2024 10:48:15 BST J. Roeleveld wrote:
> > On Thursday, 11 April 2024 11:35:10 CEST Michael wrote:
> > > On Thursday, 11 April 2024 06:19:57 BST J. Roeleveld wrote:
> > > > Hi all,
> > > > 
> > > > For a while I've been seeing the following ERROR-messages when booting
> > > > 1
> > > > of
> > > > my systems:
> > > > 
> > > > * ERROR: cannot start multipathd as localmount would not start
> > > > 
> > > >  * ERROR: cannot start zfs-import as localmount would not start
> > > > 
> > > > This isn't a big concern as these services will start correctly later:
> > > > 
> > > > INIT: Entering runlevel: 3
> > > > 
> > > >  * Starting multipathd ...
> > > >  [ ok ]
> > > >  * Importing ZFS pool(s)  ...
> > > >  [ ok ]
> > > > 
> > > > But I am trying to find the cause of these errors as they are
> > > > preventing
> > > > parallel-start from actually working correctly.
> > > > 
> > > > When I check with "rc-depend", I don't see an obious cause:
> > > > 
> > > > # /lib/rc/bin/rc-depend multipathd
> > > > sysfs devfs udev udev-trigger modules fsck root localmount multipathd
> > > > 
> > > > # /lib/rc/bin/rc-depend localmount
> > > > sysfs devfs udev udev-trigger modules fsck root localmount
> > > > 
> > > > # /lib/rc/bin/rc-depend zfs-import
> > > > multipath sysfs devfs udev udev-trigger modules fsck root localmount
> > > > multipathd zfs-import
> > > > 
> > > > # /lib/rc/bin/rc-depend multipath
> > > > multipath
> > > > 
> > > > From how I read these, it should be able to start "localmount"
> > > > properly
> > > > before even trying to start "multipathd" and "zfs-import"
> > > > There is also no technical dependency for "localmount" (the root
> > > > filesystem
> > > > is not on ZFS on this system)
> > > > 
> > > > Any help/suggestions on how to find the cause would be appreciated.
> > > > 
> > > > --
> > > > Joost
> > > 
> > > Check if hwclock is in the boot runlevel:
> > > 
> > > rc-update -s -v | grep hwclock
> > 
> > What does "hwclock" got to do with this?
> > It has no dependency with multipathd, zfs-import, localmount or anything
> > else that is showing an error.
> > 
> > --
> > Joost
> 
> Our systems are certainly different, but I noticed this dependency on my
> localmount which is missing on yours:
> 
> # /lib/rc/bin/rc-depend localmount
> sysfs devfs udev udev-trigger hwclock modules fsck root dmcrypt localmount
>   ^^^
> Have you compared your system services which has this problem, with other
> systems of yours which can startup properly?

Adding additional dependencies into the tree is more likely to cause further 
issues. I am actually looking for how to quickly find out which dependency is 
causing a circular dependency issue as the first time it thinks it needs to 
start a service it fails. But the 2nd time it starts, it goes correctly.

I removed hwclock from ALL VMs as they don't actually have a hwclock.

I did find out the actual cause of the problem through a lot of trial and 
error, but this is not really useful in actually quickly finding the problem.
Being able to "simulate" the startup sequence for how OpenRC wants to do 
things would have simplified and sped up the entire process.

--
Joost

Re: [gentoo-user] How to find out all openrc dependencies?

[J. Roeleveld]

On Thursday, 11 April 2024 11:35:10 CEST Michael wrote:
> On Thursday, 11 April 2024 06:19:57 BST J. Roeleveld wrote:
> > Hi all,
> > 
> > For a while I've been seeing the following ERROR-messages when booting 1
> > of
> > my systems:
> > 
> > * ERROR: cannot start multipathd as localmount would not start
> > 
> >  * ERROR: cannot start zfs-import as localmount would not start
> > 
> > This isn't a big concern as these services will start correctly later:
> > 
> > INIT: Entering runlevel: 3
> > 
> >  * Starting multipathd ...
> >  [ ok ]
> >  * Importing ZFS pool(s)  ...
> >  [ ok ]
> > 
> > But I am trying to find the cause of these errors as they are preventing
> > parallel-start from actually working correctly.
> > 
> > When I check with "rc-depend", I don't see an obious cause:
> > 
> > # /lib/rc/bin/rc-depend multipathd
> > sysfs devfs udev udev-trigger modules fsck root localmount multipathd
> > 
> > # /lib/rc/bin/rc-depend localmount
> > sysfs devfs udev udev-trigger modules fsck root localmount
> > 
> > # /lib/rc/bin/rc-depend zfs-import
> > multipath sysfs devfs udev udev-trigger modules fsck root localmount
> > multipathd zfs-import
> > 
> > # /lib/rc/bin/rc-depend multipath
> > multipath
> > 
> > From how I read these, it should be able to start "localmount" properly
> > before even trying to start "multipathd" and "zfs-import"
> > There is also no technical dependency for "localmount" (the root
> > filesystem
> > is not on ZFS on this system)
> > 
> > Any help/suggestions on how to find the cause would be appreciated.
> > 
> > --
> > Joost
> 
> Check if hwclock is in the boot runlevel:
> 
> rc-update -s -v | grep hwclock

What does "hwclock" got to do with this?
It has no dependency with multipathd, zfs-import, localmount or anything else 
that is showing an error.

--
Joost

Re: [gentoo-user] acct-user/man usermod: user 'man' does not exist in /etc/passwd

[J. Roeleveld]

On Thursday, 11 April 2024 10:16:52 CEST Dale wrote:
> J. Roeleveld wrote:
> > On Thursday, 11 April 2024 03:23:22 CEST Dale wrote:
> >> Howdy,
> >> 
> >> This failed once before but I didn't worry about it.  However, since the
> >> profile update, it still fails.  I'd like to figure out how to fix it.
> >> I tried doing a emerge -C and then emerging it again.  No help.  This is
> >> the output.  It's not to long, whole thing.  :-D
> >> 
> >>>>> Failed to install acct-user/man-1-r3, Log file:
> >>>>>  '/var/log/portage/acct-user:man-1-r3:20240411-011746.log'
> >>>>> 
> >>>>> Jobs: 0 of 1 complete, 1 failed Load avg: 2.12,
> > 
> > 
> > 
> >>  * FAILED postinst: 1
> >>  *
> >>  * The following package has failed to build, install, or execute
> >>  postinst:
> >>  *
> >>  *  (acct-user/man-1-r3:0/0::gentoo, ebuild scheduled for merge), Log
> >>  file:
> >>  *   '/var/log/portage/acct-user:man-1-r3:20240411-011746.log'
> >>  *
> >>  
> >>  * GNU info directory index is up-to-date.
> >> 
> >> (chroot) root@fireball / #
> >> 
> >> 
> >> 
> >> Any ideas?  It did install once long ago when the group and user thing
> >> started.
> >> 
> >> Ideas??
> > 
> > First idea, if "man" exists, check if it matches current systems.
> > 
> > This is on a system less then 1 month old:
> > 
> > $ id man
> > uid=13(man) gid=15(man) groups=15(man)
> > 
> > --
> > Joost
> 
> Mine says: 
> 
> 
> root@fireball / # id man
> uid=14357(man) gid=0(root) groups=0(root),15(man)
> root@fireball / #
> 
> 
> It doesn't match yours but it has something there.  I'm surprised that
> doing a emerge -C and then emerging it again didn't fix it but I guess
> it adds something to those files but doesn't remove it when
> uninstalled.  So, I did some editing.  The old line, I commented it
> out.  Then it emerged and added the new line. 
> 
> 
> man:x:13:15:System user; man:/dev/null:/sbin/nologin
> #man:!:14357:0:9:7:::
> 
> 
> With it set like that, it emerges.  This is what the output of your
> command looks like now. 
> 
> 
> root@fireball / # id man
> uid=13(man) gid=15(man) groups=15(man)
> root@fireball / #
> 
> 
> Now it matches yours. 
> 
> Is this a bug or something?  I don't tend to mess with that file
> myself.  :/
> 
> Dale
> 
> :-)  :-) 

You might have some files owned by an unknown user on your system now 
(especially man-pages?)

Looking back at the original error message, it did complain about "man" being 
part of the "root" group.
doing a usermod to remove that group would have worked as well.

--
Joost

[gentoo-user] How to find out all openrc dependencies?

[J. Roeleveld]

Hi all,

For a while I've been seeing the following ERROR-messages when booting 1 of my 
systems:

* ERROR: cannot start multipathd as localmount would not start
 * ERROR: cannot start zfs-import as localmount would not start

This isn't a big concern as these services will start correctly later:

INIT: Entering runlevel: 3
 * Starting multipathd ...
 [ ok ]
 * Importing ZFS pool(s)  ...
 [ ok ]

But I am trying to find the cause of these errors as they are preventing 
parallel-start from actually working correctly.

When I check with "rc-depend", I don't see an obious cause:

# /lib/rc/bin/rc-depend multipathd
sysfs devfs udev udev-trigger modules fsck root localmount multipathd

# /lib/rc/bin/rc-depend localmount
sysfs devfs udev udev-trigger modules fsck root localmount

# /lib/rc/bin/rc-depend zfs-import
multipath sysfs devfs udev udev-trigger modules fsck root localmount 
multipathd zfs-import

# /lib/rc/bin/rc-depend multipath
multipath

>From how I read these, it should be able to start "localmount" properly before 
even trying to start "multipathd" and "zfs-import"
There is also no technical dependency for "localmount" (the root filesystem is 
not on ZFS on this system)

Any help/suggestions on how to find the cause would be appreciated.

--
Joost

Re: [gentoo-user] acct-user/man usermod: user 'man' does not exist in /etc/passwd

[J. Roeleveld]

On Thursday, 11 April 2024 03:23:22 CEST Dale wrote:
> Howdy,
> 
> This failed once before but I didn't worry about it.  However, since the
> profile update, it still fails.  I'd like to figure out how to fix it. 
> I tried doing a emerge -C and then emerging it again.  No help.  This is
> the output.  It's not to long, whole thing.  :-D
> 
> >>> Failed to install acct-user/man-1-r3, Log file:
> >>>  '/var/log/portage/acct-user:man-1-r3:20240411-011746.log'
> >>>
> >>> Jobs: 0 of 1 complete, 1 failed Load avg: 2.12,
> 



>  * FAILED postinst: 1
>  *
>  * The following package has failed to build, install, or execute postinst:
>  *
>  *  (acct-user/man-1-r3:0/0::gentoo, ebuild scheduled for merge), Log file:
>  *   '/var/log/portage/acct-user:man-1-r3:20240411-011746.log'
>  *
> 
>  * GNU info directory index is up-to-date.
> (chroot) root@fireball / #
> 
> 
> 
> Any ideas?  It did install once long ago when the group and user thing
> started. 
> 
> Ideas??

First idea, if "man" exists, check if it matches current systems.

This is on a system less then 1 month old:

$ id man
uid=13(man) gid=15(man) groups=15(man)

--
Joost

Re: [gentoo-user] Re: New profiles 23.0

[J. Roeleveld]

On Saturday, 30 March 2024 19:34:42 CEST Walter Dnes wrote:
>   Thanks for the help. I've migrated my 3 operating Gentoo machines;
> main desktop, backup desktop, and an old used Lenovo Thinkpad X201.  The
> poor thing was thrashing away for over 18 hours with 657 packages on the
> emerge --emptytree!!!  And that's after using a homebrew bash script to
> select the max available speed on the CPU.  "time" output...

What does that script do?

> > real1086m47.440s
> > user1732m29.120s
> > sys 146m54.026s
> > 
> > > I got the news item when I ran "emerge --sync".  My understanding is
> > > that step 1 in the news item says "Please also update your system
> > > fully and depclean before proceeding" so I should update world
> > > first.
> > 
> > Yes.  And depclean.
> 
>   I ended up unmerging specific items manually.  Depclean is "rather
> agressive", and wants to remove all but the latest kernel, *EVEN A
> KERNEL THAT I'M CURRENTLY USING*.  I'm currently on 6.1.67...

Unless you plan on recompiling that kernel, there is no need to actually KEEP 
the sources.

> [x8940][waltdnes][~] eselect kernel list
> Available kernel symlink targets:
>   [1]   linux-6.1.57-gentoo
>   [2]   linux-6.1.67-gentoo *
>   [3]   linux-6.6.13-gentoo
>   [4]   linux-6.6.21-gentoo
> 
>   I ran into the Intel integrated graphics problem described in...
> https://discussion.fedoraproject.org/t/f39-kernel-6-6-x-no-video-on-intel-in
> tegrated-graphics/98360
> 
>   His solution...
> 
> > I was filling out the details for a bug report. Under the description,
> > it asked if I have tried rawhide. I installed 6.7.0-0.rc4.35.fc40
> > and it fixed the issue!
> 
>   This appears to be a bug in the 6.6.x kernels, which is fixed in
> 6.7.x.  My 2 desktops and the Thinkpad all have integrated Intel
> graphics, so I'll sit at 6.1.67 until 6.7.x, or higher, goes stable.
> /var/db/repos/gentoo/sys-kernel/gentoo-kernel/gentoo-kernel-6.7.10.ebuild
> is already present, but is keyworded "~amd64".

Re: [gentoo-user] System crash on "Detecting C compiler ABI info"

[J. Roeleveld]

On Tuesday, 2 April 2024 16:11:20 CEST Paul Sopka wrote:
> Hello Gentoo,
> 
> my entire system crashes reliably on "Detecting C compiler ABI info"
> when compiling some packages, happened on sci-libs/netcdf and
> media-libs/svt-av1.
> 
> I am pretty sure it isn't a hardware instability since I can compile
> everything that doesn't run "Detecting C compiler ABI info" without any
> issues at all.
> 
> Does anybody have an idea why this happens?
> 
> I wish you all a nice afternoon!
> 
> Nanderty

Did you upgrade GCC recently?
If yes, did you follow the gcc-upgrade guide:

https://wiki.gentoo.org/wiki/Upgrading_GCC

?

--
Joost

Re: [gentoo-user] some problems moving to 23.0 profile

[J. Roeleveld]

On Tuesday, 2 April 2024 10:14:11 CEST Michael wrote:
> On Tuesday, 2 April 2024 07:03:42 BST J. Roeleveld wrote:
> > On Monday, 1 April 2024 23:46:49 CEST John Covici wrote:
> > > Hi.  Well, I followed the steps in the news item,  to move
> > > todefault/linux/amd64/23.0/desktop/gnome/systemd
> > > 
> > > and it all worked till it wants me to emerge  the whole world file.
> > > Here is what I get:
> > > 
> > > emerge --ask --emptytree @world
> > > 
> > > These are the packages that would be merged, in order:
> > > 
> > > Calculating dependencies   done!
> > > Dependency resolution took 4.58 s (backtrack: 0/200).
> > > 
> > > 
> > > !!! Problems have been detected with your world file
> > > !!! Please run emaint --check world
> > > 
> > > 
> > > !!! Ebuilds for the following packages are either all
> > > !!! masked or don't exist:
> > > www-apps/nextcloud:26.0.10
> > > 
> > > emerge: there are no ebuilds to satisfy
> > > "sys-kernel/gentoo-sources:6.1.69".
> > > (dependency required by "@kernels" [set])
> > > (dependency required by "@selected" [set])
> > > (dependency required by "@world" [argument])
> > > 
> > > I don't want to unmerge that kernel -- its my backup kernel, so I
> > > definitely want to keep it.  I am using the nextcloud they are
> > > complaining about , I will upgrade it soon, but I want to keep it for
> > > now.
> > 
> > Do you actually need to keep the kernel-sources?
> > Once the kernel is compiled and you moved the image to /boot/..., you
> > don't
> > need to keep the sources.
> > 
> > I also keep an older kernel just in case, but I don't tend to actually
> > keep
> > the sources around once I have confirmed the new kernel will boot.
> > 
> > --
> > Joost
> 
> When gentoo-sources are tree-cleaned, it is typically because they have been
> superseded by later kernel patches to improve security and resolve bugs.
> Therefore it is usually a 'good idea' to emerge a later kernel when this
> happens, even if we're talking about a backup kernel.
> 
> Last week I came upon a similar problem on an old system I was trying to
> migrate to profile 23.0, only this happened not with my backup but with the
> running kernel.  This PC had not been updated for 5-6 months.  It's resource
> constrained and I didn't want to spend many days updating most of its
> deprecated packages, only to have to re-emerge them as part of the profile
> migration.  I can't recall if it was the same kernel as John's.  During the
> migration I came across some package (llvm?) which required a more up to
> date kernel to be able to emerge.  This forced me to upgrade the kernel
> first, before I could continue with the migration.  I'm mentioning this
> since the utility of a backup kernel would be limited when you can't use it
> to run your software.

This is my experience as well.
A "backup kernel" is, in my opinion, only useful as a fall-back in case the 
system won't boot with a new kernel.
But, once it booted with the new kernel correctly, there is no reason to 
actually keep the old kernel.

--
Joost

Re: [gentoo-user] some problems moving to 23.0 profile

[J. Roeleveld]

On Tuesday, 2 April 2024 11:17:25 CEST John Covici wrote:
> On Tue, 02 Apr 2024 02:02:08 -0400,
> 
> J. Roeleveld wrote:
> > On Tuesday, 2 April 2024 05:51:08 CEST John Covici wrote:
> > > On Mon, 01 Apr 2024 18:05:47 -0400,
> > > 
> > > Dale wrote:
> > > > John Covici wrote:
> > > > > Hi.  Well, I followed the steps in the news item,  to move
> > > > > todefault/linux/amd64/23.0/desktop/gnome/systemd
> > > > > 
> > > > > and it all worked till it wants me to emerge  the whole world file.
> > > > > Here is what I get:
> > > > > 
> > > > > emerge --ask --emptytree @world
> > > > > 
> > > > > These are the packages that would be merged, in order:
> > > > > 
> > > > > Calculating dependencies   done!
> > > > > Dependency resolution took 4.58 s (backtrack: 0/200).
> > > > > 
> > > > > 
> > > > > !!! Problems have been detected with your world file
> > > > > !!! Please run emaint --check world
> > > > > 
> > > > > 
> > > > > !!! Ebuilds for the following packages are either all
> > > > > !!! masked or don't exist:
> > > > > www-apps/nextcloud:26.0.10
> > > > > 
> > > > > emerge: there are no ebuilds to satisfy
> > > > > "sys-kernel/gentoo-sources:6.1.69".
> > > > > (dependency required by "@kernels" [set])
> > > > > (dependency required by "@selected" [set])
> > > > > (dependency required by "@world" [argument])
> > > > > 
> > > > > I don't want to unmerge that kernel -- its my backup kernel, so I
> > > > > definitely want to keep it.  I am using the nextcloud they are
> > > > > complaining about , I will upgrade it soon, but I want to keep it
> > > > > for
> > > > > now.
> > > > > 
> > > > > So, should I just not do the whole world file at all -- do I really
> > > > > need to do that, or wait till I upgrade nextcloud and till I am no
> > > > > longer using that kernel and then do it?
> > > > > 
> > > > > Thanks in advance for any suggestions.
> > > > 
> > > > I ran into the problem of it complaining about my world file too.
> > > > Running the command it gave didn't show any problems.  I went ahead
> > > > with
> > > > the rest of the change.  After it was all done, that error went away
> > > > on
> > > > its own.  No idea what triggered it or what removed the trigger.  Must
> > > > be something to do with the profile switching process.  You can likely
> > > > ignore that for now.  See if it goes away for you too.
> > > > 
> > > > I don't know what nascloud is but the error says it is masked or not
> > > > there at all.  I'd suspect the mask part since there are several
> > > > versions in the tree.  You may want to check your package.mask file
> > > > and
> > > > see if there is something in there that masks it.  Could be you meant
> > > > to
> > > > add the entry to keyword or unmask file but hit the wrong file.  Did
> > > > that once myself.  One easy way to see if it exists or is masked, use
> > > > this command, provided you have the package for it installed.  I think
> > > > gentools has this command.
> > > > 
> > > > 
> > > > root@fireball / # equery list -p www-apps/nextcloud
> > > > 
> > > >  * Searching for nextcloud in www-apps ...
> > > > 
> > > > [-P-] [  ] www-apps/nextcloud-26.0.8:26.0.8
> > > > [-P-] [ ~] www-apps/nextcloud-26.0.11:26.0.11
> > > > [-P-] [ ~] www-apps/nextcloud-26.0.12:26.0.12
> > > > [-P-] [  ] www-apps/nextcloud-27.1.5:27.1.5
> > > > [-P-] [ ~] www-apps/nextcloud-27.1.6:27.1.6
> > > > [-P-] [ ~] www-apps/nextcloud-27.1.7:27.1.7
> > > > [-P-] [  ] www-apps/nextcloud-28.0.1:28.0.1
> > > > [-P-] [  ] www-apps/nextcloud-28.0.2:28.0.2
> > > > [-P-] [ ~] www-apps/nextcloud-28.0.3:28.0.3
> > > > root@fireball / #
> > > > 
> > > > 
> > > > Yours should look something like that.
> > > > 
> > > > For the kernels, I don't upgrade the kernel as much as I should.  I
> > > > keep
> > > > all versions mas

Re: [gentoo-user] some problems moving to 23.0 profile

[J. Roeleveld]

On Monday, 1 April 2024 23:46:49 CEST John Covici wrote:
> Hi.  Well, I followed the steps in the news item,  to move
> todefault/linux/amd64/23.0/desktop/gnome/systemd
> 
> and it all worked till it wants me to emerge  the whole world file.
> Here is what I get:
> 
> emerge --ask --emptytree @world
> 
> These are the packages that would be merged, in order:
> 
> Calculating dependencies   done!
> Dependency resolution took 4.58 s (backtrack: 0/200).
> 
> 
> !!! Problems have been detected with your world file
> !!! Please run emaint --check world
> 
> 
> !!! Ebuilds for the following packages are either all
> !!! masked or don't exist:
> www-apps/nextcloud:26.0.10
> 
> emerge: there are no ebuilds to satisfy
> "sys-kernel/gentoo-sources:6.1.69".
> (dependency required by "@kernels" [set])
> (dependency required by "@selected" [set])
> (dependency required by "@world" [argument])
> 
> I don't want to unmerge that kernel -- its my backup kernel, so I
> definitely want to keep it.  I am using the nextcloud they are
> complaining about , I will upgrade it soon, but I want to keep it for
> now.

Do you actually need to keep the kernel-sources?
Once the kernel is compiled and you moved the image to /boot/..., you don't 
need to keep the sources.

I also keep an older kernel just in case, but I don't tend to actually keep 
the sources around once I have confirmed the new kernel will boot.

--
Joost

Re: [gentoo-user] some problems moving to 23.0 profile

[J. Roeleveld]

On Tuesday, 2 April 2024 05:51:08 CEST John Covici wrote:
> On Mon, 01 Apr 2024 18:05:47 -0400,
> 
> Dale wrote:
> > John Covici wrote:
> > > Hi.  Well, I followed the steps in the news item,  to move
> > > todefault/linux/amd64/23.0/desktop/gnome/systemd
> > > 
> > > and it all worked till it wants me to emerge  the whole world file.
> > > Here is what I get:
> > > 
> > > emerge --ask --emptytree @world
> > > 
> > > These are the packages that would be merged, in order:
> > > 
> > > Calculating dependencies   done!
> > > Dependency resolution took 4.58 s (backtrack: 0/200).
> > > 
> > > 
> > > !!! Problems have been detected with your world file
> > > !!! Please run emaint --check world
> > > 
> > > 
> > > !!! Ebuilds for the following packages are either all
> > > !!! masked or don't exist:
> > > www-apps/nextcloud:26.0.10
> > > 
> > > emerge: there are no ebuilds to satisfy
> > > "sys-kernel/gentoo-sources:6.1.69".
> > > (dependency required by "@kernels" [set])
> > > (dependency required by "@selected" [set])
> > > (dependency required by "@world" [argument])
> > > 
> > > I don't want to unmerge that kernel -- its my backup kernel, so I
> > > definitely want to keep it.  I am using the nextcloud they are
> > > complaining about , I will upgrade it soon, but I want to keep it for
> > > now.
> > > 
> > > So, should I just not do the whole world file at all -- do I really
> > > need to do that, or wait till I upgrade nextcloud and till I am no
> > > longer using that kernel and then do it?
> > > 
> > > Thanks in advance for any suggestions.
> > 
> > I ran into the problem of it complaining about my world file too. 
> > Running the command it gave didn't show any problems.  I went ahead with
> > the rest of the change.  After it was all done, that error went away on
> > its own.  No idea what triggered it or what removed the trigger.  Must
> > be something to do with the profile switching process.  You can likely
> > ignore that for now.  See if it goes away for you too. 
> > 
> > I don't know what nascloud is but the error says it is masked or not
> > there at all.  I'd suspect the mask part since there are several
> > versions in the tree.  You may want to check your package.mask file and
> > see if there is something in there that masks it.  Could be you meant to
> > add the entry to keyword or unmask file but hit the wrong file.  Did
> > that once myself.  One easy way to see if it exists or is masked, use
> > this command, provided you have the package for it installed.  I think
> > gentools has this command.
> > 
> > 
> > root@fireball / # equery list -p www-apps/nextcloud
> >  * Searching for nextcloud in www-apps ...
> > [-P-] [  ] www-apps/nextcloud-26.0.8:26.0.8
> > [-P-] [ ~] www-apps/nextcloud-26.0.11:26.0.11
> > [-P-] [ ~] www-apps/nextcloud-26.0.12:26.0.12
> > [-P-] [  ] www-apps/nextcloud-27.1.5:27.1.5
> > [-P-] [ ~] www-apps/nextcloud-27.1.6:27.1.6
> > [-P-] [ ~] www-apps/nextcloud-27.1.7:27.1.7
> > [-P-] [  ] www-apps/nextcloud-28.0.1:28.0.1
> > [-P-] [  ] www-apps/nextcloud-28.0.2:28.0.2
> > [-P-] [ ~] www-apps/nextcloud-28.0.3:28.0.3
> > root@fireball / #
> > 
> > 
> > Yours should look something like that. 
> > 
> > For the kernels, I don't upgrade the kernel as much as I should.  I keep
> > all versions masked except the ones I have installed and I add those
> > versions to the world file, that way --depclean and other stuff, won't
> > remove or complain so much about it.  Just emerge -n --select y = > kernel name and version here>.  Don't forget the equal sign when
> > including the version. 
> > 
> > Hope one or more of those things help. 
> 
> My kernels are not in the world file at all, so I am confused why
> portage should care about them when I am updating the world file.  My
> question is why do I  need to do this at all -- could I just keep
> updating as normal?

Actually, based on the output, it is in your world file:

> > > emerge: there are no ebuilds to satisfy
> > > "sys-kernel/gentoo-sources:6.1.69".
> > > (dependency required by "@kernels" [set])
> > > (dependency required by "@selected" [set])
> > > (dependency required by "@world" [argument])

It's referenced in " /var/lib/portage/world_sets "

--
Joost

Re: [gentoo-user] openrc - parallel start - timeouts

[J. Roeleveld]

On Friday, 29 March 2024 15:16:18 CEST Michael wrote:
> On Friday, 29 March 2024 13:30:23 GMT J. Roeleveld wrote:
> > Hi All,
> > 
> > To improve the bootup time of my server, I want to enable "parallel",
> > however, I run into an issue where some of the services take longer than
> > 60
> > seconds to start, causing this to be classed as "not started", which then
> > kills the entire boot sequence.
> > 
> > Boot, obviously, goes fine with parallel off, but I am certain I can
> > reduce
> > the boot time significantly using parallel. I also spent quite some time
> > checking dependencies between the services to ensure they are all present.
> > 
> > Now, how do I configure a longer timeout? Preferably on a per-service
> > basis, but at the very least, globally.
> > 
> > Many thanks in advance,
> > 
> > Joost
> 
> Take a look in /etc/rc.conf, the section titled "SERVICE CONFIGURATION
> VARIABLES".

None of the entries in that entire file have any influence on the timeout.
I did manage to find out a way by looking at the code:
1) The timeout is hardcoded to 60 seconds (which is stupid design)
2) The only option is to disable the timeout for a service entirely and add a 
more sane timeout to the init-script.

--
Joost

[gentoo-user] openrc - parallel start - timeouts

[J. Roeleveld]

Hi All,

To improve the bootup time of my server, I want to enable "parallel", however, 
I run into an issue where some of the services take longer than 60 seconds to 
start, causing this to be classed as "not started", which then kills the 
entire boot sequence.

Boot, obviously, goes fine with parallel off, but I am certain I can reduce the 
boot time significantly using parallel. I also spent quite some time checking 
dependencies between the services to ensure they are all present.

Now, how do I configure a longer timeout? Preferably on a per-service basis, 
but at the very least, globally.

Many thanks in advance,

Joost

Re: [gentoo-user] How to synchronise between 2 locations

[J. Roeleveld]

On Wednesday, 27 March 2024 19:58:47 CET J. Roeleveld wrote:
> Hi all,
> 
> I am looking for a way to synchronise a filesystem between 2 servers.
> Changes can occur on both sides which means I need to have it synchronise
> in both directions.
> 
> Does anyone have any thoughts on this?
> 
> Also, both servers are connected using a slow VPN link, which is why I can't
> simply access files on the remote server.
> 
> --
> Joost

All,

Many thanks to everyone. I will be investigating and testing "net-misc/unison" 
and "net-p2p/syncthing" to see which fits my requirement best. 

Thanks again,

Joost

Re: [gentoo-user] Re: How to synchronise between 2 locations

[J. Roeleveld]

On Thursday, 28 March 2024 14:51:42 CET Grant Edwards wrote:
> On 2024-03-27, Mark Knecht  wrote:
> > On Wed, Mar 27, 2024 at 11:59 AM J. Roeleveld  wrote:
> >> I am looking for a way to synchronise a filesystem between 2
> >> servers.  Changes can occur on both sides which means I need to
> >> have it synchronise in both directions.
> > 
> > How synchronized? For instance, does it need to handle identicals where
> > a file is on both sides but has been moved?
> 
> Does it need to handle the case where the same file is modified
> independently on both sides?

Yes, as this is something that could happen

Re: [gentoo-user] Issue with new hardened profiles 23.0

[J. Roeleveld]

On Thursday, 28 March 2024 12:01:54 CET Michael wrote:
> On Thursday, 28 March 2024 10:23:29 GMT Matthias Hanft wrote:
> > J. Roeleveld wrote:
> > > Do you use the binary packages supplied by Gentoo?
> > > Or all local-compiled?
> > 
> > All local-compiled, with the exemption of "monster-packages" which
> > would take hours or even days to compile (e.g. rust - here I use
> > "dev-lang/rust-bin" instead).
> > 
> > I don't even have any of /etc/portage/binrepos.conf or /var/cache/binpkgs/
> > (and "emerge --getbinpkg ..." displays a warning that it won't work).
> > 
> > -Matt
> 
> You mentioned you have created your custom profile with hardened and desktop
> - could this action have inadvertently mixed merged with split /usr
> profiles in your system?

No, because the server uses hardened and the desktop uses a desktop profile.
These are 2 different systems.

> What does 'tree -L 1 /' show on your server?

After the migration, no symlinks for /bin, /sbin or /lib.

I have just migrated to merge-usr to make sure this particular issue won't 
occur again.

Hope this does warn others using gentoo-provided binary packages that some 
weird issues can happen:
- desktop profile: prevent the use of binaries for "libtool"
- hardened profile: prevent the use of binaries for "libtool" + make symlinks 
for /usr/sbin/openrc* in /sbin/

The symlinks will be handled correctly when doing the usr-merge afterwards.

--
Joost

Re: [gentoo-user] Issue with new hardened profiles 23.0

[J. Roeleveld]

On Thursday, 28 March 2024 11:23:29 CET Matthias Hanft wrote:
> J. Roeleveld wrote:
> > Do you use the binary packages supplied by Gentoo?
> > Or all local-compiled?
> 
> All local-compiled, with the exemption of "monster-packages" which
> would take hours or even days to compile (e.g. rust - here I use
> "dev-lang/rust-bin" instead).
> 
> I don't even have any of /etc/portage/binrepos.conf or /var/cache/binpkgs/
> (and "emerge --getbinpkg ..." displays a warning that it won't work).
> 
> -Matt

Then I assume the issue is caused by the packages Gentoo supplies.
I'll work around it :)

--
Joost

Re: [gentoo-user] Issue with new hardened profiles 23.0

[J. Roeleveld]

On Thursday, 28 March 2024 08:42:57 CET Matthias Hanft wrote:
> J. Roeleveld wrote:
> > When rebooting, I noticed the "openrc" program was moved from
> > "/sbin/openrc" to "/usr/sbin/openrc". I understand this is related to the
> > merge-usr stuff, but I am planning on doing this change later.
> > The profile I selected has the "split-usr" in the name (just as
> > described).
> > 
> > Has anyone else seen this as well?
> 
> Not here.  Moved from
> 
>   [3]   default/linux/amd64/17.1/hardened (exp)
> 
> to
> 
>   [58]  default/linux/amd64/23.0/split-usr/hardened (stable) *
> 
> and openrc still remains in /sbin:
> 
> gentoo64 ~ # which openrc
> /sbin/openrc
> gentoo64 ~ #
> 
> So if your openrc has been moved, there must have been a reason
> for this other than simply changing the profile...

Do you use the binary packages supplied by Gentoo?
Or all local-compiled?

If you don't use them, then that explains it. (As I had to prevent the libtool 
one to be used to avoid issues later with my desktop)

--
Joost

[gentoo-user] Issue with new hardened profiles 23.0

[J. Roeleveld]

Hi all,

After succesfully migrating my desktop to 23.0, I decided to do the same for 
my server.
The only difference is that the server uses a hardened profile.

When rebooting, I noticed the "openrc" program was moved from "/sbin/openrc" 
to "/usr/sbin/openrc". I understand this is related to the merge-usr stuff, but 
I am planning on doing this change later.
The profile I selected has the "split-usr" in the name (just as described).

Has anyone else seen this as well?

Thanks,

Joost

Re: [gentoo-user] How to synchronise between 2 locations

[J. Roeleveld]

On Wednesday, 27 March 2024 20:37:27 CET Frank Steinmetzger wrote:
> Am Wed, Mar 27, 2024 at 08:18:14PM +0100 schrieb ralfconn:
> > Il 27/03/24 19:58, J. Roeleveld ha scritto:
> > > Hi all,
> > > 
> > > I am looking for a way to synchronise a filesystem between 2 servers.
> > > Changes can occur on both sides which means I need to have it
> > > synchronise in both directions.
> > > 
> > > Does anyone have any thoughts on this?
> > > 
> > > Also, both servers are connected using a slow VPN link, which is why I
> > > can't simply access files on the remote server.
> 
> +1 for Unison. I’ve been using it for many years now to synchronise between
> the four PC systems in my household.
> 
> > I use it just for that but can't say anything about the VPN bit, my
> > servers are on local network.
> 
> Unison creates a local index of all files it syncronised. So when you move a
> file around on one end, Unison will notice that because the file at the new
> location has the same hash as the file at the old location. As a result, it
> does not transmit the file anew to the remote host, but instead copies it
> locally on the remote host.
> 
> Since Unison uses ssh underneath, you can use ssh’s transparent compression
> to speed up the transfer.

Unison sounds interesting. How does it handle conflicts (eg, file is changed on 
both sides?)

--
Joost

signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-user] How to synchronise between 2 locations

[J. Roeleveld]

On Wednesday, 27 March 2024 20:54:14 CET Frank Steinmetzger wrote:
> Am Wed, Mar 27, 2024 at 03:42:07PM -0400 schrieb Matt Connell:
> > On Wed, 2024-03-27 at 19:58 +0100, J. Roeleveld wrote:
> > > Hi all,
> > > 
> > > I am looking for a way to synchronise a filesystem between 2 servers.
> > > Changes can occur on both sides which means I need to have it
> > > synchronise in both directions.
> > > 
> > > Does anyone have any thoughts on this?
> > > 
> > > Also, both servers are connected using a slow VPN link, which is why
> > > I can't simply access files on the remote server.
> > 
> > I've been using syncthing for years and am extremely pleased with it.
> > It works so well that I sometimes forget that its there, truly in the
> > It Just Works category of software.
> 
> Syncthing is also a good idea. The major difference: syncthing is a
> permanently running daemon, so changes are synced very fast (the interval is
> configurable, IIRC). OTOH, Unison is run individually by you. That’s why I
> prefer the latter: in case I broke some file on my machine, I can get it
> back from another machine without having to break out the backup disk
> (which may not even have what I need because my backup interval is too
> big).

I had a quick look and it seems to depend on systems running outside of my 
network. Can it be used without any link to a "centralised" server?

--
Joost


signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-user] How to synchronise between 2 locations

[J. Roeleveld]

On Wednesday, 27 March 2024 20:08:00 CET Mark Knecht wrote:
> On Wed, Mar 27, 2024 at 11:59 AM J. Roeleveld  wrote:
> > Hi all,
> > 
> > I am looking for a way to synchronise a filesystem between 2 servers.
> 
> Changes
> 
> > can occur on both sides which means I need to have it synchronise in both
> > directions.
> > 
> > Does anyone have any thoughts on this?
> > 
> > Also, both servers are connected using a slow VPN link, which is why I
> 
> can't
> 
> > simply access files on the remote server.
> > 
> > --
> > Joost
> 
> How synchronized? For instance, does it need to handle identicals where
> a file is on both sides but has been moved?

Yes, it needs to handle changes to files/movements/deletions that occur on both 
sides. Most of the time, there will only be changes on 1 side, but this is not 
guaranteed.
I also need it to identify conflicts and handle this in a way that is 
managable. (creating a copy with a different name would be sufficient)

--
Joost

signature.asc
Description: This is a digitally signed message part.

[gentoo-user] How to synchronise between 2 locations

[J. Roeleveld]

Hi all,

I am looking for a way to synchronise a filesystem between 2 servers. Changes 
can occur on both sides which means I need to have it synchronise in both 
directions.

Does anyone have any thoughts on this?

Also, both servers are connected using a slow VPN link, which is why I can't 
simply access files on the remote server.

--
Joost

Re: [gentoo-user] Help with local mail, please

[J. Roeleveld]

On Friday, February 16, 2024 6:19:25 AM CET Peter Humphrey wrote:
> On Wednesday, 14 February 2024 11:35:18 GMT J. Roeleveld wrote:
> > I've been using postfix for longer than I can remember.
> > The config entries I changed from default are:
> > 
> > --- main.cf ---
> > myhostname = 
> > mydomain = 
> > myorigin = 
> > mynetworks =  > 192.168.1.0/24>
> 
> That's helpful - thanks Joost.

You're welcome. Was this enough to get it working?

> > smtpd_relay_restrictions = permit_mynetworks, permit_sasl_authenticated,
> > reject_unauth_destination
> 
> No relaying here; all outgoing mail goes to my ISP.

If your mailserver (postfix) is sending emails to your ISP, that is (from the 
viewpoint of postfix) relaying by definition.

> > Also, are all emails sent to "mydomain" or do you have additional domains
> > configured?
> > If the latter, did you set "virtual_alias_domains = hash:/etc/postfix/
> > virtual_domains"
> > Include all virtual domains the file and converted the file to a .db ?
> 
> Just the one "mydomain"

If you have only 1 domain, keep that "virtual_alias_domains" empty.

--
Joost

Re: [gentoo-user] Help with local mail, please

[J. Roeleveld]

On Tuesday, February 13, 2024 4:52:03 PM CET Peter Humphrey wrote:
> Hello list,
> 
> For years, I've been using postfix to accept mail from LAN hosts, and from
> the Internet via my ISP. This has never worked as I want it - it's just so
> complex to set up and understand. Well, it is for a bear of little brain
> like me.
> 
> Can someone tell me how to make postfix accept all mail addressed to any
> host or user on the LAN - and not forward any mail to anywhere at all? It's
> running on a single-homed host on the LAN, and all other hosts are also
> single homed. Any of four hosts can originate mail, and I have fetchmail
> running on the same host to collect POP3 mail from my ISP. Dovecot serves
> IMAP4 to KMail clients on the LAN.
> 
> At present, postfix is insisting on forwarding mail addressed to root on a
> LAN machine, but it's supposed to be acting on behalf of that machine. Two
> other hosts' mails never show up anywhere.
> 
> Or perhaps there's a more suitable MTA out there?

I've been using postfix for longer than I can remember.
The config entries I changed from default are:

--- main.cf ---
myhostname = 
mydomain = 
myorigin = 
mynetworks = 
smtpd_relay_restrictions = permit_mynetworks, permit_sasl_authenticated, 
reject_unauth_destination
---

Also, are all emails sent to "mydomain" or do you have additional domains 
configured?
If the latter, did you set "virtual_alias_domains = hash:/etc/postfix/
virtual_domains"
Include all virtual domains the file and converted the file to a .db ?

--
Joost

Re: [gentoo-user] Re: Suggestions for backup scheme?

[J. Roeleveld]

On Thursday, February 8, 2024 6:44:50 PM CET Wols Lists wrote:
> On 08/02/2024 06:38, J. Roeleveld wrote:
> > ZFS doesn't have this "max amount of changes", but will happily fill up
> > the
> > entire pool keeping all versions available.
> > But it was easier to add zpool monitoring for this on ZFS then it was to
> > add snapshot monitoring to LVM.
> > 
> > I wonder, how do you deal with snapshots getting "full" on your system?
> 
> As far as I'm, concerned, snapshots are read-only once they're created.
> But there is a "grow the snapshot as required" option.
> 
> I don't understand it exactly, but what I think happens is when I create
> the snapshot it allocates, let's say, 1GB. As I write to the master
> copy, it fills up that 1GB with CoW blocks, and the original blocks are
> handed over to the backup snapshot. And when that backup snapshot is
> full of blocks that have been "overwritten" (or in reality replaced),
> lvm just adds another 1GB or whatever I told it to.

That works with a single snapshot.
But, when I last used LVM like this, I would have multiple snapshots. When I 
change something on the LV, the original data would be copied to the snapshot.
If I would have 2 snapshots for that LV, both would grow at the same time.

Or is that changed in recent versions?

> So when I delete a snapshot, it just goes through those few blocks,
> decrements their use count (if they've been used in multiple snapshots),
> and if the use count goes to zero they're handed back to the "empty" pool.

I know this is how ZFS snapshots work. But am not convinced LVM snapshots work 
the same way.

> All I have to do is make sure that the sum of my snapshots does not fill
> the lv (logical volume). Which in my case is a raid-5.

I assume you mean PV (Physical Volume)?

I actually ditched the whole idea of raid-5 when drives got bigger than 1TB. I 
currently use Raid-6 (or specifically RaidZ2, which is the ZFS "equivalent")

--
Joost

Re: [gentoo-user] Re: Suggestions for backup scheme?

[J. Roeleveld]

On Thursday, February 8, 2024 6:36:56 PM CET Wols Lists wrote:
> On 08/02/2024 06:32, J. Roeleveld wrote:
> >> After all, there's nothing stopping*you*  from combining Linux and ZFS,
> >> it's just that somebody else can't do that for you, and then give you
> >> the resulting binary.
> > 
> > Linux (kernel) and ZFS can't be merged. Fine.
> 
> But they can.

Not if you want to release it

> > But, Linux (the OS, as in, kernel + userspace) and ZFS can be merged
> > legally.
> Likewise here, they can.
> 
> The problem is, the BINARY can NOT be distributed. And the problem is
> the ZFS licence, not Linux.

You can distribute the binary of both, just not embedded into a single binary.

> What Linus, and the kernel devs, and that crowd *think* is irrelevant.

It is, as they are actively working on removing API calls that filesystems like 
ZFS actually need and hiding them behind a GPL wall.

> What matters is what SUSE, and Red Hat, and Canonical et al think. And
> if they're not prepared to take the risk of distributing the kernel with
> ZFS built in, because they think it's a legal minefield, then that's
> THEIR decision.

I'm not talking about distributing ZFS embedded into the kernel. It's 
perfectly fine to distribute a distribution with ZFS as a kernel module. The 
issue is caused by the linux kernel devs blocking access to (previously 
existing and open) API calls and limiting them to GPL only.

> That problem doesn't apply to gentoo, because it distributes the linux
> kernel and ZFS separately, and combines them ON THE USER'S MACHINE. But
> the big distros are not prepared to take the risk of combining linux and
> ZFS, and distributing the resulting *derived* *work*.

I would class Ubuntu as a big distribution and proxmox is also used a lot.
Both have ZFS support.

--
Joost

Re: [gentoo-user] Re: Suggestions for backup scheme?

[J. Roeleveld]

On Wednesday, February 7, 2024 10:50:07 PM CET Wols Lists wrote:
> On 07/02/2024 11:07, J. Roeleveld wrote:
> >> Because snapshotting uses so much less space?
> >> 
> >> So much so that, for normal usage, I probably have no need to delete any
> >> snapshots, for YEARS?
> > 
> > My comment was based on using rsync to copy from the source to the backup
> > filesystem.
> 
> Well, that's EXACTLY what I'm doing too. NO DIFFERENCE. Actually, there
> is a minor difference - because I'm using lvm, I'm also using rsync's
> "overwrite in place" switch. In other words, it compares source and
> destination *in*place*, and if any block has changed, it overwrites the
> change, rather than creating a complete new copy.

I must have missed that in the man-page last time I used rsync. Will have to 
recheck and update my notes just in case I need to use rsync again in the 
future.

> Because lvm is COW, that means I have two copies of the file, in two
> different snapshots, but inasmuch as the files are identical, there's
> only one copy of the identical bits.
> 
> >> Okay, space is not an expensive commodity, and you don't want too many
> >> snapshots, simply because digging through all those snapshots would be a
> >> nightmare, but personally I wouldn't use a crude rsync simply because I
> >> prefer to be frugal in my use of resources.
> > 
> > What is "too many"?
> > I currently have about 1800 snapshots on my server. Do have a tool that
> > ensures it doesn't get out of hand and will remove several over time.
> 
> "Too many" is whatever you define it to be. I'm likely to hang on to my
> /home snapshots for yonks. My / snapshots, on the other hand, I delete
> anything more than a couple of months old.
> 
> If I can store several years of /home snapshots without running out of
> space, why shouldn't I? The problem, if I *am* running out of space, I'm
> going to have to delete a *lot* of snapshots to make much difference...

One of the things I didn't like about LVM was that it would have trouble 
dealing with a lot (100+, due to a bug in my script at the time) of snapshots. 
And having to manually (or using a script) increase the size given to these 
snapshots when a lot of changes are occuring.

ZFS doesn't have this "max amount of changes", but will happily fill up the 
entire pool keeping all versions available.
But it was easier to add zpool monitoring for this on ZFS then it was to add 
snapshot monitoring to LVM.

I wonder, how do you deal with snapshots getting "full" on your system?

--
Joost

Re: [gentoo-user] Re: Suggestions for backup scheme?

[J. Roeleveld]

On Wednesday, February 7, 2024 10:59:38 PM CET Wols Lists wrote:
> On 07/02/2024 11:11, J. Roeleveld wrote:
> > On Tuesday, February 6, 2024 9:27:35 PM CET Wols Lists wrote:
> >> On 06/02/2024 13:12, J. Roeleveld wrote:
> >>>> Clearly Oracle likes this state of affairs.  Either that, or they are
> >>>> encumbered in some way from just GPLing the ZFS code.  Since they on
> >>>> paper own the code for both projects it seems crazy to me that this
> >>>> situation persists.
> >>> 
> >>> GPL is not necessarily the best license for releasing code. I've got
> >>> some
> >>> private projects that I could publish. But before I do that, I'd have to
> >>> decide on a License. I would prefer something other than GPL.
> >> 
> >> Okay. What do you want to achieve. Let's just lump licences into two
> >> categories to start with and ask the question "Who do you want to free?"
> > 
> > I want my code to be usable by anyone, but don't want anyone to fork it
> > and
> > start making money off of it without giving me a fair share.
> 
> Okay, that instantly says you want a copyleft licence. So you're stuck
> with a GPL-style licence, and if they want to include it in a commercial
> closed source product, they need to come back to you and dual licence it.
> 
> Personally, I'd go the MPL2 route, but that's my choice. It might not
> suit you. But to achieve what you want, you need a copyleft, GPL-style
> licence.

I'll have a look at that one.

> >> If that sounds weird, it's because both Copyleft and Permissive claim to
> >> be free, but have completely different target audiences. Once you've
> >> answered that question, it'll make choosing a licence so much easier.
> >> 
> >> GPL gives freedom to the END USER. It's intended to protect the users of
> >> your program from being held to ransom.
> > 
> > That's not how the kernel devs handle the GPL. They use it to remove
> > choice
> > from the end user (me) to use what I want (ZFS).
> > And it's that which I don't like about the GPL.
> 
> No. That's Oracle's fault. The kernel devs can't include ZFS in linux,
> because Oracle (or rather Sun, at the time, I believe) deliberately
> *designed* the ZFS licence to be incompatible with the GPL.

Maybe not included fully into the kernel, but there is nothing preventing it 
to be packaged with a Linux distribution.
It's just the hostility from Linus Torvalds and Greg Kroah-Hartman against ZFS 
causing the issues.

See the following post for a clear description (much better written than I 
can):
https://eerielinux.wordpress.com/2019/01/28/zfs-and-gpl-terror-how-much-freedom-is-there-in-linux/

Especially the lkml thread linked from there:
https://lore.kernel.org/lkml/20190110182413.ga6...@kroah.com/

> After all, there's nothing stopping *you* from combining Linux and ZFS,
> it's just that somebody else can't do that for you, and then give you
> the resulting binary.

Linux (kernel) and ZFS can't be merged. Fine.
But, Linux (the OS, as in, kernel + userspace) and ZFS can be merged legally.

> At the end of the day, if someone wants to be an arsehole, there's not a
> lot you can do to stop them, and with ZFS that honour apparently goes to
> Sun.

See what I put above.

--
Joost

Re: [gentoo-user] Re: Suggestions for backup scheme?

[J. Roeleveld]

On Tuesday, February 6, 2024 6:22:34 PM CET Grant Edwards wrote:
> On 2024-02-06, J. Roeleveld  wrote:
> > On Tuesday, February 6, 2024 4:38:11 PM CET Grant Edwards wrote:

> >> I presume that boot/root on ext4 and home on ZFS would not require an
> >> initrd?
> > 
> > Yes, that wouldn't require an initrd. But why would you limit this?
> 
> Because I really, really dislike having to use an initrd. That's
> probably just an irrational 30 year old prejudice, but over the
> decades I've found live to be far simpler and more pleasant without
> initrds. Maybe things have improved over the years, but way back when
> I did use distros that required initrds, they seem to be a constant,
> nagging source of headaches.

In the past, initrd's were a nightmare. Even the current tools (dracut, 
genkernel) are a pain and force the user to do it their way.
The only initramfs generator I use is the "bliss-initramfs" one and that is 
because it actually works and doesn't get in the way.
And I don't build a new kernel for the server.

For my desktops and laptops, I embed the initramfs into the kernel using a 
very simple set of files (script with the commands and a config detailing which 
files to include)
the total size of both files is about 8K and was mostly grabbed from a howto 
page about 10 years ago and has stayed unchanged since then. (I added a little 
script to update the config when library versions change, but that is it)

> > ZFS works best when given the FULL drive.
> 
> Where do you put swap?

My swap is a ZFS volume. I find using the recommended method of configuring it 
is safe and I have not seen any kind of lockup due to swap.
Did have some due to a bug in the HBA-driver when some deranged dev decided to 
change sensible defaults though. But it would freeze before even getting to 
enabling swap.

> > For my server, I use "bliss-initramfs" to generate the initramfs and
> > have not had any issues with this since I started using ZFS.
> > 
> > Especially the ease of generating snapshots also make it really easy
> > to roll back an update if anything went wrong. If your
> > root-partition isn't on ZFS, you can't easily roll back.
> 
> True. However, I've never adopted the practice of backing up my root
> fs (except for a few specific directories like /etc), and haven't ever
> really run into situations where I wished I had. It's all stuff that
> can easily be reinstalled.

I did start backup up the full system as restoring from backup (especially 
rolling back a snapshot, but same is true when grabbing the backup from tape) 
is a lot faster than reinstalling all the software and making sure the config 
(which these days isn't just in /etc anymore) is still the same.

--
Joost

Re: [gentoo-user] Re: Suggestions for backup scheme?

[J. Roeleveld]

On Wednesday, February 7, 2024 12:17:03 AM CET Wols Lists wrote:
> On 06/02/2024 16:19, J. Roeleveld wrote:
> >> Ah! Got it. That's one of the things I've been trying to figure out
> >> this entire thread, do I need to switch home and root to ZFS to take
> >> advantage of its snapshot support for backups? In the case you're
> >> describing the "source" filesystem(s) can be anything. It's only the
> >> _backup_  filesystem that needs to be ZFS (or similar).
> > 
> > If you want to use snapshots, the filesystem will need to support it.
> > (either LVM or ZFS). If you only want to create snapshots on the
> > backupserver, I actually don't see much benefit over using rsync.
> 
> Because snapshotting uses so much less space?
> 
> So much so that, for normal usage, I probably have no need to delete any
> snapshots, for YEARS?

My comment was based on using rsync to copy from the source to the backup 
filesystem.

> Okay, space is not an expensive commodity, and you don't want too many
> snapshots, simply because digging through all those snapshots would be a
> nightmare, but personally I wouldn't use a crude rsync simply because I
> prefer to be frugal in my use of resources.

What is "too many"?
I currently have about 1800 snapshots on my server. Do have a tool that 
ensures it doesn't get out of hand and will remove several over time.

--
Joost

Re: [gentoo-user] Re: Suggestions for backup scheme?

[J. Roeleveld]

On Tuesday, February 6, 2024 9:27:35 PM CET Wols Lists wrote:
> On 06/02/2024 13:12, J. Roeleveld wrote:
> >> Clearly Oracle likes this state of affairs.  Either that, or they are
> >> encumbered in some way from just GPLing the ZFS code.  Since they on
> >> paper own the code for both projects it seems crazy to me that this
> >> situation persists.
> > 
> > GPL is not necessarily the best license for releasing code. I've got some
> > private projects that I could publish. But before I do that, I'd have to
> > decide on a License. I would prefer something other than GPL.
> 
> Okay. What do you want to achieve. Let's just lump licences into two
> categories to start with and ask the question "Who do you want to free?"

I want my code to be usable by anyone, but don't want anyone to fork it and 
start making money off of it without giving me a fair share.

> If that sounds weird, it's because both Copyleft and Permissive claim to
> be free, but have completely different target audiences. Once you've
> answered that question, it'll make choosing a licence so much easier.
> 
> GPL gives freedom to the END USER. It's intended to protect the users of
> your program from being held to ransom.

That's not how the kernel devs handle the GPL. They use it to remove choice 
from the end user (me) to use what I want (ZFS).
And it's that which I don't like about the GPL.

> Permissive gives freedom to the DEVELOPER. It's intended to let other
> programmers take advantage of your code and use it.
> 
> Once you've decided what sort of licence you want, it'll be easier to
> decide what licence you want.

See above

--
Joost

Re: [gentoo-user] Re: Suggestions for backup scheme?

[J. Roeleveld]

On Tuesday, February 6, 2024 6:29:09 PM CET Grant Edwards wrote:
> On 2024-02-06, J. Roeleveld  wrote:
> > If you want to use snapshots, the filesystem will need to support it.
> > (either LVM or ZFS). If you only want to create snapshots on the
> > backupserver, I actually don't see much benefit over using rsync.
> 
> Upthread I've been told that ZFS snapshots
> 
>  1. Require far less disk space than rsync's snapshots.
>  2. Are far faster.
>  3. Are atomic.

True, but the speed is reduced by relying on rsync to copy data from your PC 
to the backupserver.

> > They are normally not visible though, you need to access them specifically
> > using "/filesystem/path/.zfs/snapshot"
> 
> Great, that's exactly what I would hope for. I'm reading up on ZFS,
> and from what I've gleaned so far, it seems lake ZFS source and ZFS
> backup certainly would be ideal.
> 
> It's almost like the ZFS filesystem designers had thought about "how
> to backup" from the start. Something that all of the old-school
> filesystem designers clearly hadn't. :)

I think it's also mainly there to quickly keep a backup server on standby for 
a quick switch over.

--
Joost

Re: [gentoo-user] Re: Suggestions for backup scheme?

[J. Roeleveld]

On Tuesday, February 6, 2024 4:35:34 PM CET Grant Edwards wrote:
> On 2024-02-05, Wols Lists  wrote:
> > On 04/02/2024 15:48, Grant Edwards wrote:
> >> OK I see. That's a bit different than what I'm doing.  I'm backing up
> >> a specific set of directory trees from a couple different
> >> filesystems. There are large portions of the "source" filesystems that
> >> I have no need to back up.  And within those directory trees that do
> >> get backed up there are also some excluded subtrees.
> > 
> > But my scheme still works here. The filesystem I'm snapshotting is the
> > backup. As such, it only contains the stuff I want backed up, copied
> > across using rsync.
> > 
> > There's nothing stopping me running several rsyncs from the live system,
> > from several different partitions, to the backup partition.
> 
> Ah! Got it. That's one of the things I've been trying to figure out
> this entire thread, do I need to switch home and root to ZFS to take
> advantage of its snapshot support for backups? In the case you're
> describing the "source" filesystem(s) can be anything. It's only the
> _backup_ filesystem that needs to be ZFS (or similar).

If you want to use snapshots, the filesystem will need to support it. (either 
LVM or ZFS). If you only want to create snapshots on the backupserver, I 
actually don't see much benefit over using rsync.

> If (like rsnapshot/rsync's hard-link scheme) ZFS snapshots are normal
> directory trees that can be "browsed" with normal filesystem tools,
> that would be ideal. [I'll do some googling...]

ZFS snapshots can be accessed using normal tools and can even be exposed over 
NFS mounts making it super easy to find the files again.

They are normally not visible though, you need to access them specifically 
using "/filesystem/path/.zfs/snapshot"

--
Joost

Re: [gentoo-user] Re: Suggestions for backup scheme?

[J. Roeleveld]

On Tuesday, February 6, 2024 4:38:11 PM CET Grant Edwards wrote:
> On 2024-02-05, J. Roeleveld  wrote:
> > On Wednesday, January 31, 2024 6:56:47 PM CET Rich Freeman wrote:
> >> On Wed, Jan 31, 2024 at 12:40 PM Thelma  wrote:
> >> > If zfs file system is superior to ext4 and it seems to it is.
> >> > Why hasn't it been adopted more widely in Linux?
> >> 
> >> The main barrier is that its license isn't GPL-compatible.  It is
> >> FOSS, but the license was basically designed to keep it from being
> >> incorporated into the mainline kernel.
> > 
> > Which isn't as much of an issue as it sounds. You can still add it
> > into the initramfs and can easily load the module.
> 
> What if you don't use an initrd?
> 
> I presume that boot/root on ext4 and home on ZFS would not require an
> initrd?

Yes, that wouldn't require an initrd. But why would you limit this?
ZFS works best when given the FULL drive.

For my server, I use "bliss-initramfs" to generate the initramfs and have not 
had any issues with this since I started using ZFS.

Especially the ease of generating snapshots also make it really easy to roll 
back an update if anything went wrong. If your root-partition isn't on ZFS, 
you can't easily roll back.

--
Joost

Re: [gentoo-user] Re: Suggestions for backup scheme?

[J. Roeleveld]

On Monday, February 5, 2024 2:35:12 PM CET Rich Freeman wrote:
> First, thanks for the Ars link in the other email.  I'll give that a read.

You're welcome. I found that when I was looking for the latest state of btrfs. 
I was actually hoping that the biggest issues had been resolved by now.

> On Mon, Feb 5, 2024 at 7:55 AM J. Roeleveld  wrote:
> > On Wednesday, January 31, 2024 6:56:47 PM CET Rich Freeman wrote:
> > > The main barrier is that its license isn't GPL-compatible.  It is
> > > FOSS, but the license was basically designed to keep it from being
> > > incorporated into the mainline kernel.
> > 
> > Which isn't as much of an issue as it sounds. You can still add it into
> > the
> > initramfs and can easily load the module.
> > And the code still works with the functions the kernel devs pushed behind
> > the GPL-wall if you simply remove that wall from your own kernel.
> > (Which is advisable as it will improve performance)
> 
> So, that's great for random individuals, but companies are going to be
> hesitant to do that, especially for anything they redistribute.  This
> is part of why it isn't mainstream.

Not for Linux. *BSD has no such issues and that is why the mainstream SAN/NAS 
distributions are based on *BSD. (replace '*' with your preferred flavour)

> A big part of the reason that Linux is mainstream is that it doesn't
> have any legal/license encumbrances.  If you have 100 instances of
> something and want to have 200 instances, you just turn a dial or add
> hardware.  There isn't anybody you need to get permission from or pay.
> 


> The result is that the murky legal situation makes ZFS unattractive.
> If I were publishing some large commercial software package, I'd
> personally be hesitant to embrace ZFS on Linux in it for that reason,
> even though I use it all the time personally.

Proxmox has ZFS native and afaik, it is using Linux?

> > > The odd thing is that right now Oracle controls both ZFS and btrfs,
> > > with the latter doing mostly the same thing and being GPL-compatible,
> > > but it hasn't tended to be as stable.  So we're in a really long
> > > transitional period to btrfs becoming as reliable.
> > 
> > After all this time, I have given up on waiting for btrfs. As mentioned in
> > my other reply, it's still nowhere near reliable.
> 
> Clearly Oracle likes this state of affairs.  Either that, or they are
> encumbered in some way from just GPLing the ZFS code.  Since they on
> paper own the code for both projects it seems crazy to me that this
> situation persists.

GPL is not necessarily the best license for releasing code. I've got some 
private projects that I could publish. But before I do that, I'd have to 
decide on a License. I would prefer something other than GPL.

> > To make this easier, there is a compatiblity option when creating a new
> > zpool. It's also listed in the zfs-kmod ebuild:
> > - zpool create -o compatibility=*grub*2 ...
> > - Refer to /usr/share/zfs/compatibility.d/*grub*2 for list of features.
> 
> Oh, that is VERY helpful.  I've found random many-years-old webpages
> with the appropriate instructions, but something that is part of the
> maintained project is much more useful.
> 
> Granted, I think the bottom line is that boot probably shouldn't be on
> the same filesystem as large volumes of data, as these feature
> restrictions are going to be cumbersome.  I'm guessing you can't
> shrink vdevs, for example.

I actually have the kernel and initramfs on a EFI boot partition and that is 
enough to get the zpool mounted for use.

There is also "ZFSBootMenu" which, afaik, doesn't need this:

https://docs.zfsbootmenu.org/en/latest/index.html

--
Joost

Re: [gentoo-user] Re: Suggestions for backup scheme?

[J. Roeleveld]

On Wednesday, January 31, 2024 6:56:47 PM CET Rich Freeman wrote:
> On Wed, Jan 31, 2024 at 12:40 PM Thelma  wrote:
> > If zfs file system is superior to ext4 and it seems to it is.
> > Why hasn't it been adopted more widely in Linux?
> 
> The main barrier is that its license isn't GPL-compatible.  It is
> FOSS, but the license was basically designed to keep it from being
> incorporated into the mainline kernel.

Which isn't as much of an issue as it sounds. You can still add it into the 
initramfs and can easily load the module.
And the code still works with the functions the kernel devs pushed behind the 
GPL-wall if you simply remove that wall from your own kernel.
(Which is advisable as it will improve performance)

> The odd thing is that right now Oracle controls both ZFS and btrfs,
> with the latter doing mostly the same thing and being GPL-compatible,
> but it hasn't tended to be as stable.  So we're in a really long
> transitional period to btrfs becoming as reliable.

After all this time, I have given up on waiting for btrfs. As mentioned in my 
other reply, it's still nowhere near reliable.

> ZFS also cannot be shrunk as easily.  I think that is something that
> has been improved more recently, but I'm not certain of the state of
> it.  Also, bootloaders like grub aren't 100% compatible with all of
> its later features, and it isn't even clear in the docs which ones are
> and aren't supported.  So it doesn't hurt to keep /boot off of zfs.

To make this easier, there is a compatiblity option when creating a new zpool. 
It's also listed in the zfs-kmod ebuild:
- zpool create -o compatibility=*grub*2 ...
- Refer to /usr/share/zfs/compatibility.d/*grub*2 for list of features.

--
Joost

Re: [gentoo-user] Re: Suggestions for backup scheme?

[J. Roeleveld]

On Wednesday, January 31, 2024 2:01:32 PM CET Rich Freeman wrote:
> On Wed, Jan 31, 2024 at 6:45 AM John Covici  wrote:
> > I know you said you wanted to stay with ext4, but going to zfs reduced
> > my backup time on my entire system from several hours to just a few
> > minutes because taking a snapshot is so quick and copying to another
> > pool is also very quick.
> 
> Honestly, at this point I would not run any storage I cared about on
> anything but zfs.  There are just so many benefits.
> 
> I'd consider btrfs, but I'd have to dig into whether the reliability
> issues have been solved. I was using that for a while, but I found
> that even features that were touted as reliable had problems from time
> to time.  That was years ago, however.  On paper I think it is the
> better option, but I just need to confirm whether I can trust it.

I actually looked into the state of btrfs last week and it's still far from 
usable and not even 
close to what ZFS offers.

For a good read:
https://arstechnica.com/gadgets/2021/09/examining-btrfs-linuxs-perpetually-half-finished-filesystem/[1]

In short:
- raid5/6/.. are still broken.
- Missing drive prevent boot unless you tell it to accept a missing drive.
- Replacing a broken drive requires a lot of steps to make it sane again

--
Joost


[1] 
https://arstechnica.com/gadgets/2021/09/examining-btrfs-linuxs-perpetually-half-finished-filesystem/

Re: [gentoo-user] amavis/postfix and port 10025

[J. Roeleveld]

On Monday, July 3, 2023 7:23:12 AM CEST William Kenworthy wrote:
> Inline:
> 
> On 3/7/23 12:52, J. Roeleveld wrote:
> > On Sunday, July 2, 2023 4:16:54 AM CEST William Kenworthy wrote:
> >> Hi all,
> >> 
> >>  I have been  using a gentoo mail gateway for many years - its
> >>  currently
> >> 
> >> running under LXC and is upgraded using a generic LXC "golden master"
> >> image
> >> with the various email related packages being installed and config files
> >> copied across roughly a month or two apart.  This is always a trial,
> >> particularly with permissions and has become much worse with gentoo's
> >> attempt at using the acct packages to manage user and group ID's.
> > 
> > I actually find this easier to solve issues. What do you find difficult
> > here?
> Trying to interpret an error message that says "it cant connect" with no
> detail as to why when started via the openrc service script - but it
> works fine when started as the amavis user in debug mode.
> 
> If I try and run it in debug mode from root it produces lots of perl
> errors that do not occur with either the openrc service script or amavis
> user:
> 
> fetch_modules: error loading optional module Razor2/Client/Agent.pm:
>Can't locate Getopt/Long.pm:   lib/Getopt/Long.pm: Permission denied
> at
> /usr/lib64/perl5/vendor_perl/5.36/aarch64-linux-thread-multi/Razor2/Client/A
> gent.pm line 15.
>BEGIN failed--compilation aborted at
> /usr/lib64/perl5/vendor_perl/5.36/aarch64-linux-thread-multi/Razor2/Client/A
> gent.pm line 15.
>Compilation failed in require at /usr/sbin/amavisd line 212.
> fetch_modules: error loading optional module Mail/DKIM.pm:
>Can't locate Mail/DKIM.pm:   lib/Mail/DKIM.pm: Permission denied at
> /usr/sbin/amavisd line 212.
> fetch_modules: error loading optional module Mail/DKIM/Verifier.pm:
>Can't locate Mail/DKIM/Verifier.pm:   lib/Mail/DKIM/Verifier.pm:
> Permission denied at /usr/sbin/amavisd line 212.
> fetch_modules: error loading optional module Image/Info.pm:
>Can't locate Image/Info.pm:   lib/Image/Info.pm: Permission denied at
> /usr/sbin/amavisd line 212.
> fetch_modules: error loading optional module Image/Info/GIF.pm:
> and many more!

Which USE-flags do you have?
I only have "clamav spamassassin" (the other parts are implemented differently 
for me)
As these are perl modules, did you try "perl-cleaner" to see if that fixes 
anything?

> >> The latest problem driving me up the wall is amavis-new wouldn't start
> >> after the upgrade.  I have postfix sending email to port 1024 where
> >> amavis is listening (this time required a new setting in amavisd.conf
> >> not previously needed) but postfix now wont accept email back from
> >> amavis on port 10025 so mail is mostly queued (some leaks at times - no
> >> idea why).
> > 
> > I assume you mean port 10024 ?
> 
> NO, 10025 - postix is configured to send mail to amavis on 10024 for
> scanning via clamav, and forward back to postix on 10025 where its
> getting the error

In your original email: " I have postfix sending email to port *1024* where "

> - note that this configuration has been working for
> over 20 years with the same basic configuration until now.  I originally
> set it up under a "mailuser" group ID and I am increasingly finding that
> on startup I have to check files to make sure their permissions are
> unchanged.  From the reading I have done on this I am suspecting that
> this latest version of amavis is trying to enforce "something" but not
> telling me what - at this stage I suspect amavis is the root cause and
> not postfix.

Are you still using "mailuser" ?
In " /etc/amavisd.conf ", what is configured for:

$daemon_user  = ...
$daemon_group = ...

> >> and what has thrown me: I can stop amavisd, then log in as user "amavis"
> >> and run "amavisd -c /etc/amavisd.conf debug" then everything works as
> >> intended! WHY?
> > 
> > Does postfix start before or after amavis?
> 
> The startup scripts start amavisd first, but there is no difference if I
> manually start amavis after postfix (unless I run it as the amavis user)

Ok, so when started as init-script, from root, it fails. when run as amavis, 
it works.
Am wondering if the 2 settings mentioned above have something other then 
amavis.

> >> I am preparing a new mail gateway LXC image as a clean install to try and
> >> straighten out the underlying permissions, but a fix for my current
> >> dilemma
> >> would be appreciated!
> > 
> > If a clean install works, I'd recommend a comparison between the 2 (start
> > with a diff for both "/etc") to check the cause.
> 
> Thats what I am working up to but I was hoping someone has seen this
> before to save time - its going to be a couple of days before I can get
> back to it.

I haven't seen this myself, but I have used the default user and group since I 
set this up.

--
Joost

Re: [gentoo-user] amavis/postfix and port 10025

[J. Roeleveld]

On Sunday, July 2, 2023 4:16:54 AM CEST William Kenworthy wrote:
> Hi all,
> 
> I have been  using a gentoo mail gateway for many years - its currently
> running under LXC and is upgraded using a generic LXC "golden master" image
> with the various email related packages being installed and config files
> copied across roughly a month or two apart.  This is always a trial,
> particularly with permissions and has become much worse with gentoo's
> attempt at using the acct packages to manage user and group ID's.

I actually find this easier to solve issues. What do you find difficult here?

> The latest problem driving me up the wall is amavis-new wouldn't start after
> the upgrade.  I have postfix sending email to port 1024 where amavis is
> listening (this time required a new setting in amavisd.conf not previously
> needed) but postfix now wont accept email back from amavis on port 10025 so
> mail is mostly queued (some leaks at times - no idea why).

I assume you mean port 10024 ?

> The main error message is:
> 
> 
> Jul  2 10:00:14 mail amavis[6074]: (06074-02-3) about to connect to
> smtp:[127.0.0.1]:10025, JZ76UHvsOKBa FWD from  ->
>  Jul  2 10:00:14 mail amavis[6074]: (06074-02-3) smtp
> session: setting up a new session Jul  2 10:00:14 mail amavis[6074]:
> (06074-02-3) new socket using IO::Socket::IP to [127.0.0.1]:10025, timeout
> 35 Jul  2 10:00:14 mail amavis[6074]: (06074-02-3) (!)connect to
> [127.0.0.1]:10025 failed, attempt #1: Unrecognised protocol tcp at
> /usr/sbin/amavisd line 8392. Jul  2 10:00:14 mail amavis[6074]:
> (06074-02-3) mail_via_smtp: session failed: All attempts (1) failed
> connecting to smtp:[127.0.0.1]:10025

This is postfix rejecting the connection.
Do you have the following:

# grep 10025 * 
master.cf:127.0.0.1:10025 inet n-   n -   -  smtpd

> and what has thrown me: I can stop amavisd, then log in as user "amavis" and
> run "amavisd -c /etc/amavisd.conf debug" then everything works as intended!
> WHY?

Does postfix start before or after amavis?

> I am preparing a new mail gateway LXC image as a clean install to try and
> straighten out the underlying permissions, but a fix for my current dilemma
> would be appreciated!

If a clean install works, I'd recommend a comparison between the 2 (start with 
a diff for both "/etc") to check the cause.

--
Joost

[gentoo-user] InfluxDB python library?

[J. Roeleveld]

Hi all,

I've been using a python script for the past 4 years to read out my solar 
panels and write the data into InfluxDB to get nice graphs in Grafana.

Unfortunately, the library I use to write to InfluxDB has a dead upstream and 
doesn't work with Python 3.11 anymore.
Does anyone know of a different way to load the data into InfluxDB using Python 
that is likely to be supported for the foreseeable future?

Many thanks,

Joost

Re: [gentoo-user] Solution for Remote Desktop

[J. Roeleveld]

On Sunday, May 7, 2023 8:49:46 PM CEST the...@sys-concept.com wrote:

> This setting is turn ON but still log user out.

Not sure if this is related, but if you have the "Home" version on the windows 
side, you might be limited to only 1 login.

> I found another solution, Chrome Remote Desktop" works perfectly.  User on
> Windows is not logged out when someone connect to it.

Good to hear you found a solution to this.

Re: [gentoo-user] Nextcloud-24.0.7 and proxy settings

[J. Roeleveld]

On Sunday, February 19, 2023 1:17:41 PM CET Alexander Puchmayr wrote:
> Hi there,
> 
> I'm trying to setup a nextcloud instance inside a DMZ that has no direct
> connection to the outside world, only via proxy. The proxy itself is working
> fine, but I fail to configure nextcloud to actually use this proxy.
> 
> According to the docs, I tried setting the proxy field in config/config.php,
> i.e.
> 
>  $CONFIG = array (
>   ...
>   'proxy' => '10.46.1.109:3128',
>   'proxyuserpwd' => '',
>   ...
> );
> 
> But this setting seems to be ignored, tcpdump does not show any connection
> attempt to that ip and nextcloud complains that it cannot connect to the
> internet.
> 
> In another forum I found that in
> /etc/apache2/vhosts.d/10_nextcloud_vhosts.conf, environment variables like
> HTTP_PROXY and HTTPS_PROXY may be set, i.e.
> 
> 
> ...
> SetEnv HTTP_PROXY 10.46.1.109:3128
> SetEnv HTTPS_PROXY 10.46.1.109:3128
> ...
> 
> 
> But this also does not seem to work.
> 
> Any other ideas?

If using "fpm-php", add the following to your config:
env[HTTP_PROXY] = 10.46.1.109:3128
env[HTTPS_PROXY] = 10.46.1.109:3128

I have this in:
/etc/php/fpm-php8.1/site.conf

After this, run:
/etc/init.d/fpm-php restart

This is how I got it working on my end.

--
Joost

Re: [gentoo-user] Jobs and load-average

[J. Roeleveld]

On Wednesday, February 15, 2023 10:56:22 AM CET Peter Humphrey wrote:
> Hello list,
> 
> Not long ago I read that we should allow 2GB RAM for every emerge job - that
> is, we should divide our RAM size by 2 to get the maximum number of
> simultaneous jobs. I'm trying to get that right, but I'm not there yet.
> 
> I have these entries in make.conf:
> EMERGE_DEFAULT_OPTS="--jobs=16 --load-average=32 --autounmask=n --quiet-
> unmerge-warn --ke>
> MAKEOPTS="-j16"
> 
> Today, though, I saw load averages going up to 72. Can anyone suggest better
> values to suit my 24 threads and 64GB RAM?

One other item I missed in the replies:
"--load-average" is also a valid option for make.

If you want to keep the load down, I would suggest adding this to MAKEOPTS as 
well:

MAKEOPTS="--jobs=16 --load-average=32"

I write the options out full because I had some weird errors in the past 
because the "-j" wasn't handled correctly at some point.

--
Joost

Re: [gentoo-user] Anybody using refind?

[J. Roeleveld]

On Thursday, February 2, 2023 1:15:51 PM CET Klaus Dittrich wrote:
> On 02.02.23 12:33, J. Roeleveld wrote:
> > Please only send mails to the list.
> > There is no need to add me to the TO or CC lists.
> > 
> > On Thursday, February 2, 2023 12:22:49 PM CET Klaus Dittrich wrote:
> >> On 02.02.23 10:51, J. Roeleveld wrote:
> >>> On Thursday, February 2, 2023 10:42:40 AM CET Klaus Dittrich wrote:
> >>>> Is there andbody using refind? Especially in Germany?
> >>>> I have some questions about.
> >>> 
> >>> If you have questions, feel free to ask on this list.
> >>> I am sure I am not the only one using it.
> >>> 
> >>> --
> >>> Joost
> >> 
> >> Joost,
> >> 
> >> my current system boots from bios(mbr) not uefi.
> >> 
> >> My knowlege of uefi is poor and uefi is still confusing me.
> >> So it may be some of my questions will looking silly for you.
> > 
> > Did you read the well-written howto on the Gentoo website?
> > 
> > https://wiki.gentoo.org/wiki/Refind[1]
> > 
> > 
> > 
> > 
> > [1] https://wiki.gentoo.org/wiki/Refind
> 
> Joost,
> 
> no. Thanks for the hint, I had just a short look into at it.
> 
> There is presupposed that one is at a uefi system already.
> 
> I have to install refind to a disk mounted to my mbr-booted-system,
> make all necessary adjustments there and then put it to the uefi-only
> machine and boot it there.
> 
> So I cannot call efibootmanger to pin refind to nvram of the uefi-system.

Use a USB boot-stick with EFI image, like the gentoo one. See the relevant 
section in the 
Gentoo installguide:

https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/Media[1]



[1] https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/Media

Re: [gentoo-user] Anybody using refind?

[J. Roeleveld]

Please only send mails to the list.
There is no need to add me to the TO or CC lists.

On Thursday, February 2, 2023 12:22:49 PM CET Klaus Dittrich wrote:
> On 02.02.23 10:51, J. Roeleveld wrote:
> > On Thursday, February 2, 2023 10:42:40 AM CET Klaus Dittrich wrote:
> >> Is there andbody using refind? Especially in Germany?
> >> I have some questions about.
> > 
> > If you have questions, feel free to ask on this list.
> > I am sure I am not the only one using it.
> > 
> > --
> > Joost
> 
> Joost,
> 
> my current system boots from bios(mbr) not uefi.
> 
> My knowlege of uefi is poor and uefi is still confusing me.
> So it may be some of my questions will looking silly for you.

Did you read the well-written howto on the Gentoo website?

https://wiki.gentoo.org/wiki/Refind[1]




[1] https://wiki.gentoo.org/wiki/Refind

Re: [gentoo-user] Anybody using refind?

[J. Roeleveld]

On Thursday, February 2, 2023 10:42:40 AM CET Klaus Dittrich wrote:
> Is there andbody using refind? Especially in Germany?
> I have some questions about.

If you have questions, feel free to ask on this list.
I am sure I am not the only one using it.

--
Joost

[gentoo-user] gentoo-sources-5.15.59 un-keyworded?

[J. Roeleveld]

Hi,

Is there a specific reason why the keywords (apart from hppa) have been removed 
from gentoo-sources-5.15.59?

I am seeing some issues with systems running this version, but not all. And as 
this is recent, I don't have an older kernel to quickly fall back to. 
Currently working on testing different versions to see if it is the kernel.

If anyone knows why this was done, please let me know. I couldn't find anything 
in b.g.o and my google-fu is failing me on this.

Many thanks,

Joost

Re: [gentoo-user] Backup program that compresses data but only changes new files.

[J. Roeleveld]

On Monday, August 15, 2022 9:07:41 PM CEST Dale wrote:
> J. Roeleveld wrote:
> > On Monday, August 15, 2022 12:44:11 AM CEST Dale wrote:
> >> Howdy,
> >> 
> >> With my new fiber internet, my poor disks are getting a work out, and
> >> also filling up.  First casualty, my backup disk.  I have one directory
> >> that is . . . well . . . huge.  It's about 7TBs or so.  This is where it
> >> is right now and it's still trying to pack in files.
> >> 
> >> /dev/mapper/8tb7.3T  7.1T  201G  98% /mnt/8tb
> > 
> > 
> > 
> >> Thoughts?  Ideas?
> > 
> > Plenty, see below:
> > 
> > For backups to external disks, I would recommend having a look at "dar" :
> > $ eix -e dar
> > * app-backup/dar
> > 
> >  Available versions:  2.7.6^t ~2.7.7^t {argon2 curl dar32 dar64 doc
> >  gcrypt
> > 
> > gpg lz4 lzo nls rsync threads xattr}
> > 
> >  Homepage:http://dar.linux.free.fr/
> >  Description: A full featured backup tool, aimed for disks
> > 
> > It's been around for a while and the developer is active and responds
> > quite
> > well to questions.
> > It supports compression (different compression methods), incremental
> > backups (only need a catalogue of the previous backup for the
> > incremental) and encryption.
> > 
> > The NAS options others mentioned would also work as they can compress data
> > on disk and you'd only notice a delay in writing/reading (depending on
> > the compression method used). I would recommend using one that uses ZFS
> > on-disk as it's more reliable and robust then BTRFS.
> > 
> > One option that comes available for you now that you are no longer limited
> > to slow ADSL: Cloud backups.
> > 
> > I use Backblaze (B2) to store compressed backups that haven't been stored
> > on tape to off-site locations.
> > 
> > But, you can also encrypt the backups locally and store the
> > encrypted+compressed backupfiles on other cloud storage.
> > 
> > --
> > Joost
> 
> Dar does sound interesting.  It sounds a lot like what I used way back
> in the 90's.  I'm sure it is different software but could work on
> floppies then like it does on USB sticks etc today.  Same principal. 

If it was during the 90's, then it wasn't. First version was released in 2002.

> I looked into ZFS as well.  Google helped me find a interesting page.  I
> notice it is also used on some NAS setups as well.  It seems to be
> advanced and maintained well.  It sounds a little like LVM but may have
> more features, such as compression maybe?  I haven't read that far yet. 
> I notice it mentions snapshots which LVM also uses. 

ZFS does a lot more then just LVM+Ext4. But it really needs multiple disks for 
all the anti-corruption features as well.

> Getting plenty of ideas.  I just wish I had a separate building to put a
> NAS in that would be safe and climate controlled.  I got a out building
> but it gets plenty hot in the summer.  No A/C or anything.  I only heat
> it enough to prevent freezing but computers would likely like that anyway. 

If you can keep it between optimal temperatures (and stable) the NAS should 
manage. There is NO need to keep it at 18C (like some places do).

Also, consider a small AC unit that only cools a small box big enough for the 
NAS. No need to cool an entire room.

--
Joost

Re: [gentoo-user] Backup program that compresses data but only changes new files.

[J. Roeleveld]

On Monday, August 15, 2022 8:56:30 PM CEST Rich Freeman wrote:
> On Mon, Aug 15, 2022 at 2:34 PM J. Roeleveld  wrote:
> > Actually, there still is a piece of software that does this:
> > " app-backup/dar "
> > You can tell it to split the backups into slices of a specific size.
> 
> dar is a great tool, but unless something changed I don't think you
> can tell it to pause to remount the destination directory when it
> fills up.  As was pointed out, tar does do this (which I thought was
> limited to tape devices, but apparently it works for disk as well).

Actually, you can with the "-p / --pause" option.
Also, as per the man-page, if you forget this, the process will simply inform 
you the target location is full and you can move slices away to a different 
location:
"
If the destination filesystem is too small to contain all the slices of the 
backup, the -p option (pausing before starting new slices) might be of 
interest. Else, in the case the filesystem is full, dar will suspend the 
operation, asking for the user to  make  free  space, then  continue its 
operation. To make free space, the only thing you cannot do is to touch the 
slice being written.
"

The pause-option will actually stop between slices and you can umount the 
target location and mount a different disk there.

This option has been around for a while.

--
Joost

Re: [gentoo-user] Backup program that compresses data but only changes new files.

[J. Roeleveld]

On Monday, August 15, 2022 9:52:26 AM CEST Dale wrote:
> Julien Roy wrote:
> > Hello,
> > 
> > On 8/14/22 18:44, Dale wrote:
> >> Thoughts?  Ideas?
> > 
> > You might be interested in borgbackup [1]
> > It takes delta backups and has de-duplication and compression to save
> > some space. It supports encryption too.
> > It's packaged in ::gentoo and you run it on whatever machine you want
> > to backup and give it its destination, it can be local or on a remote
> > machine.
> > 
> > I've been using it for a while and it works well. I have it configured
> > on a crontab and it backups my files every night
> > 
> > [1] https://www.borgbackup.org/
> 
> Since my drives are external, I do my backups manually.  Well, I start
> it when the drives are connected and ready.  I think borgbackup was one
> I looked into and it sounded more like a online backup where you store
> the data on a server somewhere.  I may be wrong on that tho.  I looked
> at several and it got confusing after a bit.  Plus, some were still as
> clear as mud.  Why do people link to a place that doesn't tell what
> their software does and how anyway.  It seems most think github and such
> are good places to link to when it really doesn't tell you anything
> unless you want to help develop the software or something.  It would be
> like Ford linking to CAD models to sell cars.  :/ 
> 
> To all:  I found a good deal on a 10TB drive.  That should suffice for
> now.  I might add, it will give me time to figure out a path forward and
> I can make other use of that SMR drive.  One thing I thought of as a
> negative for a NAS, I can't lock it into my safe, unless it is really
> tiny.  As it is, even if a fire comes along, I still got backups. 

I looked into this as well. A safe works like a nice little oven and the 
temperature inside the safe can go really high if it's inside a fire.
Not all storage system (HDDs included) are reliable when the temperatures go 
to these extremes.

Make sure the safe is, apart from resistent to fire, also capable of keeping 
the heat outside.

--
Joost

Re: [gentoo-user] Backup program that compresses data but only changes new files.

[J. Roeleveld]

On Monday, August 15, 2022 12:11:34 PM CEST Rich Freeman wrote:



> The main issue I think you're going to have is having support for
> multi-volume backups if you need to be able to split a backup across
> drives.  The only thing I've found on Linux that does this is bacula,
> and it is a royal pain that I'm embarrassed to even mention.  If
> somebody knows of another backup solution that can write the output to
> disk (a filesystem, not /dev/rmt) and then pause to mount a new disk
> when one fills up, I'm all ears.

app-backup/dar

For a "brief" guide on how to use it: 
http://dar.linux.free.fr/doc/Tutorial.html

Re: [gentoo-user] Backup program that compresses data but only changes new files.

[J. Roeleveld]

On Monday, August 15, 2022 9:05:24 AM CEST Dale wrote:
> Rich Freeman wrote:
> > On Sun, Aug 14, 2022 at 6:44 PM Dale  wrote:
> >> Right now, I'm using rsync which doesn't compress files but does just
> >> update things that have changed.  I'd like to find some way, software
> >> but maybe there is already a tool I'm unaware of, to compress data and
> >> work a lot like rsync otherwise.
> > 
> > So, how important is it that it work exactly like rsync?
> > 
> > I use duplicity, in part because I've been using it forever.  Restic
> > seems to be a similar program most are using these days which I
> > haven't looked at super-closely but I'd look at that first if starting
> > out.
> > 
> > Duplicity uses librsync, so it backs up exactly the same data as rsync
> > would, except instead of replicating entire files, it creates streams
> > of data more like something like tar.  So if you back up a million
> > small files you might get out 1-3 big files.  It can compress and
> > encrypt the data as you wish.  The downside is that you don't end up
> > with something that looks like your original files - you have to run
> > the restore process to extract them all back out.  It is extremely
> > space-efficient though - if 1 byte changes in the middle of a 10GB
> > file you'll end up just backing up maybe a kilobyte or so (whatever
> > the block size is), which is just like rsync.
> > 
> > Typically you rely on metadata to find files that change which is
> > fast, but I'm guessing you can tell these programs to do a deep scan
> > which of course requires reading the entire contents, and that will
> > discover anything that was modified without changing ctime/mtime.
> > 
> > The output files can be split to any size, and the index info (the
> > metadata) is separate from the raw data.  If you're storing to
> > offline/remote/cloud/whatever storage typically you keep the metadata
> > cached locally to speed retrieval and to figure out what files have
> > changed for incrementals.  However, if the local cache isn't there
> > then it will fetch just the indexes from wherever it is stored
> > (they're small).
> > 
> > It has support for many cloud services - I store mine to AWS S3.
> > 
> > There are also some options that are a little closer to rsync like
> > rsnapshot and burp.  Those don't store compressed (unless there is an
> > option for that or something), but they do let you rotate through
> > multiple backups and they'll set up hard links/etc so that they are
> > de-duplicated.  Of course hard links are at the file level so if 1
> > byte inside a file changes you'll end up with two full copies.  It
> > will still only transfer a single block so the bandwidth requirements
> > are similar to rsync.
> 
> Duplicity sounds interesting except that I already have the drive
> encrypted.  Keep in mind, these are external drives that I hook up long
> enough to complete the backups then back in a fire safe they go.  The
> reason I mentioned being like rsync, I don't want to rebuild a backup
> from scratch each time as that would be time consuming.  I thought of
> using Kbackup ages ago and it rebuilds from scratch each time but it
> does have the option of compressing.  That might work for small stuff
> but not many TBs of it.  Back in the early 90's, I remember using a
> backup software that was incremental.  It would only update files that
> changed and would do it over several floppy disks and compressed it as
> well.  Something like that nowadays is likely rare if it exists at all
> since floppies are long dead.  I either need to split my backup into two
> pieces or compress my data.  That is why I mentioned if there is a way
> to backup first part of alphabet in one command, switch disks and then
> do second part of alphabet to another disk. 

Actually, there still is a piece of software that does this:
" app-backup/dar "
You can tell it to split the backups into slices of a specific size.

--
Joost

Re: [gentoo-user] Backup program that compresses data but only changes new files.

[J. Roeleveld]

On Monday, August 15, 2022 12:44:11 AM CEST Dale wrote:
> Howdy,
> 
> With my new fiber internet, my poor disks are getting a work out, and
> also filling up.  First casualty, my backup disk.  I have one directory
> that is . . . well . . . huge.  It's about 7TBs or so.  This is where it
> is right now and it's still trying to pack in files. 
> 
> /dev/mapper/8tb7.3T  7.1T  201G  98% /mnt/8tb



> Thoughts?  Ideas? 

Plenty, see below:

For backups to external disks, I would recommend having a look at "dar" : 
$ eix -e dar
* app-backup/dar
 Available versions:  2.7.6^t ~2.7.7^t {argon2 curl dar32 dar64 doc gcrypt 
gpg lz4 lzo nls rsync threads xattr}
 Homepage:http://dar.linux.free.fr/
 Description: A full featured backup tool, aimed for disks

It's been around for a while and the developer is active and responds quite 
well to questions.
It supports compression (different compression methods), incremental backups 
(only need a catalogue of the previous backup for the incremental) and 
encryption.

The NAS options others mentioned would also work as they can compress data on 
disk and you'd only notice a delay in writing/reading (depending on the 
compression method used). I would recommend using one that uses ZFS on-disk as 
it's more reliable and robust then BTRFS.

One option that comes available for you now that you are no longer limited to 
slow ADSL: Cloud backups.

I use Backblaze (B2) to store compressed backups that haven't been stored on 
tape to off-site locations.

But, you can also encrypt the backups locally and store the 
encrypted+compressed backupfiles on other cloud storage.

--
Joost

Re: [gentoo-user] hibernate... /bin/echo: write error: No such device

[J. Roeleveld]

On Wednesday, 27 July 2022 23:53:08 CEST Walter Dnes wrote:
>   One of the last few items on the laptop setup.  I emerged hibernate
> and copied over the /etc/hibernate/ directory from my desktop.  When I
> try to hibernate the laptop, I get...
> 
> [thimk][root][~] hibernate
> /bin/echo: write error: No such device
> 
> ...with a beep, and the machine comes back.  Grepping for the "echo"
> command, I get...
> 
> [thimk][root][~] grep -r echo /etc/hibernate/
> /etc/hibernate/common.conf:# OnSuspend 20 echo "Good night!"
> /etc/hibernate/common.conf:# OnResume 20 echo "Good morning!"
> 
>   The latest entries in /var/log/hibernate.log are...
> 
> Starting suspend at Wed Jul 27 05:00:47 PM EDT 2022
> hibernate: [01] Executing CheckLastResume ...
> hibernate: [01] Executing CheckRunlevel ...
> hibernate: [01] Executing LockFileGet ...
> hibernate: [01] Executing NewKernelFileCheck ...
> hibernate: [10] Executing EnsureSysfsPowerStateCapable ...
> hibernate: [11] Executing XHacksSuspendHook1 ...
> hibernate: [59] Executing RemountXFSBootRO ...
> hibernate: [89] Executing SaveKernelModprobe ...
> hibernate: [91] Executing ModulesUnloadBlacklist ...
> hibernate: [95] Executing XHacksSuspendHook2 ...
> hibernate: [98] Executing CheckRunlevel ...
> hibernate: [99] Executing DoSysfsPowerStateSuspend ...
> hibernate: Activating sysfs power state disk ...
> hibernate: [90] Executing ModulesLoad ...
> hibernate: [89] Executing RestoreKernelModprobe ...
> hibernate: [85] Executing XHacksResumeHook2 ...
> hibernate: [70] Executing ClockRestore ...
> hibernate: [59] Executing RemountXFSBootRW ...
> hibernate: [11] Executing XHacksResumeHook1 ...
> hibernate: [01] Executing NoteLastResume ...
> hibernate: [01] Executing LockFilePut ...
> Resumed at Wed Jul 27 05:00:50 PM EDT 2022


I wonder at which point it actually shows that message.
I assume you are using:
# eix hibernate 
* sys-power/hibernate-script 
Available versions:  2.0-r6 {vim-syntax} 
Homepage:https://gitlab.com/nigelcunningham/Hibernate-Script 
Description: Hibernate script supporting multiple suspend methods


If yes, then try it with the verbose-option, maybe then you get to see where 
that is generated.

Does suspend itself work? Or is the script failing as well?
Do you have hibernate configured in the kernel?

To check, you can try forcing hibernate manually by:

# echo disk > /sys/power/state

To resume, you will need to inform the kernel where to resume from.

Kernel command line option:  resume="/path/to/swap/device" 

If your swap-partition is a LV (eg, using LVM), you'll need to use an initrd 
that will find the correct device-ID to pass to the resume-functionality.

I created my own init-ramfs for this and use the "hibernate" option provided 
by kde/plasma. I can't remember when I last used a third-party script to get 
this to work as it's generally quite simple to get working.

--
Joost

Re: [gentoo-user] virtualbox woes

[J. Roeleveld]

On Monday, 1 August 2022 02:46:32 CEST Matthew Sacks wrote:
> The vbox log and screenshot are all I have to go off. Next time I?ll provide
> that upfront. New to these parts (gentoo lists).
> 
> It crashes on boot actually to answer your question.

Please stop top-posting. It makes these emails difficult to read.

Did you enable hardware virtualisation (VT) in the BIOS?
What are the VM-settings you use for the VM?
Does your host have sufficient CPU-cores and memory for the Host + the VM?

--
Joost


> 
> Sent from Mail for Windows
> 
> From: Jack
> Sent: Sunday, July 31, 2022 7:48 AM
> To: gentoo-user@lists.gentoo.org
> Subject: Re: [gentoo-user] virtualbox woes
> 
> On 7/31/22 09:27, Matthew Sacks wrote:
> I cant get gentoo to run on virtualbox, segfaults or core dumps or
> something.
> 
> Please learn how to ask for help.  You have provided next to no useful
> information.  From the other thread, you provided a log, and the fact that
> you are trying to run a Gentoo guest on a Windows (which version?) host. 
> Which version of VB?  The latter might matter, as there have been some
> reports of VBox instability, although I believe those are on Linux hosts,
> not guests.
> 
> In the other thread, you mention that the crash happens on shutting down. 
> How did you shut down?  Have you tried other ways?  Have you checked the
> virtualbox forums for similar reports?
> 
> Jack

Re: [gentoo-user] Any way to automate login to host and su to root?

[J. Roeleveld]

On Monday, 18 July 2022 08:03:44 CEST Grant Taylor wrote:
> On 7/17/22 11:48 PM, J. Roeleveld wrote:
> > It could, but that would open up an unsecured key to interception if
> > an intermediate host is compromised.
> 
> What are you thinking?  --  I've got a few ideas, but rather than
> speculating, I'll just ask.

See below

> > See previous answer, the agent, as far as I know, will have the keys
> > in memory and I haven't seen evidence that it won't provide the keys
> > without authenticating the requestor.
> 
> Are you concerned about a rogue requestor on the host where the agent is
> running or elsewhere?

Either on the client where the agent is running, but also on the system I 
connected to.
But, I just noticed the following, which is hopeful, but need to read up on 
this:
https://www.openssh.com/agent-restrict.html[1]

> > Yes, copy/paste has no issues with multi-page texts. But manually
> > reading a long password and copying that over by typing on a keyboard
> > when the font can make the difference between "1" (ONE), "l" (small
> > letter L) and "|" (pipe- character) and similar characters make it
> > annoying to say the least.
> 
> Agreed.
> 
> > Currently, when that comment pops up, the first thing I do is wait
> > and wonder why it's asking for it. As all the systems are already
> > added to the list.
> 
> Such a pop-up would be a very likely indication of a problem.

Agreed, which is why I always stop and think when I see that.
Usually the answer is: "Oh, yes, I didn't access this host from my laptop yet". 
But that is usually 
after the 2nd or 3rd connection attempt with retyping the hostname and 
verifying the IP-address 
that is resolved for it first.

--
Joost


[1] https://www.openssh.com/agent-restrict.html

Re: [gentoo-user] Any way to automate login to host and su to root?

[J. Roeleveld]

On Friday, 15 July 2022 18:39:25 CEST Grant Taylor wrote:
> On 7/14/22 3:22 PM, Steve Wilson wrote:
> > Have you looked at dev-tcltk/expect?
> 
> Expect has it's place.
> 
> Just be EXTREMELY careful when using it for anything security related.

I agree

> Always check for what is expected before sending data.  Don't assume
> that something comes next and blindly send it (possibly after a pause).
> 
> Things break in a really weird and unexpected way.  (No pun intended.)
> 
> Also, do as much logic outside of expect as possible.  E.g. don't try to
> add a user and then respond to a failure.  Instead check to see if the
> user exists /before/ trying to add it.

I've been using ansible for some of my automation scripts and am happy with 
the way that works. The existing implementations for "adding users" and such 
is tested plenty by others and does actually check if the user exists before 
trying to add one.

> Plan on things failing and try to control the likely ways that it can fail.
> 
> Paying yourself forward with time and effort developing (expect) scripts
> will mean that you reap the rewards for years to come.

I only use expect to automate the login-process as mentioned in the original 
email. The line it's expecting is more then just "*?assword" like in all the 
examples.

Currently, SSH puts the password-prompt as:
(@) Password:

As I know both, the expected string is this full line. If SSH changes its 
behaviour, the script will simply fail.

--
Joost

Re: [gentoo-user] Any way to automate login to host and su to root?

[J. Roeleveld]

On Sunday, 17 July 2022 21:15:05 CEST Grant Taylor wrote:
> On 7/15/22 11:46 PM, J. Roeleveld wrote:
> > Hmm... interesting. I will look into this.
> :
> :-)
> :
> > But, it needs the agent to be running, which will make it tricky for
> > automation.
> 
> Why can't automation start an agent?

It could, but that would open up an unsecured key to interception if an 
intermediate host is compromised.

> Why can't there be an agent
> running that automation has access to?

See previous answer, the agent, as far as I know, will have the keys in memory 
and I haven't seen evidence that it won't provide the keys without 
authenticating the requestor.

> > I know, which is why I was investigating automating it. The passwords
> > are too long to comfortably copy by hand.
> 
> I assume that you mean "type" when you say "copy".

Yes, copy/paste has no issues with multi-page texts. But manually reading a 
long password and copying that over by typing on a keyboard when the font can 
make the difference between "1" (ONE), "l" (small letter L) and "|" (pipe-
character) and similar characters make it annoying to say the least.

> > I will definitely investigate this. They sound interesting. I'd set
> > the validity to a lot less if this can be automated easily.
> 
> Yes, it can be fairly easily automated.
> 
> One of the other advantages of SSH /certificates/ is when you flip
> things around and use a /host/ certificate.  Clients can recognize that
> the target host's certificate is signed by the trusted SSH CA and not
> prompt for the typical Trust On First Use (TOFU) scenario.  Thus you can
> actually leverage the target host SSH fingerprint and not need to ignore
> that security aspect like so many people do.

Currently, when that comment pops up, the first thing I do is wait and wonder 
why it's asking for it. As all the systems are already added to the list.

--
Joost

Re: [gentoo-user] Any way to automate login to host and su to root?

[J. Roeleveld]

On Sunday, 17 July 2022 21:10:52 CEST Grant Taylor wrote:
> On 7/15/22 11:42 PM, J. Roeleveld wrote:
> > True, properly done automation is necessary to make our lives easier.
> 
> #truth
> 
> > I tried this approach in the past and some levels of automation still
> > use this, but for being able to login myself, I found having different
> > keys become cumbersome and I ended up never actually replacing them.
> 
> I'm curious what you found to be cumbersome.

If I have 1 desktop and 1 laptop, that means 2 client machines.
Add 5 servers/vms.
That means 10 ssh-keys per person to manage and keep track off.
When a laptop gets replaced, I need to ensure the keys get removed from the 
authorized_keys section. Same goes for when the ssh-keys need refreshing. 
Which, due to the amount, I never got round to.

I actually have more then the amount mentioned above, the amount of ssh-keys 
gets too much to manage without an automated tool to keep track of them and 
automate the changing of the keys. I never got the time to create that tool 
and never found anything that would make it easier.

> I make extensive use of the client SSH configuration file
> (~/.ssh/config) such that I don't need to worry about which key is used
> for which host.  This means that anything that uses ssh / sftp / scp
> /just/ /works/ (tm) using the contents of the configuration file.

When hosts can get added and removed regularly for testing purposes, this 
requires a management tool.

> > The goal is to have whichever authentication system used, the
> > passwords/keys to be replaced often with hard to brute-force
> > passwords/keys. I can currently replace all passwords on a daily
> > basis and not have a problem with accessing any system.
> 
> I agree in concept.  Though I question the veracity of that statement
> when things aren't working normally.  E.g. system is offline for X hours
> do to hardware failure or an old version restored from backup that is
> now out of sync with the central system.

Down due to hardware issues means the password-refresh fails for this host.
Backup-restore scripts have a step added to update the passwords updated to 
whatever is in the vault before the system is brought back online.

I actually considered these and made sure it can handle this. The most common 
issue is a network link being down due to ISP issues.

> > For normal use, most systems don't need to be logged into a shell. For
> > the few where this is needed, individual accounts exists.  But, no
> > individual account is a member of "wheel".  For admin access, there are
> > admin accounts on the machines. (they are all named individually and
> > you won't find the same admin-account-username on more then 1 system)
> 
> I've wondered about having the account for UID / GID 0 be named
> something other than root.  But the testing that I did showed that there
> were too many things that assumed "root".  :-/

You could put "root" without a valid password, making it impossible to "su -" 
into and add a 2nd uid/gid 0 account with a valid password. I know of 1 
organisation where they had a 2nd root account added which could be used by 
the orgs sys-admins for emergency access. (These were student owned servers 
directly connected to the internet)

> Though I did find that I was able to successfully convert a test VM to
> use something other than root and the proof of concept was a success.
> It's just that the PoC was too much effort / fragile to be used in
> production.
> 
> I find that the wheel group is mostly for su and a few other commands.
> But the concept of you must be a member of a group or have special
> permissions applied directly to your account is conceptually quite
> similar to being a member of the wheel group.  As such I don't think the
> abstraction makes much difference other than obfuscation.

I expect the "wheel" group to only be for changing into "root", that's what 
it's advertised as.

> > True, but this needs to run from the client. Not the server. Which
> > means it will need to be triggered manually and not scheduled.
> 
> The algorithm could be refactored such that it is run from the server.
> E.g. if you can ensure that the old key is replaced with the new key, it
> can safely be done server side.  I did this for a few colleagues that
> had forgotten the passphrase for their old private key and needed their
> new public key to be put into place.

Still needs the clients to be actually running when the server runs the 
script. Or it needs to be added to a schedule and gets triggered when the 
client becomes available. This would make the scheduler too complex.

> > I don't even have sudo installed on most systems, only where it's
> > needed for certain scrip

Re: [gentoo-user] Any way to automate login to host and su to root?

[J. Roeleveld]

On Friday, 15 July 2022 14:44:10 CEST Neil Bothwick wrote:
> On Fri, 15 Jul 2022 09:53:44 +0200, J. Roeleveld wrote:
> > > There's no reason you cannot change SSH keys as regularly, and good
> > > reasons why you should. It's just that people don't bother to do it.
> > 
> > I agree, but that is a tedious process.
> > 
> > I have multiple machines I use as desktop depending on where I am. And
> > either I need to securely share the private keys between them or set up
> > different keys per desktop.
> > I assume the same is true for most people.
> 
> I don't share keys, each desktop/laptop has its own keys.

I agree this is more secure as you can remove potentially leaked keys 
individually. But with more devices, the amount of keys and places where these 
need to be removed increases.

> > Never mind that access to the servers needs to be possible for others
> > as well.
> > 
> > Either way, to do this automatically, all the desktop machines need to
> > be powered and running while changing the keys.
> 
> Not if they use their own keys. It should be simple to script generating
> a new key, then SSHing to a list of machines and replacing the old key
> with the new one in authorized_keys.

This script will need to be run by the individual user. I prefer to control 
this centrally.

> > Changing passwords for servers and storing them in a password vault is
> > easier to automate.
> 
> Indeed it is, and now you've found a way to do what you want with
> passwords, all is well.
> 
> However, I will look at scripting regular replacements for SSH keys, for
> my own peace of mind.

Most security improvements start with "simple" questions like these :)

Good luck with your scripts :)

--
Joost

Re: [gentoo-user] Any way to automate login to host and su to root?

[J. Roeleveld]

On Friday, 15 July 2022 18:32:52 CEST Grant Taylor wrote:
> On 7/15/22 1:53 AM, J. Roeleveld wrote:
> > I agree, but that is a tedious process.
> 
> Yes, it can be.  That's where some automation comes into play.

True, properly done automation is necessary to make our lives easier.

> > I have multiple machines I use as desktop depending on where I am. And
> > either I need to securely share the private keys between them or set
> > up different keys per desktop.
> 
> I /currently/ use unique keys /per/ /client/ /system/.
> 
> I am /planing/ on starting to use unique keys /per/ /client/ /per/
> /server/.  Meaning that each client will use a different key for each
> remote server.  I think that this combined with location restrictions in
> the authorized_keys file will mean that SSH keys (or certificates) can't
> be used from anywhere other than their approved location or for anything
> other than their intended purpose.

I tried this approach in the past and some levels of automation still use 
this, but for being able to login myself, I found having different keys become 
cumbersome and I ended up never actually replacing them.

> > I assume the same is true for most people.
> 
> Yes.  It depends what security posture you / your organization want.

The goal is to have whichever authentication system used, the passwords/keys 
to be replaced often with hard to brute-force passwords/keys. I can currently 
replace all passwords on a daily basis and not have a problem with accessing 
any system.

> > Never mind that access to the servers needs to be possible for others
> > as well.
> 
> I assume that other users will use their own individual accounts to log
> into the target systems with a similar configuration.
> 
> E.g. I log into remote systems as "gtaylor" and you log into remote
> systems as "joost", and Neil logs into remote systems as "neil".  We
> would all then escalate to root via "su -" with the automation providing
> the password to su.

For normal use, most systems don't need to be logged into a shell. For the few 
where this is needed, individual accounts exists.
But, no individual account is a member of "wheel".
For admin access, there are admin accounts on the machines. (they are all 
named individually and you won't find the same admin-account-username on more 
then 1 system)

> > Either way, to do this automatically, all the desktop machines need
> > to be powered and running while changing the keys.
> 
> No, they don't.
>
> You just need to account for current and prior keys.
> 
> I've done exactly this on a fleet of about 800 Unix systems that I
> helped administer at my last job.  You do something like the following:
> 
> 1)  Log into the remote system explicitly using the prior key.
> 2)  Append the current key to the ~/.ssh/authorized_keys file.
> 3)  Logout of the remote system.
> 4)  Log into the remote system explicitly using the current key.
> 5)  Remove the prior key from the ~/.ssh/authorized_keys file.
> 6)  Logout of the remote system.
> 
> This can be fairly easily automated.

True, but this needs to run from the client. Not the server. Which means it 
will need to be triggered manually and not scheduled.
 
> You can then loop across systems using this automation to update the key
> on systems that are online.
> 
> You can relatively easily deal with systems that are offline currently
> later when they are back online.  --  There are ways to differentiate
> between offline and bad credentials during day to day operations.  So
> when you hit the bad credentials you leverage the automation that tries
> old credentials to update them.
> 
> You end up bifurcating the pool of systems into different groups that
> need to be dealt with differently.  Online and doing what you want;
> online but not doing what you want; and offline.
> 
> > Changing passwords for servers and storing them in a password vault
> > is easier to automate.
> 
> I disagree.
> 
> Using passwords tends to negate things like authenticating to sudo with
> SSH keys / certificates, thus prompting the use of NOPASSWD:.

I don't even have sudo installed on most systems, only where it's needed for 
certain scripts to work and there it's only used to avoid "setuid" which is an 
even bigger issue.

--
Joost

Re: [gentoo-user] Any way to automate login to host and su to root?

[J. Roeleveld]

On Friday, 15 July 2022 18:15:04 CEST Grant Taylor wrote:
> On 7/15/22 1:15 AM, J. Roeleveld wrote:
> > Yes.
> 
> Okay.
> 
> That simply means that SSH keys won't be used to authenticate to the
> remote system.
> 
> > How would it not prompt for a password.
> 
> There is a PAM module; pam_ssh_agent_auth, which can be used to enable
> users to authenticate to sudo using SSH keys.  This means that the user
> /does/ authenticate to sudo as necessary.  It's just that the
> authentication happens behind the scenes and they don't need to enter
> their password.  Thus you can avoid the NOPASSWD: option which means a
> better security posture.

Hmm... interesting. I will look into this.
But, it needs the agent to be running, which will make it tricky for 
automation. (I have some scripts that need to do things on different systems 
in a sequence for which this could help)

> > I need something that will take the password from the vault (I
> > can do this in Python and shell-scripting. Probably also in other
> > scripts). Authenticating to the vault can be done on a session basis
> > and shared. So locally, I'd only login once.
> 
> Sure.
> 
> > Currently, yes. I never physically see the password as it currently
> > goes into the clipboard and gets wiped from there after a short time
> > period. Enough time to paste it into the password-prompt. It's
> > the copy/pasting that I am looking to automate into a single
> > "login-to-remote-host" script.
> 
> I would not consider the copy and paste method to be secure.  There are
> plenty of utilities to monitor the clipboard et al. and copy the new
> contents in extremely short order.  As such, users could arrange to
> acquire copies of the password passing through the clipboard.

I know, which is why I was investigating automating it. The passwords are too 
long to comfortably copy by hand.

> I would strongly suggest exploring options that don't use the clipboard
> and instead retrieve the password from the vault and inject it into the
> remote system without using the clipboard.
> 
> Or, authenticate to sudo a different way that doesn't involve a
> password.  This will work for 90+ percent of the use cases.  Meaning
> that the sensitive password is needed for 10 percent or less of the
> time.  Thereby reducing the possible sensitive password exposure.  }:-)
> 
> > I prefer not to use SSH keys for this as they tend to exist for years
> > in my experience. And one unnoticed leak can open up a lot of systems.
> 
> That is a valid concern.
> 
> I'd strongly suggest that you research SSH /certificates/.  SSH
> /certificates/ support a finite life time /and/ can specify what
> command(s) / action(s) they can be used for.
> 
> My $EMPLOYER uses SSH /certificates/ that last about 8 hours.  I've
> heard of others that use SSH /certificates/ that last for a single digit
> number of minutes or even seconds.  The idea being that the SSH
> /certificate/ only lasts just long enough for it to be used for it's
> intended purpose and no longer.

I will definitely investigate this. They sound interesting. I'd set the 
validity to a lot less if this can be automated easily.

> The ability to specify the command; e.g. "su -" that is allowed to be
> executed means that people can't use them to start any other command.  }:-)
> 
> > This is why I use passwords. (passwords are long random strings that
> > are changed regularly)
> 
> Fair enough.  I only counter with take a few minutes to research SSH
> /certificates/ and see if they are of any interest to you.

Added to my research-list.

--
Joost

Re: [gentoo-user] Any way to automate login to host and su to root?

[J. Roeleveld]

On Friday, 15 July 2022 10:13:12 CEST J. Roeleveld wrote:
> On Thursday, 14 July 2022 23:22:46 CEST Steve Wilson wrote:
> > On 14/07/2022 07:35, J. Roeleveld wrote:
> > > Hi All,
> > > 
> > > I am looking for a way to login to a host and automatically change to
> > > root
> > > using a password provided by an external program.
> > > 
> > > The root passwords are stored in a vault and I can get passwords out
> > > using
> > > a script after authenticating.
> > > 
> > > Currently, I need to do a lot of the steps manually:
> > > ssh @
> > > su -
> > > (copy/paste password from vault)
> > > 
> > > I would like to change this to:
> > >  
> > > 
> > > Does anyone have any hints on how to achieve this without adding a
> > > "NOPASSWD" entry into /etc/sudoers ?
> > > 
> > > Thanks in advance,
> > > 
> > > Joost
> > 
> > Have you looked at dev-tcltk/expect?
> > 
> > There's possibly an example you could try at
> > <https://www.journaldev.com/1405/expect-script-ssh-example-tutorial>
> > although you probably want to prompt for the password or retreive it
> > programatically rather than putting it on the command line :o
> > 
> > Steve.
> 
> This looks promising. Will have a look to see if this can be made to work.
> I will need to find a way to get the password programmatically inside the
> script as I will not put it on the commandline and definitely not hard-coded
> in a script.

Thank you, this works.
Got the script to grab all the details needed from the vault and ends up 
giving me a remote root-prompt.

--
Joost

Re: [gentoo-user] Any way to automate login to host and su to root?

[J. Roeleveld]

On Thursday, 14 July 2022 23:22:46 CEST Steve Wilson wrote:
> On 14/07/2022 07:35, J. Roeleveld wrote:
> > Hi All,
> > 
> > I am looking for a way to login to a host and automatically change to root
> > using a password provided by an external program.
> > 
> > The root passwords are stored in a vault and I can get passwords out using
> > a script after authenticating.
> > 
> > Currently, I need to do a lot of the steps manually:
> > ssh @
> > su -
> > (copy/paste password from vault)
> > 
> > I would like to change this to:
> >  
> > 
> > Does anyone have any hints on how to achieve this without adding a
> > "NOPASSWD" entry into /etc/sudoers ?
> > 
> > Thanks in advance,
> > 
> > Joost

> Have you looked at dev-tcltk/expect?
> 
> There's possibly an example you could try at
> <https://www.journaldev.com/1405/expect-script-ssh-example-tutorial>
> although you probably want to prompt for the password or retreive it
> programatically rather than putting it on the command line :o
> 
> Steve.
> 

This looks promising. Will have a look to see if this can be made to work.
I will need to find a way to get the password programmatically inside the 
script as I will not put it on the commandline and definitely not hard-coded 
in a script.

--
Joost

Re: [gentoo-user] Any way to automate login to host and su to root?

[J. Roeleveld]

On Friday, 15 July 2022 09:29:14 CEST Neil Bothwick wrote:
> On Fri, 15 Jul 2022 09:15:02 +0200, J. Roeleveld wrote:
> > I prefer not to use SSH keys for this as they tend to exist for years
> > in my experience. And one unnoticed leak can open up a lot of systems.
> > This is why I use passwords. (passwords are long random strings that
> > are changed regularly)
> 
> There's no reason you cannot change SSH keys as regularly, and good
> reasons why you should. It's just that people don't bother to do it.

I agree, but that is a tedious process.

I have multiple machines I use as desktop depending on where I am. And either 
I need to securely share the private keys between them or set up different 
keys per desktop.
I assume the same is true for most people.

Never mind that access to the servers needs to be possible for others as well.

Either way, to do this automatically, all the desktop machines need to be 
powered and running while changing the keys.

Changing passwords for servers and storing them in a password vault is easier 
to automate.

--
Joost

Re: [gentoo-user] Any way to automate login to host and su to root?

[J. Roeleveld]

On Thursday, 14 July 2022 17:30:28 CEST Grant Taylor wrote:
> On 7/14/22 12:35 AM, J. Roeleveld wrote:
> > Hi All,
> 
> Hi,
> 
> > I am looking for a way to login to a host and automatically change
> > to root using a password provided by an external program.
> 
> Please clarify if you want to /require/ a password?

Yes.

> I can think of some options that would authenticate, thus avoiding
> sudo's NOPASSWD:, but not prompt for a password.  I want to know if
> those types of options are on the table or if they should be discarded.

How would it not prompt for a password. I need something that will take the 
password from the vault (I can do this in Python and shell-scripting. Probably 
also in other scripts). Authenticating to the vault can be done on a session 
basis and shared. So locally, I'd only login once.

> > The root passwords are stored in a vault and I can get passwords out
> > using a script after authenticating.
> 
> Okay.
> 
> > Currently, I need to do a lot of the steps manually:
> > ssh @
> > su -
> 
> You could alter that slightly to be:
> 
> ssh @ su -
> 
> That would combine the steps into one.
> 
> > (copy/paste password from vault)
> 
> Are you actually copying & pasting the password?  Or will you be using
> something to retrieve the password from the vault and automatically
> provide it to su?

Currently, yes. I never physically see the password as it currently goes into 
the clipboard and gets wiped from there after a short time period. Enough time 
to paste it into the password-prompt. It's the copy/pasting that I am looking 
to automate into a single "login-to-remote-host" script.

> I think that removing the human's need ~> ability to copy & paste would
> close some security exposures.
> 
> Aside:  This remove the human's ability to copy ~> know the password
> from the mix as a security measure can be a slippery slope and I
> consider it to be questionable at best.  --  Conversely, doing it on
> behalf of the human with a password that they know simply as automation
> is fine.
> 
> > I would like to change this to:
> >  
> 
> I think that's doable.  I've done a lot of that.  I'll take it one step
> further and put " " in a for loop to do my bidding on
> a number of systems.
> 
> I think the "ssh @ su -" method might be a bit cleaner from
> a STDIN / TTY / FD perspective.
> 
> > Does anyone have any hints on how to achieve this without adding a
> > "NOPASSWD" entry into /etc/sudoers ?
> 
> Flag on the play:  You've now mixed privilege elevation mechanism.  You
> originally talked about "su" and now you're talking about "sudo".  They
> are distinctly different things.  Though admittedly they can be used in
> concert with each other.
> 
> If you are using SSH keys /and/ sudo, then I'd recommend that you
> investigate authenticating to sudo via (forwarded) SSH keys.  This means
> that your interactions with sudo are /always/ authenticated *and* done
> so without requiring an interactive prompt.

I prefer not to use SSH keys for this as they tend to exist for years in my 
experience. And one unnoticed leak can open up a lot of systems.
This is why I use passwords. (passwords are long random strings that are 
changed regularly)

> > Thanks in advance,
> 
> There's more than a little bit here.  There are a number of ways that
> this could go.

Re: [gentoo-user] Any way to automate login to host and su to root?

[J. Roeleveld]

On Thursday, 14 July 2022 17:32:07 CEST Grant Taylor wrote:
> On 7/14/22 3:54 AM, J. Roeleveld wrote:
> > For security reasons, I do not want direct login to root under any
> > circumstances. This is disabled on all systems and will stay this way.
> 
> +10 for security
> 
> > Currently, to login as root, you need to know:
> > - admin user account name
> > - admin user account password
> > - root user account password
> 
> Please describe what an ideal scenario would be from a flow perspective,
> independent of the underlying technology.

What I am looking for is:
1) Lookup credentials from password vault (I can do this in script-form, 
already doing this in limited form for ansible-scripts, but this doesn't give 
me an interactive shell)

2) Use admin-account credentials to login via SSH into host

3) On remote host, initiate "su -" to switch to root and provide root-password 
over SSH link at the right time

4) Give me an interactive root-shell on remote-host

When I close the shell, I expect to be fully logged out (eg, I go straight 
back to the local host, not to the admin-account)


> > I do not want to reduce this to a single ssh-key-passphrase.
> 
> Please elaborate as I suspect that the reasoning behind that statement
> is quite germane to this larger discussion.

I see plenty of google-results and also as answers for ssh directly to "root" 
using ssh-keys.  I do not consider this a safe method, I use it for un-
priviliges accounts (not member of "wheel"). I don't use it for admin-
accounts.

Re: [gentoo-user] Any way to automate login to host and su to root?

[J. Roeleveld]

On Thursday, 14 July 2022 10:04:21 CEST Mickaël Bucas wrote:
> Le jeu. 14 juil. 2022 à 08:35, J. Roeleveld  a écrit :
> > Hi All,
> > 
> > I am looking for a way to login to a host and automatically change to root
> > using a password provided by an external program.
> > 
> > The root passwords are stored in a vault and I can get passwords out using
> > a script after authenticating.
> > 
> > Currently, I need to do a lot of the steps manually:
> > ssh @
> > su -
> > (copy/paste password from vault)
> 
> Why not use directly ssh root@ ?
> With an SSH key protected by a passphrase that would be a single step
> to connect.
> You would have a passphrase to manage but you already are using a tool for
> that. If you accept the risks, you could also use an SSH key without a
> passphrase.
> 
> sshd on the host must be configured with
> PermitRootLogin=prohibit-password at minimum, which is the default
> value.

For security reasons, I do not want direct login to root under any 
circumstances. This is disabled on all systems and will stay this way.

Currently, to login as root, you need to know:
- admin user account name
- admin user account password
- root user account password

I do not want to reduce this to a single ssh-key-passphrase.

--
Joost

Re: [gentoo-user] Any way to run multiple commands from single script in parallel?

[J. Roeleveld]

On Monday, March 14, 2022 11:51:44 AM CET Björn Fischer wrote:
> Hello Joost,
> 
> > Is there a tool/method to execute multiple lines/commands
> > simultaneously? Like having 3 or 4 run together and when 1 is
> > finished, it will grab the next one in the list?
> 
> probably, GNU Parallel is what you are looking for:
> 
> https://www.gnu.org/software/parallel/parallel.html#examples
> 
> The tool can handle most variants of batch processing scenarios without
> the steep learning curve of fully fledged (clustered) job schedulers.

Thanks, this seems to do what I need it to do.

Re: [gentoo-user] Why has genkernel initramfs changed behaviour!?

[J. Roeleveld]

On Thursday, November 18, 2021 11:31:57 AM CET Neil Bothwick wrote:
> On Thu, 18 Nov 2021 09:12:41 +0100, J. Roeleveld wrote:
> > I've never been happy with genkernel and roll my own using a
> > config-file that's configured directly into the kernel.
> > All it needs to do is enable LVM, mount the partitions and then
> > continue with booting. On my laptop, it also decrypts the partition
> > that contains LVM. All very simple and easy.
> 
> I used to do the same, building the initramfs into the kernel, avoiding
> the need for a separate file. But now I use dracut, which does it all for
> me without the need for me to keep on top it.

My script is simpler and faster than what dracut or genkernel come up with. I 
also only need to enter a decryption password once, with dracut and genkernel, 
that is not (easily?) to set up.

I actually did the opposite as I got fed up with trying to get dracut and 
genkernel to produce something that would actually boot properly without too 
many questions during boot.

--
Joost

Re: [gentoo-user] Why has genkernel initramfs changed behaviour!?

[J. Roeleveld]

On Thursday, November 18, 2021 12:15:30 AM CET Wol wrote:
> Just filed bug 824282.
> 
> In the past, I've always done "make kernel, make kernel_modules, make
> install, make modules_install, genkernel initramfs ...".
> 
> This worked fine, and I then ran grub-mkconfig, sorted out grub.cfg, and
> all was well.
> 
> My new setup, I have a /boot WHICH I WANT TO SORT OUT MYSELF! I got
> thoroughly confused because genkernel was finding /boot in fstab,
> mounting it by default, and sticking the initramfs there. So of course,
> grub-mkconfig screwed up because the kernel was in the /boot directory,
> but the initramfs was in the /boot partition!
> 
> So I told genkernel not to mount the boot partition ...
> 
> WAH WAH WAH FATAL ERROR YOU WON'T LET ME MOUNT BOOT SULK SULK SULK.
> 
> If I tell it not to mount boot then that's my lookout, not for genconfig
> to nanny me and sulk!
> 
> 
> And it gets worse. I've always done "make modules_install, genkernel
> initramfs". Which now seems to be an unsupported option. genkernel is
> now looking in /var/tmp/genkernel/... for the modules - no surprise the
> modules aren't there! The error says "did you forget to compile the
> kernel" - no I didn't - it is compiled, the modules are installed, I
> just didn't use genkernel to do it.
> 
> Why oh why does everything change ... for the worse ... now let's see if
> allowing it to mount the boot partition makes it work properly ...
> 
> and allowing it to mount boot made everything work perfectly afaict ...
> what a mess ...

I've never been happy with genkernel and roll my own using a config-file that's 
configured directly into the kernel.
All it needs to do is enable LVM, mount the partitions and then continue with 
booting. On my laptop, it also decrypts the partition that contains LVM. All 
very simple and easy.

For this reason, I'm also not happy with the mandatory dependency on "sys-
kernel/installkernel-gentoo"

--
Joost

Re: [gentoo-user] [OT] DisplayPort KVM switch advice please

[J. Roeleveld]

On Tuesday, November 16, 2021 4:53:12 PM CET Peter Humphrey wrote:
> Hello list,
> 
> Can anyone recommend a reliable KVM switch for DisplayPort with USB keyboard
> and mouse? I've tried a couple of devices over the past few years, but they
> both failed - or at least they appeared to fail, in subtle ways.
> 
> I have two PC workstations and an Intel NUC box to connect to my 1920 x 1080
> 60Hz monitor. The PCs have DP outputs and I can connect the NUC via an
> adapter cable. I don't want to route sound through the switch, and of
> course it must not intercept control-key sequences but pass them on intact.
> 
> Any advice or experiences welcome.

No real advice/experience as I'm looking for one myself.

I've been looking for one that also keeps the signal to the device active (so 
the connected system doesn't think the display disappeared) unlike the 
mechanical switched ones I used in the past.

One vendor that had some good reviews is "ConnectPro", but they're not cheap 
and often out-of-stock. For your list, the 4-port single should be ok and 
currently is in stock:
https://connectpro.com/product/udp-14ap-kit-4-port-usb-displayport-kvm

If anyone has experience with these (or others) please share.

--
Joost

Re: [gentoo-user] deactivate (bluetooth) hardware based on irq/mac address

[J. Roeleveld]

On Saturday, July 3, 2021 1:59:24 PM CEST Tamer Higazi wrote:
> Hi
> 
> The problem is that the bluetooth circuit seems to be damaged, as I have
> recently the same result on Windows (not only on linux.
> 
> At Windows, I can deactivate the hardware at the "device manager", I
> want this the same to be done on Linux.
> 
> Not the driver, just to ignore the hardware.
> Why do I ask this ?
> 
> I ordered now a new bluetooth 5 stick, what if it uses the same driver ?
> So loading the driver should not be suppressed more the hardware should
> be ignored

Can't you disable the onboard bluetooth in the BIOS?

I managed to do that on my system and it's not detected during boot/use of 
either OS.

--
Joost

Re: [gentoo-user] system.map file in /boot. How to manage?

[J. Roeleveld]

On Saturday, July 3, 2021 1:54:13 AM CEST Dale wrote:
> Daniel Frey wrote:
> > On 6/30/21 11:59 PM, Dale wrote:
> >> Howdy,
> >> 
> >> The subject line pretty much describes this.  How does one manage the
> >> system.map file in /boot?  Is it needed?  Should it be updated with each
> >> kernel?  I tend to keep 2 to 3 kernels installed.  I tend to keep 2 that
> >> I know are stable and one testing.  After a while, I may remove the
> >> oldest one and only have two, just in case.  Should I version the
> >> system.map file the same as kernels?  Does just one with no version get
> >> the job done?  Update the file with each kernel upgrade or install one
> >> and done?
> >> 
> >> While at it, what does it even do?  If it needs it, it doesn't matter
> >> but just curious.
> >> 
> >> Thanks for any tips on this.
> >> 
> >> Dale
> >> 
> >> :-)  :-)
> > 
> > I never copy it over unless I have some kernel panic (so not for well
> > over a decade.) So there's nothing for me to manage (I only copy the
> > kernel and kernel config to /boot.)
> > 
> > Dan
> 
> So if it isn't there or something, it isn't going to break anything. 
> That's good to know too. 

I only copy the kernel image (and initrd if required)
I haven't done anything with the System.map or config (apart from keeping it 
updated for compiling the kernel) in over a decade.

My boot-partition isn't even mounted unless I update the kernel, so a file 
there wouldn't even be visible to the system.

--
Joost

Re: [gentoo-user] [OT] Using an odd number of drives in ZFS RaidZ

[J. Roeleveld]

On Thursday, July 1, 2021 3:47:08 PM CEST Robert David wrote:

> In any of my data arrays I have long time migrated off the RAIDZ to the
> MIRROR or RAID10. You will find finally that the RAIDZ is slow and not
> very flexible. Only think you gain is the extra space in constrained
> array spaces. For RAID10 it is much easier to raise the size, just
> resilvering to new bigger disks, removing old and expanding. The
> resilvering speed is magnitude faster. And anyway much easier to recover
> in cases of failure.

multiple RAIDZ2 vdevs with a fast enough I/O can easily saturate multiple 
10Gbit links. I actually have 2 pools in my system, one is using tripple-
mirrored VDEVs, the other 6-disk RAIDZ2 sets.

Both are easily capable of saturating the 10gbit link I use.

--
Joost

Re: [gentoo-user] [OT] Using an odd number of drives in ZFS RaidZ

[J. Roeleveld]

On Thursday, July 1, 2021 3:29:03 AM CEST William Kenworthy wrote:
> On 1/7/21 7:31 am, Frank Steinmetzger wrote:
> > Am Wed, Jun 30, 2021 at 09:45:13PM +0100 schrieb Neil Bothwick:
> >> On Tue, 29 Jun 2021 15:56:49 +0200, Frank Steinmetzger wrote:
> >>> I reached 80 % usage (which is the recommended maximum for ZFS) and am
> >>> now evaluating my options for the coming years.
> >>> ...
> 
> Are you welded to ZFS? Is BTRFS or another alternative viable as it
> might handle the different drive sizes more elegantly? (e.g., btrfs raid
> handles different sized disks quite well)

Last I checked, BTRFS doesn't have any RAID5/6 equivalent.
And mirrored has too much storage-loss.

--
Joost

Re: [gentoo-user] is "scp" reliable?

[J. Roeleveld]

On Tuesday, June 1, 2021 6:21:52 PM CEST n952162 wrote:
> On 6/1/21 6:42 AM, J. Roeleveld wrote:
> > If there are differences, I would definitely suspect memory and CPU.
> > 
> > --
> > Joost
> 
> CPU?  USB was mentioned which set off alarm bells for me.  In general
> though, I would suspect the media  - either source of destination.

The issue was first seen using SCP-copies. Eg, over network.
USB was only mentioned as being used as an alternative, which also didn't 
work.

--
Joost

Re: [gentoo-user] Issues with Python 3.8?

[J. Roeleveld]

On Thursday, June 3, 2021 10:53:45 AM CEST Adam Carter wrote:
> On Thu, Jun 3, 2021 at 6:19 PM J. Roeleveld  wrote:
> > All,
> > 
> > I've got several packages complaining about "PYTHON_TARGETS" missing
> > "python3_8"
> 
> You should add --changed-use to your emerge flags and clean any
> PYTHON_TARGETS cruft out of package.use. On ~arch the default python has
> just changed from 3.8 to 3.9. You may well find --depclean will remove 3.8
> after that.

I have no "python_target" entries anywhere under " /etc/portage/ " (apart from 
what I have to add now to get the upgrade to actually work).

--
Joost

[gentoo-user] Issues with Python 3.8?

[J. Roeleveld]

All,

I've got several packages complaining about "PYTHON_TARGETS" missing 
"python3_8":

  The following REQUIRED_USE flag constraints are unsatisfied:
python_targets_python3_8

  The above constraints are a subset of the following complete expression:
any-of ( python_targets_python3_8 )

I saw the news-item, but it doesn't specify what to do with packages that only 
list 3.8 and occasionally 3.7.

Thanks,

Joost

Re: [gentoo-user] app-misc/ca-certificates

[J. Roeleveld]

On Wednesday, June 2, 2021 12:28:49 AM CEST Fannys wrote:
> On June 1, 2021 4:45:45 AM UTC, "J. Roeleveld"  wrote:
> >On Saturday, May 29, 2021 8:26:57 AM CEST Walter Dnes wrote:
> >> On Sat, May 29, 2021 at 03:08:39AM +0200, zca...@gmail.com wrote
> >> 
> >> > 125 config files in /etc/ssl/certs needs update.
> >> > 
> >> > For certificates I would expect the old and invalid ones to be
> >
> >replaced
> >
> >> > by newer ones without user intervention.
> >> > 
> >>   Looking through them is "interesting".  There seem to be a lot of
> >> 
> >> /etc/ssl/certs/.0 files, where "?" is either a random number
> >
> >or
> >
> >> a lower case letter.  These all seem to be symlinks to
> >> /etc/ssl/certs/.pem.  Each of those files is in turn a
> >> symlink to /usr/share/ca-certificates/mozilla/.crt.  How
> >
> >much
> >
> >> do we trust China?  There are a couple of certificates in there named
> >> /usr/share/ca-certificates/mozilla/Hongkong_Post_Root_CA_1.crt  and
> >> /usr/share/ca-certificates/mozilla/Hongkong_Post_Root_CA_3.crt.  Any
> >> other suspicious regimes in there?
> >
> >I've always wondered about the amount of CAs that are auto-trusted on
> >any
> >system. Including several from countries with serious human rights
> >issues.
> >
> >I could do with a tool where I can easily select which CAs to trust
> >based on
> >country.
> >
> >--
> >Joost
> 
> Is there actually any tool that can let me pick my certificates?
> If i go and start deleting randomly certificates from regimes i dont like
> will there be any "breaking change"? I suppose firefox uses its own
> certificate store though.

If the CA is removed from your system/app/..., any key signed by that CA will 
be seen as "untrusted" (treated as if self-signed) and you need to go through 
the usual hoops to allow that certificate to be used.

--
Joost

Re: [gentoo-user] app-misc/ca-certificates

[J. Roeleveld]

On Wednesday, June 2, 2021 3:51:06 AM CEST Grant Taylor wrote:
> On 6/1/21 3:38 PM, Michael Orlitzky wrote:
> > All browsers will treat their fake certificate corresponding to the
> > fake key on their fake web server as completely legitimate. The "real"
> > original key that you generated has no special technical properties
> > that distinguish it.
> 
> Not /all/ browsers.  I know people that have run browser extensions to
> validate the TLS certificate that they receive against records published
> via DANE in DNS, which is protected by DNSSEC.  So it's effectively
> impossible for a rogue CA and malicious actor to violate that chain of
> trust in a way that can't be detected and acted on.

Do you know which extensions add this?

Re: Letsencrypt (was Re: [gentoo-user] app-misc/ca-certificates)

[J. Roeleveld]

On Tuesday, June 1, 2021 12:44:47 PM CEST k...@aspodata.se wrote:
> BillK:
> ...
> 
> > And another "wondering" - all the warnings about trusting self signed
> > certs seem a bit self serving. Yes, they are trying to certify who you
> > are, but at the expense of probably allowing access to your
> > communications by "authorised parties" (such as commercial entities
> > purchasing access for MITM access - e.g. certain router/firewall
> > companies doing deep inspection of SSL via resigning or owning both end
> > points). If its only your own communications and not with a third,
> > commercial party self signed seems a lot more secure.
> 
> ...
> 
> You can use https://letsencrypt.org/ instead of a self-signed cert:
> 
>  Let's Encrypt is a free, automated, and open certificate authority
>  brought to you by the nonprofit Internet Security Research Group (ISRG).
> 
> It was pretty simple to get it to work with
>  https://github.com/diafygi/acme-tiny

It's not that easy to do it with internal-only systems as Let's Encrypt 
requires the hostname to be known externally.
And there are plenty of devices you do not want the whole internet to know 
about.

--
Joost

Re: [gentoo-user] Qustions re Dell M.2 PCIe NVMe Solid State Drives under Gentoo

[J. Roeleveld]

On Thursday, May 27, 2021 11:35:33 PM CEST Alarig Le Lay wrote:
> On Thu 27 May 2021 17:05:07 GMT, Walter Dnes wrote:
> > * do NVMe drives function well under Gentoo (driver issues, etc)?
> 
> I have a NVMe drive on my corporate laptop (a lenovo) and I don’t have
> any issue with it.
> 
> PS: I agree on the linux bloatiness, I have 4G of RAM on my personal
> laptop and it begins to run out sometime…

Multiple machines with NVMe's, no issues.

Not all can boot from NVMe though, that's a BIOS-issue.

Re: [gentoo-user] is "scp" reliable?

[J. Roeleveld]

On Saturday, May 29, 2021 11:04:44 PM CEST Mark Knecht wrote:
> On Sat, May 29, 2021 at 1:33 PM  wrote:
> 
> 
> > Another mystery.
> > I copied the file to USB 1TB sandisk.
> > md5sum check OK same as my computer
> > 
> > 

> Different revisions of md5sum possibly?

I have never had issues with different md5sum tools.
I often use md5sum along with sha1sum to check file-integrity of downloaded 
files. The checksums being provided by the source.

If there are differences, I would definitely suspect memory and CPU.

--
Joost

Re: [gentoo-user] app-misc/ca-certificates

[J. Roeleveld]

On Saturday, May 29, 2021 8:26:57 AM CEST Walter Dnes wrote:
> On Sat, May 29, 2021 at 03:08:39AM +0200, zca...@gmail.com wrote
> 
> > 125 config files in /etc/ssl/certs needs update.
> > 
> > For certificates I would expect the old and invalid ones to be replaced
> > by newer ones without user intervention.
> 
>   Looking through them is "interesting".  There seem to be a lot of
> /etc/ssl/certs/.0 files, where "?" is either a random number or
> a lower case letter.  These all seem to be symlinks to
> /etc/ssl/certs/.pem.  Each of those files is in turn a
> symlink to /usr/share/ca-certificates/mozilla/.crt.  How much
> do we trust China?  There are a couple of certificates in there named
> /usr/share/ca-certificates/mozilla/Hongkong_Post_Root_CA_1.crt  and
> /usr/share/ca-certificates/mozilla/Hongkong_Post_Root_CA_3.crt.  Any
> other suspicious regimes in there?

I've always wondered about the amount of CAs that are auto-trusted on any 
system. Including several from countries with serious human rights issues.

I could do with a tool where I can easily select which CAs to trust based on 
country.

--
Joost

Re: [gentoo-user] one machine out of many has nothing to update

[J. Roeleveld]

On Thursday, May 6, 2021 9:40:21 AM CEST n952162 wrote:
> On 5/6/21 7:42 AM, n952162 wrote:
> > On 5/6/21 7:30 AM, n952162 wrote:
> >> On 5/5/21 8:24 PM, Michael Orlitzky wrote:
> >>> On Wed, 2021-05-05 at 20:11 +0200, n952162 wrote:
>  Hello.
>  
>  I update several machines every month.  This month, most of the
>  machines
>  needed to update more than 300 packages.  But one, which isn't any
>  different than the others, has 0 to update, after running --sync.
>  
>  Does anybody have an explanation for that?
> >>> 
> >>> The manifest verification failed during emerge --sync and you didn't
> >>> notice it. Happens to me every time.
> >> 
> >> Yes!  It did!  It does, every time!  But why?  I have enough disk
> >> space...
> > 
> > This forum entry from 2018:
> > 
> > https://forums.gentoo.org/viewtopic-t-1085556-start-0.html
> > 
> > 
> > lists various possible causes, all of which involve complicated or
> > 
> > expensive fixes:
> >  1. with "hard-links"
> >  2. without "hard-links"
> >  3. /usr/portage is corrupted (/var/db/repos/gentoo in my case, I guess)
> >  4. rsync-excludes?
> >  5. use a different mirror (but it works for other machines)
> > 
> > Is there a consensus in the mean time?
> 
> Here's the error:
> 
>   * Verifying /var/db/repos/gentoo/.tmp-unverified-download-quarantine
> ...!!! Manifest verification failed:
> Manifest mismatch for metadata/Manifest.gz
>__size__: expected: 3019, have: 3011
> 
> I diffed the two metadata/Manifest.gz files, here as a png to retain the
> spacing.
> 
> I'm not sure what __size__ is being displayed there.

What's the size of the "Manifest.gz" file?
And did you try re-synching?

--
Joost

Re: [gentoo-user] Rusty problems

[J. Roeleveld]

On Tuesday, April 27, 2021 10:53:11 PM CEST Alan Mackenzie wrote:
> Hello, Gentoo.
> 
> I'm having problems building rust.
> 
> I build everything in a ram disk, and last night my 13 Gb ram disk
> proved too small to build rust in.  So I increased its size to 14 Gb,
> and tried again this evening.  Same result.  The pre-check on the disk
> size gave an OK both times, and both runs lasted about 45 minutes before
> running out of space.
> 
> Help!  What am I supposed to do?  I've got 16 Gb RAM (I'm _not_ going to
> use the word "only" here), and wondering just how big a chunk a ram disk
> can take out of that before the machine siezes up altogether.  But if I
> increase the ram disk to 15 Gb, even assuming it'd work, it's only going
> to be a small number of releases before the clever people at rust
> increase their bloat even more.
> 
> I know I could plump for the -bin package.  Maybe I should.  But before
> I do that, is it possible to redirect this one ebuild away from
> /var/tmp/portage (my ram disk) without disturbing the other builds?  If
> so, how would I do this (or where should I look for documentation)?
> 
> Thanks for the help!

I managed to get it to compile with a 16GB ramdisk using "system-llvm":


$ grep -r rust /etc/portage
/etc/portage/profile/package.use.mask:dev-lang/rust -system-llvm
/etc/portage/package.use/rust:dev-lang/rust system-llvm parallel-compiler

It claims it wants:
Checking for at least 13312 MiB

>From the build-log:

==
 * Final size of build directory: 12344956 KiB ( 11.7 GiB)
 * Final size of installed tree:366492 KiB (357.9 MiB)

 * The ebuild is installing to one or more unexpected paths:
 * 
 *   /usr/share/doc/rust-1.51.0
 * 
 * Please fix the ebuild to use correct FHS/Gentoo policy paths.
===

Hope this helps,

Joost

Re: [gentoo-user] IPsec

[J. Roeleveld]

On Monday, April 5, 2021 3:46:37 AM CEST Grant Taylor wrote:
> Hi,
> 
> Does anyone have any experience with IPsec?  Preferably on Gentoo or
> Linux in general?
> 
> I'd like to discuss some things (probably off list) while wading into
> the IPsec pool.  E.g.:
> 
>   - ip xfrm ...
>   - strongSwan
>   - Libraswan
>   - X.509 certificate based authentication, preferably /mutual/
>   - Opportunistic Encryption
>   - Transport Mode
>   - Tunnel Mode

I only managed to get it working between off-the-shelve devices, but would 
prefer to do it from Linux.

Please keep it on the list so I can participate in the process.

--
Joost

Re: [gentoo-user] Console scrollback is back again!

[J. Roeleveld]

On Monday, April 5, 2021 7:12:07 PM CEST Alan Mackenzie wrote:
> Hello, Gentoo.
> 
> Yes, console soft scrolling is back!  That essential feature that was
> stripped out of the kernel at around 5.4.x has returned!
> 
> Only this time, it's even better!  Instead of one scrollback buffer
> shared between all tty's, there's now a buffer for each tty.
> 
> How to get it working:
> (i) Extract the enclosed patch file to your /usr/src/linux-5.10.27-gentoo
> directory (or probably any 5.10 kernel's).
> (ii) From that directory run $ patch -p0 < diff.20210405.diff.
> (iii)(a) If you've already got a working 5.10.27, do a $ make oldconfig.
>   That should bring up the new configuration items, for which you can
>   accept the defaults.  These two items are a flag to enable the
>   scrollback, and a buffer size defaulting to 128kB.
> (iii)(b) If you haven't yet got a 5.10.27, just configure your kernel in
>   the usual way.  The two new items are under Device Drivers/Graphics
>   support/Console display driver support.
> (iv) Build the kernel.
> (v) Put the new kernel into your usual boot manager.
> (vi) Reboot and enjoy!
> 

Thank you!

I will add this to my "/etc/portage/patches/sys-kernel/gentoo-sources" 
directory. Right next to the gpl-defang patch.

Re: [gentoo-user] root on nfs and multiple ip addresses

[J. Roeleveld]

On Wednesday, March 17, 2021 7:31:52 AM CET William Kenworthy wrote:
> Hi,
> 
> I have just reinstalled my 32 bit Gentoo on a raspberry pi 3B with a
> Gentoo aarch64 image also with boot on an sdcard and root on nfs (both
> working fine with boot on an sdcard and root on an nfs share hosted on
> an moosefs cluster).  Both have a problem where the initial boot loader
> requests an IP address using "IP=dhcp", then the main operating system
> requests it again on initialising the interface.  Despite asking using
> the same MAC address, ISC dhcp issues a new and different IP address -
> this seems counter intuitive so why is it happening?  Have I missed a
> configuration option somewhere?
> 
> The standard advice is to stop the OS requesting an address (Ubuntu and
> the like) - but shouldn't dhcp know that its already issued a valid IP
> address to that MAC address?
> 
> BillK

Normally I get the same IP, but not if the lease is expired:

What is the "default-lease-time" and "max-lease-time" set to on your DHCP 
server?

Or it gets "released", which can happen when the client informs the DHCP 
server it no longer needs the IP.

Also, why not fix the IP for the MAC on the DHCP server?

Re: [gentoo-user] routine maintenance.

[J. Roeleveld]

On Thursday, February 18, 2021 12:10:45 PM CET Peter Humphrey wrote:
> On Thursday, 18 February 2021 08:20:54 GMT Hund wrote:
> > A SSD is just fine. You're not gaining any performance with a M.2 disk
> > anyway.
> 
> Sorry, but that just isn't true. The difference is dramatic. I speak from
> experience.

Provided the M.2 is using NVME instead of SATA

And also decent specced. (I've seen badly specced NVME models)

Re: [gentoo-user] network transfer speed

[J. Roeleveld]

On Monday, January 18, 2021 8:13:24 PM CET the...@sys-concept.com wrote:
> On 1/18/21 4:41 AM, bobwxc wrote:
> > 在 2021/1/18 下午6:19, J. Roeleveld 写道:
> >> On Monday, January 18, 2021 10:58:24 AM CET Raffaele BELARDI wrote:
> >>>> -Original Message-
> >>>> From: J. Roeleveld 
> >>>> Sent: Monday, January 18, 2021 09:47
> >>>> To: gentoo-user@lists.gentoo.org
> >>>> Subject: Re: [gentoo-user] network transfer speed
> >>>> 
> >>>> Some cheaper switches fail-over to hub-mode when the traffic exceeds
> >>>> what it can manage.
> >>> 
> >>> Interesting, do you have pointers to such switches' specs? Or is it from
> >>> experience?
> > 
> > That's true.
> > Some cheap switches may become like a network cable when it fails or
> > exceeds network flow. And there is also no log to check, will cause some
> > strange problems.
> > 
> > Actually , some business machines also have such a policy that they become
> > straight through or completely blocked in case of failure. But they have
> > a fail log for review.> 
> >> This is from experience, also 1 of the reasons why I switched to managed
> >> switches.
> 
> What switch would you folks recommend?
> I'm planning getting to the bottom of it including replacing switch and
> cable if I have to.

I currently use a mixture of:
GS716T and GS108TV3 Netgear switches without issues and manage full 1Gb 
transfer rates between any 2 systems.

--
Joost

Re: [gentoo-user] network transfer speed

[J. Roeleveld]

On Monday, January 18, 2021 10:58:24 AM CET Raffaele BELARDI wrote:
> > -Original Message-
> > From: J. Roeleveld 
> > Sent: Monday, January 18, 2021 09:47
> > To: gentoo-user@lists.gentoo.org
> > Subject: Re: [gentoo-user] network transfer speed
> > 
> > Some cheaper switches fail-over to hub-mode when the traffic exceeds
> > what it can manage.
> 
> Interesting, do you have pointers to such switches' specs? Or is it from
> experience?

This is from experience, also 1 of the reasons why I switched to managed 
switches.

--
Joost

Re: [gentoo-user] network transfer speed

[J. Roeleveld]

On Monday, January 18, 2021 8:57:38 AM CET Raffaele BELARDI wrote:
> > -Original Message-
> > From: Jack 
> > Sent: Saturday, January 16, 2021 22:00
> > To: gentoo-user@lists.gentoo.org
> > Subject: Re: [gentoo-user] network transfer speed
> > 
> > I may be way off base here, but if the switch is connected to a router,
> > packets from one PC go to the switch and then to everything else connected
> > to it, including both the other PC and the router.  Is there any chance
> > the
> > router is passing packets back to the switch to get to the second PC?  I
> > can imagine that causing lots of problems.  However, I would hope it is
> > smart enough to know it doesn't need to do so, since both PCs show up on
> > the same router port.
> 
> A switch uses the Ethernet MAC destination address to forward a packet only
> on the 'interested' ports. What you describe would be a 'hub' [1], I don't
> think it's easy to find one of those on recent networks.
> 
> Raffaele
> 
> [1] https://en.wikipedia.org/wiki/Ethernet_hub

Some cheaper switches fail-over to hub-mode when the traffic exceeds what it 
can 
manage.

--
Joost

Re: [gentoo-user] Re: Screen/driver/xserver freezing after suspension

[J. Roeleveld]

On Wednesday, January 13, 2021 3:58:55 PM CET Igor Mróz wrote:
> > Please don't top-post.
> 
> Sorry, trying to change my email writing habit ;)
> 
> > Reason  I asked: On one of my systems, I have the occasional issue where a
> > screenlock-task is "hanging" and using 100% CPU.
> > Killing that, restores the screen back to normal.
> 
> Unfortunately there are no signs of app/driver crash as well as any other
> system malfunction.
> 
> Igor

Then I am not certain.
There is, however, one big difference:
I am running "stable" (amd64) on my laptop.
You appear to be running "testing" (~amd64)

Anything there might be causing the issue you are seeing, but I am unable to 
check if that is actually the case.

Re: [gentoo-user] Re: Screen/driver/xserver freezing after suspension

[J. Roeleveld]

Please don't top-post.

On Wednesday, January 13, 2021 11:32:44 AM CET Igor Mróz wrote:
> You mean 100%? Nope, all looks normal. I even tried to build elogind with
> policykit USE, but with no luck. loginctl shows that session/seat is active
> all the time.
> 
> Igor
> 
> 
> On Mon, 11 Jan 2021 20:05:20 +0100
> 
> "J. Roeleveld"  wrote:
> > On 11 January 2021 19:33:55 CET, "Igor Mróz"  wrote:
> > >I don't even know where to search.
> > >
> > >I tried to SSH to 'frozen' laptop, but as I wrote earlier - there is
> > >nothing suspicious in dmesg or xorg logs. Only difference is in 'ps'
> > >i.e. process '[kworker/u16:0-i915]' changed to
> > >'[kworker/u16:0-events_unbound]' and some other kworkers received new
> > >PIDs (like those have been restarted).
> > >
> > >Igor
> > >
> > >
> > >
> > >On Mon, 11 Jan 2021 15:01:41 +0100
> > >
> > >"J. Roeleveld"  wrote:
> > >> On Monday, January 11, 2021 2:22:27 PM CET Holger Hoffstätte wrote:
> > >> > On Mon, 11 Jan 2021 13:49:35 +0100, Igor Mróz wrote:
> > >> > > I don't really know - I haven't installed anything. Now I'm not
> > >
> > >really
> > >
> > >> > > sure if this is problem with suspension or just screen resuming
> > >
> > >after
> > >
> > >> > > "turning" it off. I also don't have Nvidia card.
> > >> > > 
> > >> > > Igor
> > >> > 
> > >> > Try running xorg-server with +suid.
> > >> 
> > >> That should not be necessary for suspend/resume.
> > >> I have that flag disabled (as per default) on my laptop and that one
> > >
> > >gets
> > >
> > >> suspended regularly.
> > >> 
> > >> --
> > >> Joost
> > 
> > Anything running at 100÷?
> > 
> > Check "top" when ssh'd in

Reason  I asked: On one of my systems, I have the occasional issue where a 
screenlock-task is "hanging" and using 100% CPU.
Killing that, restores the screen back to normal.

--
Joost

Re: [gentoo-user] preventing some IP's from from being logged in apache

[J. Roeleveld]

On 12 January 2021 18:11:34 CET, the...@sys-concept.com wrote:
>On 1/11/21 11:32 PM, J. Roeleveld wrote:
>[snip]
>> If you blocked the IP in the firewall, it shouldn't show in the log.
>> 
>> Add iptables to the webserver and block that IP.
>> Personally, I would block the whole range (45.93.201.0/24), which is
>either 
>> linked to Cyprus or Russia (according to whois)
>> 
>> Any block in the apache-config might fail due to a bug or new
>configuration.
>> 
>> --
>> Joost
>
>Asus routers don't have capabilities to block external IP's (I need to
>find a better router).
>Blocking that IP range in apache works, all the IPs I blocked are
>getting 403 error (regardless what they type) 
>but you might be right, apache 2.4 
> SetEnvIf Remote_Addr "45\.93\.201\.104" dontlog  (does not work). 
>
>I wish they design blocking by country easier. 

Asus routers do allow blocking IPs, but it might be less than ideal.

I would add a firewall to your webserver and block the full ranges using that. 
Any bug in apaches blocking WILL leave you open to a hack.

--
Joost
-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.