Re: [gentoo-user] Re: Slightly corrupted file systems when resuming from hibernation

[Wols Lists]

On 17/04/2024 10:10, Michael wrote:

I am not sure the assumption "... aging hardware possibly can less and less
cope with newer and newer kernels" is correct.  As already mentioned newer
kernels have both security and bug fixes.  As long as you stick with stable
gentoo-sources you'll have these in your system.  Later kernels also come with
additional kernel drivers for new(er) hardware.  You may not need/want these
drivers if you do not run the latest hardware. Using 'make oldconfig' allows
you to exclude such new drivers, but include new security options and/or
functionality as desired.


Given that I remember the announcement that the linux kernel's memory 
requirements had increased to 6MB - in the days when Fedora et al 
demanded gigabytes simply to be able to run - I think almost any ancient 
hardware you can actually buy will be able to run the linux kernel no probs.


You might have difficulty compiling it, though, now 386 support has been 
pretty much dropped from the toolchain. Have they dropped i686 as well?


Cheers,
Wol

Re: [gentoo-user] NAS box and switching from Phenom II X6 1090T to FX-6300

[Wols Lists]

On 13/04/2024 14:23, Dale wrote:

I see lots of mobos with those little hard drives on a stick.  I think
they called NVME or something, may have spelling wrong.  For most
people, that is likely awesome.  For me, I think I'd be happy with a
regular SSD.  Given that, I'd like them to make a mobo where one can say
cut off/disable that NVME thing and make use of that "lane" as a PCIe
slot(s).  Even if that means having a cable that hooks to the mobo and
runs elsewhere to connect PCIe cards.  In other words, have one slot
that is expandable to say three or four slots with what I think is
called a back-plane.  That way if a user wants to use the NVME thing,
they can.  If they don't, they can disable it and go another route with
PCIe expansion.  Sort of reminds me of that SAS drive thing.  You have
one cable that branches out into several SATA drives or SAS drives
themselves.  I don't know a lot about SAS really.  May have to read up
on that with that new case that holds 18 drives tho.  O_o


Read up on your mobo.

Certainly on mine, that thing where you can use NVME *OR* OCIe *OR* Sata 
is a thing on mine.


There's a nice table in the mobo manual that tells you which combos are 
supported.


Cheers,
Wol

Re: [gentoo-user] Successfully upgraded to new profile 23.0

[Wols Lists]

On 08/04/2024 15:03, Dr Rainer Woitok wrote:

the upgrade on my old laptop  with two 2.7GHz  Dual-Core Skylake proces-
sors took slightly  more than 2 hours  for the manual upgrading of "bin-
utils", "gcc" and "glibc", and slightly more than 21.5 hours for the fi-
nal upgrade of "@world",  which had to process a total of 1061 packages.
I'm wondering whether  a fresh install  from a stage 3  "tar" ball would
have been faster?


Some 1500 plus packages here - took about 2 days on my 4-core Ryzen ...

Btw, where are all the messages for packages stored? I ought to go 
through them and make sure there aren't any messages of interest... I 
know I ought to update my kernel ...


Cheers,
Wol

Re: [gentoo-user] New profile - gentoo and binutils ...

[Wols Lists]

On 07/04/2024 16:08, Michael wrote:

Cool, once your system is up to date you should be able to change your profile
and follow the rest of the instructions.  I hope all goes well.  


emerge --emptytree is now running well - 122 of 1534 so it has some way 
to go ...


Cheers,
Wol

Re: [gentoo-user] New profile - gentoo and binutils ...

[Wols Lists]

On 07/04/2024 15:46, Wols Lists wrote:

On 07/04/2024 13:07, Michael wrote:

On Sunday, 7 April 2024 12:04:32 BST Wols Lists wrote:

On 07/04/2024 11:48, Wols Lists wrote:

On 07/04/2024 11:23, Michael wrote:

On Sunday, 7 April 2024 11:21:00 BST Wols Lists wrote:

On 07/04/2024 11:00, Wols Lists wrote:

What do I do here - "emerge binutils" (step 9) wants to emerge gcc,
which the instructions say "emerge AFTER binutils".

With gcc it says "don't let it emerge glibc", should I apply the 
same

logic and not let binutils emerge gcc?


Just to follow up to myself, I've just done a complete update, but 
a lot
of the dependencies are pulled in by "change-use", namely lzma, 
zstd. Is

that fallout from the XZ debacle? Would a --no-deps be safe?

Cheers,
Wol


Are you still on your original profile, or have you used eselect to
change it
to profile 23.0?

If the latter, change back to your old profile, update @world,
depclean and
then start with the rest of the migration instructions.


Just done that. See my other email.

NOTHING TO UPDATE (unless I've messed up my emerge ...)


Interesting ... just done this under the old profile ...

thewolery /usr/local/bin # emerge --ask --oneshot sys-devel/binutils

These are the packages that would be merged, in order:

Calculating dependencies... done!
Dependency resolution took 3.19 s (backtrack: 0/20).


Nothing to merge; quitting.

Cheers,
Wol



Hmm ... something is amiss with your system.  Normally you would get 
this:


# emerge --ask --oneshot sys-devel/binutils

These are the packages that would be merged, in order:

Calculating dependencies... done!
Dependency resolution took 1.30 s (backtrack: 0/20).

[ebuild   R    ] sys-devel/binutils-2.41-r5

Would you like to merge these packages? [Yes/No]

Did you emerge any packages using the new 23.0 profile, then went back 
to the

old profile to run the above command?


No ...

Ummm ... I have had trouble emerging other stuff that didn't want to 
work with oneshot ...


Let me look at my make.conf - I had to over-ride something to get 
vbox-modules to emerge, this is probably the same thing ...


Yup - as soon as I comment the EMERGE_DEFAULT_OPTS out I get it asking 
me if I want to emerge binutils.


# EMERGE_DEFAULT_OPTS = "--buildpkg --deep --newuse --oneshot --usepkg"

So I'm doing an emerge @world now with the old profile ...

Yup - this appears to have un-bunged it - it's working as per 
instructions. You might want to add to the instructions to disable 
anything in make.conf that tampers with default settings.


Cheers,
Wol

Re: [gentoo-user] New profile - gentoo and binutils ...

[Wols Lists]

On 07/04/2024 13:07, Michael wrote:

On Sunday, 7 April 2024 12:04:32 BST Wols Lists wrote:

On 07/04/2024 11:48, Wols Lists wrote:

On 07/04/2024 11:23, Michael wrote:

On Sunday, 7 April 2024 11:21:00 BST Wols Lists wrote:

On 07/04/2024 11:00, Wols Lists wrote:

What do I do here - "emerge binutils" (step 9) wants to emerge gcc,
which the instructions say "emerge AFTER binutils".

With gcc it says "don't let it emerge glibc", should I apply the same
logic and not let binutils emerge gcc?


Just to follow up to myself, I've just done a complete update, but a lot
of the dependencies are pulled in by "change-use", namely lzma, zstd. Is
that fallout from the XZ debacle? Would a --no-deps be safe?

Cheers,
Wol


Are you still on your original profile, or have you used eselect to
change it
to profile 23.0?

If the latter, change back to your old profile, update @world,
depclean and
then start with the rest of the migration instructions.


Just done that. See my other email.

NOTHING TO UPDATE (unless I've messed up my emerge ...)


Interesting ... just done this under the old profile ...

thewolery /usr/local/bin # emerge --ask --oneshot sys-devel/binutils

These are the packages that would be merged, in order:

Calculating dependencies... done!
Dependency resolution took 3.19 s (backtrack: 0/20).


Nothing to merge; quitting.

Cheers,
Wol



Hmm ... something is amiss with your system.  Normally you would get this:

# emerge --ask --oneshot sys-devel/binutils

These are the packages that would be merged, in order:

Calculating dependencies... done!
Dependency resolution took 1.30 s (backtrack: 0/20).

[ebuild   R] sys-devel/binutils-2.41-r5

Would you like to merge these packages? [Yes/No]

Did you emerge any packages using the new 23.0 profile, then went back to the
old profile to run the above command?


No ...

Ummm ... I have had trouble emerging other stuff that didn't want to 
work with oneshot ...


Let me look at my make.conf - I had to over-ride something to get 
vbox-modules to emerge, this is probably the same thing ...


Yup - as soon as I comment the EMERGE_DEFAULT_OPTS out I get it asking 
me if I want to emerge binutils.


# EMERGE_DEFAULT_OPTS = "--buildpkg --deep --newuse --oneshot --usepkg"

So I'm doing an emerge @world now with the old profile ...

Cheers,
Wol

Re: [gentoo-user] New profile - gentoo and binutils ...

[Wols Lists]

On 07/04/2024 11:48, Wols Lists wrote:

On 07/04/2024 11:23, Michael wrote:

On Sunday, 7 April 2024 11:21:00 BST Wols Lists wrote:

On 07/04/2024 11:00, Wols Lists wrote:

What do I do here - "emerge binutils" (step 9) wants to emerge gcc,
which the instructions say "emerge AFTER binutils".

With gcc it says "don't let it emerge glibc", should I apply the same
logic and not let binutils emerge gcc?


Just to follow up to myself, I've just done a complete update, but a lot
of the dependencies are pulled in by "change-use", namely lzma, zstd. Is
that fallout from the XZ debacle? Would a --no-deps be safe?

Cheers,
Wol


Are you still on your original profile, or have you used eselect to 
change it

to profile 23.0?

If the latter, change back to your old profile, update @world, 
depclean and

then start with the rest of the migration instructions.


Just done that. See my other email.

NOTHING TO UPDATE (unless I've messed up my emerge ...)


Interesting ... just done this under the old profile ...

thewolery /usr/local/bin # emerge --ask --oneshot sys-devel/binutils

These are the packages that would be merged, in order:

Calculating dependencies... done!
Dependency resolution took 3.19 s (backtrack: 0/20).


Nothing to merge; quitting.

Cheers,
Wol

Re: [gentoo-user] New profile - gentoo and binutils ...

[Wols Lists]

On 07/04/2024 11:23, Michael wrote:

On Sunday, 7 April 2024 11:21:00 BST Wols Lists wrote:

On 07/04/2024 11:00, Wols Lists wrote:

What do I do here - "emerge binutils" (step 9) wants to emerge gcc,
which the instructions say "emerge AFTER binutils".

With gcc it says "don't let it emerge glibc", should I apply the same
logic and not let binutils emerge gcc?


Just to follow up to myself, I've just done a complete update, but a lot
of the dependencies are pulled in by "change-use", namely lzma, zstd. Is
that fallout from the XZ debacle? Would a --no-deps be safe?

Cheers,
Wol


Are you still on your original profile, or have you used eselect to change it
to profile 23.0?

If the latter, change back to your old profile, update @world, depclean and
then start with the rest of the migration instructions.


Just done that. See my other email.

NOTHING TO UPDATE (unless I've messed up my emerge ...)

Cheers,
Wol

Re: [gentoo-user] New profile - gentoo and binutils ...

[Wols Lists]

On 07/04/2024 11:15, Michael wrote:

On Sunday, 7 April 2024 11:00:49 BST Wols Lists wrote:

What do I do here - "emerge binutils" (step 9) wants to emerge gcc,
which the instructions say "emerge AFTER binutils".

With gcc it says "don't let it emerge glibc", should I apply the same
logic and not let binutils emerge gcc?

Cheers,
Wol


Step 1:

Ensure your system backups are up to date. Please also update your system
fully and depclean before proceeding.

Have you done this already after a fresh rsync of portage?


Note that my current profile is marked experimental ...

  [9]   default/linux/amd64/17.1/desktop/plasma/systemd/merged-usr (exp)

and I also have no stable 20 or 22 profiles to upgrade to.

And "emerge --changed-use" gives me nothing to emerge.

thewolery /usr/local/bin # emerge --update --deep --changed-use --newuse 
@world

Calculating dependencies... done!
Dependency resolution took 21.46 s (backtrack: 0/20).

thewolery /usr/local/bin #


Cheers,
Wol

Re: [gentoo-user] New profile - gentoo and binutils ...

[Wols Lists]

On 07/04/2024 11:15, Michael wrote:

On Sunday, 7 April 2024 11:00:49 BST Wols Lists wrote:

What do I do here - "emerge binutils" (step 9) wants to emerge gcc,
which the instructions say "emerge AFTER binutils".

With gcc it says "don't let it emerge glibc", should I apply the same
logic and not let binutils emerge gcc?

Cheers,
Wol


Step 1:

Ensure your system backups are up to date. Please also update your system
fully and depclean before proceeding.

Have you done this already after a fresh rsync of portage?


YES.

I've printed off the list, and am working my way down it ...

My system defaults to deep, changed use, etc etc. I could revert the 
profile change and try again, we'll see.


Cheers,
Wol

Re: [gentoo-user] New profile - gentoo and binutils ...

[Wols Lists]

On 07/04/2024 11:00, Wols Lists wrote:
What do I do here - "emerge binutils" (step 9) wants to emerge gcc, 
which the instructions say "emerge AFTER binutils".


With gcc it says "don't let it emerge glibc", should I apply the same 
logic and not let binutils emerge gcc?


Just to follow up to myself, I've just done a complete update, but a lot 
of the dependencies are pulled in by "change-use", namely lzma, zstd. Is 
that fallout from the XZ debacle? Would a --no-deps be safe?


Cheers,
Wol

[gentoo-user] New profile - gentoo and binutils ...

[Wols Lists]

What do I do here - "emerge binutils" (step 9) wants to emerge gcc, 
which the instructions say "emerge AFTER binutils".


With gcc it says "don't let it emerge glibc", should I apply the same 
logic and not let binutils emerge gcc?


Cheers,
Wol

Re: [gentoo-user] Every other startup results in a black screen (possibly SDDM related?)

[Wols Lists]

On 03/04/2024 19:53, Jack wrote:
Are you certain it hasn't started on some TTY other than 8?  I always 
start out on TTY1, although I start up text only, no SDDM. However, I do 
have a very vague memory of something similar, and I believe it was that 
I needed to change one of the kernel FB related settings.  Sorry not be 
have more concrete suggestions.


My system isn't configured to switch ttys on - that's something I need 
to investigate and enable. So sddm always starts on 2, with 1 being the 
console output. When I switch user, it switches to 1 and starts the new 
user there.


sddm will always start on the first available number, which with no ttys 
is why it's 2 on my system.


Cheers,
Wol

Re: [gentoo-user] Re: Emerge trouble with firefox and thunderbird ...

[Wols Lists]

On 10/03/2024 22:44, Carsten Hauck wrote:



The CPU of the machine in question is in deed an old AMD. It's good to
know the reason for that build-failures, thanks a lot.
I certainly will stick to "-clang" in my package.use.


Interesting. I'm not at all sure how old my CPU is, but at four cores it 
must be getting on a bit. Sounds like I should do the same ...


Cheers,
Wol

Re: [gentoo-user] Emerge trouble with firefox and thunderbird ...

[Wols Lists]

On 03/03/2024 23:13, Carsten Hauck wrote:

So I don't know what's going on, but basically Mozilla won't emerge,
and I don't know why ...

Cheers,
Wol



Did the other 19 package emerge OK?  Are the mozilla progs crashing
when running, or when emerging?  If emerging, the log is just console
output, as indecipherable as we know it sometimes can be.  If they
crash when running, try running from command line.



Some time ago on one of my machines Thunderbird and Firefox stopped to
compile with USE="clang". As they can be build with gcc I never digged
too deep into that problem but maybe it's worth a shot.


For anyone else who hits this sort of problem, I did an

USE=-clang emerge --update @world

(firefox and thunderbird were the only programs I thought this would 
touch), and it worked.


There were a couple of other programs that I guess got pulled in by the 
changed use, but they've upgraded which is the main thing.


Thank you very much

Cheers,
Wol

Re: [gentoo-user] Emerge trouble with firefox and thunderbird ...

[Wols Lists]

On 04/03/2024 16:20, ralfconn wrote:

Il 03/03/24 10:47, Wols Lists ha scritto:

I'm getting this output from

emerge --update --newuse --deep --with-bdeps=y @world


Calculating dependencies... done!
 * Dependencies could not be completely resolved due to
 * the following required packages not being installed:
 *
 *   >=dev-libs/icu-73.1:0/73.1= pulled in by:
 * www-client/firefox-115.6.0
 *
 * Have you forgotten to do a complete update prior to depclean? The
 * most comprehensive command for this purpose is as follows:
 *
 *   emerge --update --newuse --deep --with-bdeps=y @world
 *
 * Note that the --with-bdeps=y option is not required in many
 * situations. Refer to the emerge manual page (run `man emerge`)
 * for more information about --with-bdeps.
 *
 * Also, note that it may be necessary to manually uninstall
 * packages that no longer exist in the repository, since it may not
 * be possible to satisfy their dependencies.
thewolery /usr/local/bin #

icu is at 74.2

firefox failed to update ...

*  www-client/firefox
  Latest version available: 115.8.0
  Latest version installed: 115.6.0
  Size of files: 496,244 KiB
  Homepage:  https://www.mozilla.com/firefox
  Description:   Firefox Web Browser
  License:   MPL-2.0 GPL-2 LGPL-2.1

as did thunderbird ...

*  mail-client/thunderbird
  Latest version available: 115.8.0
  Latest version installed: 115.6.0
  Size of files: 528,920 KiB
  Homepage:  https://www.thunderbird.net/
  Description:   Thunderbird Mail Client
  License:   MPL-2.0 GPL-2 LGPL-2.1

Andy ideas? Or is the mozilla emerge stuff slightly broken on my 
system? I've been having trouble with those two for the last few weeks 
...


Cheers,
Wol


Here I see:

[I] www-client/firefox
  Available versions:
  (esr)  115.7.0 115.8.0
  (rapid) (~)122.0.1 (~)123.0

You have 115.6.0 installed which apparently is out of tree. That may be 
confusing emerge. You could try to un-merge firefox, depclean and 
re-emerge it.


Hmm ... that sounds like it's been removed from the tree, then, and 
emerge can't cope with stuff disappearing. Surely that's a bug?


Anyways, I'll try and see what happens. I was thinking something of the 
sort, removing firefox that is, seeing how it goes.


I'll probably try the gcc thing first, though ... if that works, then 
it's a simpler "let it sort itself out" approach.


Cheers,
Wol

Re: [gentoo-user] Emerge trouble with firefox and thunderbird ...

[Wols Lists]

On 03/03/2024 09:47, Wols Lists wrote:

I'm getting this output from

emerge --update --newuse --deep --with-bdeps=y @world


whoops I mean "emerge --depclean"

I'm trying to get a clean system, and don't know what exactly is wrong, 
or what to try ...


Cheers,
Wol

[gentoo-user] Emerge trouble with firefox and thunderbird ...

[Wols Lists]

I'm getting this output from

emerge --update --newuse --deep --with-bdeps=y @world


Calculating dependencies... done!
 * Dependencies could not be completely resolved due to
 * the following required packages not being installed:
 *
 *   >=dev-libs/icu-73.1:0/73.1= pulled in by:
 * www-client/firefox-115.6.0
 *
 * Have you forgotten to do a complete update prior to depclean? The
 * most comprehensive command for this purpose is as follows:
 *
 *   emerge --update --newuse --deep --with-bdeps=y @world
 *
 * Note that the --with-bdeps=y option is not required in many
 * situations. Refer to the emerge manual page (run `man emerge`)
 * for more information about --with-bdeps.
 *
 * Also, note that it may be necessary to manually uninstall
 * packages that no longer exist in the repository, since it may not
 * be possible to satisfy their dependencies.
thewolery /usr/local/bin #

icu is at 74.2

firefox failed to update ...

*  www-client/firefox
  Latest version available: 115.8.0
  Latest version installed: 115.6.0
  Size of files: 496,244 KiB
  Homepage:  https://www.mozilla.com/firefox
  Description:   Firefox Web Browser
  License:   MPL-2.0 GPL-2 LGPL-2.1

as did thunderbird ...

*  mail-client/thunderbird
  Latest version available: 115.8.0
  Latest version installed: 115.6.0
  Size of files: 528,920 KiB
  Homepage:  https://www.thunderbird.net/
  Description:   Thunderbird Mail Client
  License:   MPL-2.0 GPL-2 LGPL-2.1

Andy ideas? Or is the mozilla emerge stuff slightly broken on my system? 
I've been having trouble with those two for the last few weeks ...


Cheers,
Wol

Re: [gentoo-user] OT: how does excel find commas within fields of a csv file?

[Wols Lists]

On 28/02/2024 02:17, Jack wrote:

On 2/27/24 20:54, Adam Carter wrote:
To clean up csv files I use excel's find/replace to swap the commas 
occurring within fields for something benign. How does this magic 
work? Different character sets within the same file?


Is it possible to do this with shell scripting?
Once Excel (or LibreOffice) reads in a csv file, the commas are no 
longer present, and it just searches within the cells.  It might be 
possible for a shell script to do it, but you need to parse the file to 
distinguish any commas separating the fields from commas within the 
fields.  I'm sure there are plenty of utilities to do this, but it's 
certainly not trivial.


The other thing is, look up the definition (such as there is) of CSVs. 
Special characters (such as commas) can be quoted. Standard practice as 
far as I can tell, is that any cell containing a comma will be 
double-quoted, and the quotes are stripped on import.


The other trick I learnt is that to prevent Excel mangling text, you 
precede it with a single quote - for example I want eg "+7" in a cell, 
so I have to enter '+7.


Cheers,
Wol

Re: [gentoo-user] Re: How to set up drive with many Linux distros?

[Wols Lists]

On 23/02/2024 00:28, Grant Edwards wrote:

In my experience, 's bootloader does not boot other
installations by calling other bootloaders. It does so by rummaging
through all of the other partitions looking for kernel images, intird
files, grub.cfg files, etc.  It then adds menu entries to the config
file for 's bootloader which, when selected, directly load the
kernel image and initrd from those other partitions. Sometimes, it
works -- at least until those other installations get updated without
the knowlege of the distro that currently "owns" the MBR's bootloader
config. Then it stops working until you tell that bootloader to re-do
it's rummaging about routine.


IME distros that try that (SUSE, anyone!) generally get confused as to 
which kernel belongs to which root partition.


Hence needing to boot with a live distro to edit the resulting mess and 
get the system to actually come up without crashing ...


Cheers,
Wol

Re: [gentoo-user] Re: Suggestions for backup scheme?

[Wols Lists]

On 09/02/2024 12:57, J. Roeleveld wrote:

I don't understand it exactly, but what I think happens is when I create
the snapshot it allocates, let's say, 1GB. As I write to the master
copy, it fills up that 1GB with CoW blocks, and the original blocks are
handed over to the backup snapshot. And when that backup snapshot is
full of blocks that have been "overwritten" (or in reality replaced),
lvm just adds another 1GB or whatever I told it to.



That works with a single snapshot.
But, when I last used LVM like this, I would have multiple snapshots. When I
change something on the LV, the original data would be copied to the snapshot.
If I would have 2 snapshots for that LV, both would grow at the same time.

Or is that changed in recent versions?


Has what changed? As I understand it, the whole point of LVM is that 
everything is COW. So any individual block can belong to multiple snapshots.


When you write a block, the original block is not changed. A new block 
is linked in to the current snapshot to replace the original. The 
original block remains linked in to any other snapshots.


So disk usage basically grows by the number of blocks you write. Taking 
a snapshot will use just a couple of blocks, no matter how large your LV is.



So when I delete a snapshot, it just goes through those few blocks,
decrements their use count (if they've been used in multiple snapshots),
and if the use count goes to zero they're handed back to the "empty" pool.

I know this is how ZFS snapshots work. But am not convinced LVM snapshots work
the same way.


All I have to do is make sure that the sum of my snapshots does not fill
the lv (logical volume). Which in my case is a raid-5.

I assume you mean PV (Physical Volume)?


Quite possibly. VG, PV, LV. I know which one I need (by reading the 
docs), I don't particularly remember which is which off the top of my head.


I actually ditched the whole idea of raid-5 when drives got bigger than 1TB. I
currently use Raid-6 (or specifically RaidZ2, which is the ZFS "equivalent")

Well, I run my raid over dm-integrity so, allegedly, I can't suffer disk 
corruption. My only fear is a disk loss, which raid-5 will happily 
recover from. And I'm not worried about a double failure - yes it could 
happen, but ...


Given that my brother's ex-employer was quite happily running a raid-6 
with maybe petabytes of data, over a double disk failure (until an 
employee went into the data centre and said "what are those red 
lights"), I don't think my 20TB of raid-5 is much :-)


Cheers,
Wol

Re: [gentoo-user] Re: Suggestions for backup scheme?

[Wols Lists]

On 08/02/2024 06:38, J. Roeleveld wrote:

ZFS doesn't have this "max amount of changes", but will happily fill up the
entire pool keeping all versions available.
But it was easier to add zpool monitoring for this on ZFS then it was to add
snapshot monitoring to LVM.

I wonder, how do you deal with snapshots getting "full" on your system?


As far as I'm, concerned, snapshots are read-only once they're created. 
But there is a "grow the snapshot as required" option.


I don't understand it exactly, but what I think happens is when I create 
the snapshot it allocates, let's say, 1GB. As I write to the master 
copy, it fills up that 1GB with CoW blocks, and the original blocks are 
handed over to the backup snapshot. And when that backup snapshot is 
full of blocks that have been "overwritten" (or in reality replaced), 
lvm just adds another 1GB or whatever I told it to.


So when I delete a snapshot, it just goes through those few blocks, 
decrements their use count (if they've been used in multiple snapshots), 
and if the use count goes to zero they're handed back to the "empty" pool.


All I have to do is make sure that the sum of my snapshots does not fill 
the lv (logical volume). Which in my case is a raid-5.


Cheers,
Wol

Re: [gentoo-user] Re: Suggestions for backup scheme?

[Wols Lists]

On 08/02/2024 06:32, J. Roeleveld wrote:

Personally, I'd go the MPL2 route, but that's my choice. It might not
suit you. But to achieve what you want, you need a copyleft, GPL-style
licence.



I'll have a look at that one.


Basically, each individual source file is copyleft, but not the work as 
a whole. So if anybody copies/modifies YOUR work, they have to 
distribute your work with their binary, but this requirement does not 
extend to everyone else's work.


Maybe not included fully into the kernel, but there is nothing preventing it
to be packaged with a Linux distribution.
It's just the hostility from Linus Torvalds and Greg Kroah-Hartman against ZFS
causing the issues.

See the following post for a clear description (much better written than I
can):
https://eerielinux.wordpress.com/2019/01/28/zfs-and-gpl-terror-how-much-freedom-is-there-in-linux/

Especially the lkml thread linked from there:
https://lore.kernel.org/lkml/20190110182413.ga6...@kroah.com/


After all, there's nothing stopping*you*  from combining Linux and ZFS,
it's just that somebody else can't do that for you, and then give you
the resulting binary.



Linux (kernel) and ZFS can't be merged. Fine.


But they can.


But, Linux (the OS, as in, kernel + userspace) and ZFS can be merged legally.


Likewise here, they can.

The problem is, the BINARY can NOT be distributed. And the problem is 
the ZFS licence, not Linux.


What Linus, and the kernel devs, and that crowd *think* is irrelevant. 
What matters is what SUSE, and Red Hat, and Canonical et al think. And 
if they're not prepared to take the risk of distributing the kernel with 
ZFS built in, because they think it's a legal minefield, then that's 
THEIR decision.


That problem doesn't apply to gentoo, because it distributes the linux 
kernel and ZFS separately, and combines them ON THE USER'S MACHINE. But 
the big distros are not prepared to take the risk of combining linux and 
ZFS, and distributing the resulting *derived* *work*.


Cheers,
Wol

Re: [gentoo-user] Re: Suggestions for backup scheme?

[Wols Lists]

On 07/02/2024 11:11, J. Roeleveld wrote:

On Tuesday, February 6, 2024 9:27:35 PM CET Wols Lists wrote:

On 06/02/2024 13:12, J. Roeleveld wrote:

Clearly Oracle likes this state of affairs.  Either that, or they are
encumbered in some way from just GPLing the ZFS code.  Since they on
paper own the code for both projects it seems crazy to me that this
situation persists.


GPL is not necessarily the best license for releasing code. I've got some
private projects that I could publish. But before I do that, I'd have to
decide on a License. I would prefer something other than GPL.


Okay. What do you want to achieve. Let's just lump licences into two
categories to start with and ask the question "Who do you want to free?"


I want my code to be usable by anyone, but don't want anyone to fork it and
start making money off of it without giving me a fair share.


Okay, that instantly says you want a copyleft licence. So you're stuck 
with a GPL-style licence, and if they want to include it in a commercial 
closed source product, they need to come back to you and dual licence it.


Personally, I'd go the MPL2 route, but that's my choice. It might not 
suit you. But to achieve what you want, you need a copyleft, GPL-style 
licence.



If that sounds weird, it's because both Copyleft and Permissive claim to
be free, but have completely different target audiences. Once you've
answered that question, it'll make choosing a licence so much easier.

GPL gives freedom to the END USER. It's intended to protect the users of
your program from being held to ransom.


That's not how the kernel devs handle the GPL. They use it to remove choice
from the end user (me) to use what I want (ZFS).
And it's that which I don't like about the GPL.

No. That's Oracle's fault. The kernel devs can't include ZFS in linux, 
because Oracle (or rather Sun, at the time, I believe) deliberately 
*designed* the ZFS licence to be incompatible with the GPL.


After all, there's nothing stopping *you* from combining Linux and ZFS, 
it's just that somebody else can't do that for you, and then give you 
the resulting binary.


At the end of the day, if someone wants to be an arsehole, there's not a 
lot you can do to stop them, and with ZFS that honour apparently goes to 
Sun.


Cheers,
Wol

Re: [gentoo-user] Re: Suggestions for backup scheme?

[Wols Lists]

On 07/02/2024 11:07, J. Roeleveld wrote:

Because snapshotting uses so much less space?

So much so that, for normal usage, I probably have no need to delete any
snapshots, for YEARS?

My comment was based on using rsync to copy from the source to the backup
filesystem.


Well, that's EXACTLY what I'm doing too. NO DIFFERENCE. Actually, there 
is a minor difference - because I'm using lvm, I'm also using rsync's 
"overwrite in place" switch. In other words, it compares source and 
destination *in*place*, and if any block has changed, it overwrites the 
change, rather than creating a complete new copy.


Because lvm is COW, that means I have two copies of the file, in two 
different snapshots, but inasmuch as the files are identical, there's 
only one copy of the identical bits.



Okay, space is not an expensive commodity, and you don't want too many
snapshots, simply because digging through all those snapshots would be a
nightmare, but personally I wouldn't use a crude rsync simply because I
prefer to be frugal in my use of resources.



What is "too many"?
I currently have about 1800 snapshots on my server. Do have a tool that
ensures it doesn't get out of hand and will remove several over time.

"Too many" is whatever you define it to be. I'm likely to hang on to my 
/home snapshots for yonks. My / snapshots, on the other hand, I delete 
anything more than a couple of months old.


If I can store several years of /home snapshots without running out of 
space, why shouldn't I? The problem, if I *am* running out of space, I'm 
going to have to delete a *lot* of snapshots to make much difference...


Cheers,
Wol

Re: [gentoo-user] Re: Suggestions for backup scheme?

[Wols Lists]

On 06/02/2024 16:19, J. Roeleveld wrote:

Ah! Got it. That's one of the things I've been trying to figure out
this entire thread, do I need to switch home and root to ZFS to take
advantage of its snapshot support for backups? In the case you're
describing the "source" filesystem(s) can be anything. It's only the
_backup_  filesystem that needs to be ZFS (or similar).



If you want to use snapshots, the filesystem will need to support it. (either
LVM or ZFS). If you only want to create snapshots on the backupserver, I
actually don't see much benefit over using rsync.


Because snapshotting uses so much less space?

So much so that, for normal usage, I probably have no need to delete any 
snapshots, for YEARS?


Okay, space is not an expensive commodity, and you don't want too many 
snapshots, simply because digging through all those snapshots would be a 
nightmare, but personally I wouldn't use a crude rsync simply because I 
prefer to be frugal in my use of resources.


Cheers,
Wol

Re: [gentoo-user] Re: Suggestions for backup scheme?

[Wols Lists]

On 06/02/2024 15:35, Grant Edwards wrote:

If (like rsnapshot/rsync's hard-link scheme) ZFS snapshots are normal
directory trees that can be "browsed" with normal filesystem tools,
that would be ideal. [I'll do some googling...]


Bear in mind I'm talking lvm snapshots, not ZFS ...

And you can configure snapshots to grow as required.

I know it's nothing really to do with backups, but if you read the raid 
wiki page https://raid.wiki.kernel.org/index.php/System2020 - that's how 
I set up my system, with a smattering of lvm. Just look at the sections 
pvcreate, vgcreate, lvcreate, it'll tell you how to create the lvm. Then 
you just format your lvcreate'd partitions, and you can mount them, 
snapshot them, whatever them.


So you can either have one backup partition per source partition, or one 
backup partition and copy all your sources into just that one.


Your choice :-)

Cheers,
Wol

Re: [gentoo-user] Re: Suggestions for backup scheme?

[Wols Lists]

On 06/02/2024 13:12, J. Roeleveld wrote:

Clearly Oracle likes this state of affairs.  Either that, or they are
encumbered in some way from just GPLing the ZFS code.  Since they on
paper own the code for both projects it seems crazy to me that this
situation persists.



GPL is not necessarily the best license for releasing code. I've got some
private projects that I could publish. But before I do that, I'd have to
decide on a License. I would prefer something other than GPL.


Okay. What do you want to achieve. Let's just lump licences into two 
categories to start with and ask the question "Who do you want to free?"


If that sounds weird, it's because both Copyleft and Permissive claim to 
be free, but have completely different target audiences. Once you've 
answered that question, it'll make choosing a licence so much easier.


GPL gives freedom to the END USER. It's intended to protect the users of 
your program from being held to ransom.


Permissive gives freedom to the DEVELOPER. It's intended to let other 
programmers take advantage of your code and use it.


Once you've decided what sort of licence you want, it'll be easier to 
decide what licence you want.


Cheers,
Wol

Re: [gentoo-user] Re: Suggestions for backup scheme?

[Wols Lists]

On 04/02/2024 15:48, Grant Edwards wrote:

OK I see. That's a bit different than what I'm doing.  I'm backing up
a specific set of directory trees from a couple different
filesystems. There are large portions of the "source" filesystems that
I have no need to back up.  And within those directory trees that do
get backed up there are also some excluded subtrees.


But my scheme still works here. The filesystem I'm snapshotting is the 
backup. As such, it only contains the stuff I want backed up, copied 
across using rsync.


There's nothing stopping me running several rsyncs from the live system, 
from several different partitions, to the backup partition.


Cheers,
Wol

Re: [gentoo-user] Re: Suggestions for backup scheme?

[Wols Lists]

On 04/02/2024 06:24, Grant Edwards wrote:

On 2024-02-03, Wol  wrote:

On 03/02/2024 16:02, Grant Edwards wrote:

rsnapshot is an application that uses rsync to do
hourly/daily/weekly/monthly (user-configurable) backups of selected
directory trees. It's done using rsync to create snapshots. They are
in-effect "incremental" backups, because the snapshots themselves are
effectively "copy-on-write" via clever use of hard-links by rsync. A
year's worth of backups for me is 7 daily + 4 weekly + 12 monthly
snapshots for a total of 23 snapshots.  If nothing has changed during
the year, those 23 snapshots take up the same amount of space as a
single snapshot.


So as I understand it, it looks like you first do a "cp with hardlinks"
creating a complete new directory structure, but all the files are
hardlinks so you're not using THAT MUCH space for your new image?


No, the first snaphost is a complete copy of all files.  The snapshots
are on a different disk, in a different filesystem, and they're just
plain directory trees that you can brose with normal filesystem
tools. It's not possible to hard-link between the "live" filesystem
and the backup snapshots. The hard-links are to inodes "shared"
between different snapshot directory trees. The first snapshot copies
everything to the backup drive (using rsync).


Yes I get that. You create a new partition and copy all your files into it.

I create a new pv (physical volume), lv (logical volume), and copy all 
my files into it.


The next snapshot creates a second directory tree with all unchanged
files hard-linked to the files that were copied as part of the first
snapshot. Any changed files just-plain-copied into the second snapshot
directory tree.


You create a complete new directory structure, which uses at least one 
block per directory. You can't hard link directories.


I create a LVM snapshot. Dunno how much that is - a couple of blocks?

You copy all the files that have changed, leaving the old copy in the 
old tree and the new copy in the new tree - for a 10MB file that's 
changed, you use 10MB.


I use rsync's "Overwrite in place" mode, so if I change 10 bytes at the 
end of that 10MB file I use ONE block to overwrite it (unless sod 
strikes). The old block is left in the old volume, the new block is left 
in the new volume.


The third snapshot does the same thing (starting with the second
snapshot directory tree).


So you end up with multiple directory trees (which could be large in 
themselves), and multiple copies of files that have changed. Which could 
be huge files.


I end up with ONE copy of my current data, and a whole bunch of dated 
mount points, each of which is a full copy as of that date, but only 
actually uses enough space to store a diff of the volume - if I change 
that 10MB file every backup, but only change lets say 10KB over three 
4KB disk blocks, I've only used four blocks - 16KB - per backup!


Rinse and repeat.

Old snapshots trees are simply removed a-la 'rm -rf" when they're no
longer wanted.


So each snapshot is using the space required by the directory
structure, plus the space required by any changed files.


Sort of. The backup filesystem has to contain one copy of every file
so that there's something to hard-link to. The backup is completely
stand-alone, so it doesn't make sense to talk about all of the
snapshots containing only deltas. When you get to the "oldest"
snapshot, there's nothing to delta "from".


I get that - it's a different hard drive.



[...]

And that is why I like "ext over lvm copying with rsync" as my
strategy (not that I actually do it). You have lvm on your backup
disk. When you do a backup you do "rsync with overwrite in place",
which means rsync only writes blocks which have changed. You then
take an lvm snapshot which uses almost no space whatsoever.

So to compare "lvm plus overwrite in place" to "rsnapshot", my
strategy uses the space for an lvm header and a copy of all blocks
that have changed.

Your strategy takes a copy of the entire directory structure, plus a
complete copy of every file that has changed. That's a LOT more.


I don't understand, are you saying that somehow your backup doesn't
contain a copy of every file?

YES! Let's make it clear though, we're talking about EVERY VERSION of 
every backed up file.


And you need to get your head round the fact I'm not - actually - 
backing up my filesystem. I'm actually snapshoting my disk volume, my 
disk partition if you like.


Your strategy contains a copy of every file in your original backup, a 
full copy of the file structure for every snapshot, and a full copy of 
every version of every file that's been changed.


My version contains a complete copy of the current backup and (thanks to 
the magic of lvm) a block level diff of every snapshot, which appears to 
the system as a complete backup, despite taking up much less space than 
your typical incremental backup.


To change analogies completely - think git. My lvm snapshot is like 

Re: [gentoo-user] Re: Suggestions for backup scheme?

[Wols Lists]

On 31/01/2024 17:56, Rich Freeman wrote:

I don't think there are
any RAID implementations that do full write journaling to protect
against the write hole problem, but those would obviously underperform
zfs as well.


This feature has been added to mdraid, iirc.

Cheers,
Wol

Re: [gentoo-user] The hopeless futility of printing.

[Wols Lists]

On 29/01/2024 18:19, Alan Grimes wrote:

k...@aspodata.se wrote:

>> Absolutely suprimo HP laser jet network printer.
You didn't write what model, hard to help you then.


It's a LaserJet Pro M453-4.


I have absolutely no trouble with HP. But I always used hplip. I notice 
though it's not installed on my current server/workstation ??? That 
prints fine.


My printer's an M477 - with scanner and everything - but that's 
configured as "scan to network" so it just opens a samba share and dumps 
the scan there.


Under "make and model", cups says "HP Color LaserJet Series PCL 6 CUPS".

Cheers,
Wol

Re: [gentoo-user] Kmail does not show all imap folders

[Wols Lists]

On 22/01/2024 07:04, Alexander Puchmayr wrote:

I checked folder subscriptions in kmail, but I do not see the missing folders
there either. Also akonadi-console does not show them. I also tried
curl imaps:///
Showing all of the missing folders, so I think its an akonadi/kmail problem
and not an imap problem.

Any ideas?


I occasionally get this problem in Thunderbird, and one of the options 
in folder properties is "delete the index". If you've got that option 
for the parent folder, do it, and then kmail should rebuild from scratch 
and find the folders.


Cheers,
Wol

Re: [gentoo-user] startx error

[Wols Lists]

On 09/01/2024 22:20, the...@sys-concept.com wrote:

I today's AI age I am supersized we have add it manually to run-level. :-/


Today's clever AI runs on Berkeley. The LSD version, not BSD.

Cheers,
Wol

Re: [gentoo-user] Genlop wonky again

[Wols Lists]

On 09/01/2024 03:35, Peter Humphrey wrote:

On Sunday, 7 January 2024 08:34:15 GMT Wols Lists wrote:


Weird! I took a module on statistics in my Open University (Chemistry)
degree 40-odd years ago. Probably the same one? I've still got the
modules as a reference work, though I probably couldn't lay my hands on
them easily now ...


Could have been the same. It was M100, the first version of their maths
foundation course, in 1976.

Ah. So you predate me slightly. I took M101. But I also took the second 
level statistics course, can't remember what it was ...


Cheers,
Wol

Re: [gentoo-user] Genlop wonky again

[Wols Lists]

On 07/01/2024 00:52, Peter Humphrey wrote:

They seemed to say that the subject was founded on two
basic principles; then they proceeded to define each of them in terms of the
other.


I should add, I dug into this sort of stuff, and you do know the entire 
edifice of Peano (ie number theory), thanks to Godel, is built on the 
edifice that " true == false " :-) ?


Basically, no matter how hard you try, you cannot escape the Cretan Paradox.

To quote some famous mathematician - "If you define a religion as the 
irrational belief in the unprovable, then Mathematics is the only 
religion that can prove it is one".


That's why the Ancient Philosophers debated how many Angels can dance on 
the Head of a Pin. Set aside your prejudices, your beliefs that "that 
*must* be stupid", read Terry Pratchett's "Science of Diskworld", and 
realise that it doesn't matter WHERE you start, the application of logic 
and reason will lead you down the Rabbit Hole into Wonderland.


And modern man is no better at avoiding that trap than the ancients.

Cheers,
Wol

Re: [gentoo-user] Genlop wonky again

[Wols Lists]

On 07/01/2024 00:52, Peter Humphrey wrote:

On Saturday, 6 January 2024 19:28:05 GMT Wols Lists wrote:


Statistics is one of those areas where, if you don't know what you're
doing and you use the wrong maths, then you are going to get stupid results.

"Statistics tell you how to get from A to B. What they don't tell you is
that you're all at C".


I took a module on statistics in my Open University maths degree 40-odd years
ago. I was bemused. They seemed to say that the subject was founded on two
basic principles; then they proceeded to define each of them in terms of the
other.

Weird! I took a module on statistics in my Open University (Chemistry) 
degree 40-odd years ago. Probably the same one? I've still got the 
modules as a reference work, though I probably couldn't lay my hands on 
them easily now ...



I'm still waiting for the entire edifice to come crashing down around our ears.
:)

Nah - it's been abused for so long nobody's noticed it came down 
centuries ago :-)


Cheers,
Wol

Re: [gentoo-user] Emerge load again

[Wols Lists]

On 06/01/2024 17:52, Peter Humphrey wrote:

In other cases, there may be a hundred separate tasks, make fires off a
hundred tasks shared amongst all the resource it can find, and sits back
and waits.



And that's how the very first installation goes, with single-host distcc. Then,
when it gets to gcc, it collapses to 2 threads and everything gained so far is
lost many-fold. (I set USE=-fortran to avoid pointless recompilation, since
nothing needs it here.)


So if it's consistently gcc that collapses to two threads, then 
something (maybe explicit settings, maybe dependencies, maybe yadda 
yadda) is telling make that only two jobs can run at the same time else 
they'll trip over each other.


Could be a dev has hard-coded the "two jobs" rule to make those random 
crashes go away :-) Or maybe they found the problem, and that's why only 
two jobs can run in parallel.


Cheers,
Wol

Re: [gentoo-user] Genlop wonky again

[Wols Lists]

On 06/01/2024 17:59, Peter Humphrey wrote:

On Saturday, 6 January 2024 16:21:30 GMT Wols Lists wrote:


... it's nothing to do with more power or whatever, it's down to simple
statistics. If genloop guesses the statistical spread wrongly, it's
going to mess up its estimates.



Aren't you exaggerating genlop's complexity? I wasn't aware of any use of
statistics in it, other than a simple arithmetic mean to estimate the time
remaining. It certainly seems to do that, anyway.


Other than a simple arithmetic mean !!! Other than a simple arithmetic 
mean !!!


If that's the case, you've just confirmed my statement - genloop is 
almost certainly using the wrong statistics for the job !!!


If you take the average (arithmetic mean) of a power-law (exponential 
decay) distribution, your results are going to be garbage.


Statistics is one of those areas where, if you don't know what you're 
doing and you use the wrong maths, then you are going to get stupid results.


"Statistics tell you how to get from A to B. What they don't tell you is 
that you're all at C".


Cheers,
Wol

Re: [gentoo-user] Genlop wonky again

[Wols Lists]

On 06/01/2024 16:12, John Blinka wrote:
And it doesn’t actually take 2x longer - the new estimate is just 
grossly wrong.


I presume that the old estimate was also wrong.

And it's nothing to do with more power or whatever, it's down to simple 
statistics. If genloop guesses the statistical spread wrongly, it's 
going to mess up its estimates.


If you have a double-peak distribution, with a large short-lived peak, 
and a small long-lived peak, you can get some weird results, especially 
if you have assumed a bell curve (almost always wrong) or an exponential 
decay (which is generally, NOT ALWAYS, a good choice).


Cheers,
Wol

Re: [gentoo-user] Emerge load again

[Wols Lists]

On 29/11/2023 12:06, Peter Humphreey wrote:

The contribution of distcc isn't clear to me yet, as I said before. Sometimes
it's the bee's knees; other times it might just as well not be there. I don't
like mysteries... 


As far as I'm aware, there's no mystery. On a single machine you get the 
exact same thing ... it's all down to parallelism.


Make asks itself "how many separate tasks can I do at the same time, 
which won't interfere with each other". In gcc's case, the answer 
appears to be two. It doesn't matter how much resource is available, 
make can only make use of two cores.


In other cases, there may be a hundred separate tasks, make fires off a 
hundred tasks shared amongst all the resource it can find, and sits back 
and waits.


Think of a hundred compile jobs all running at the same time, but then 
the linker is invoked, and you can only have the one linker running, 
after all the compile jobs have finished.


And this is a HARD problem, I haven't seen it recently, but there used 
to be plenty of threads about hard-to-debug compile failures that went 
away with -j1. The obvious cause was two compile jobs being set off in 
parallel, when in reality one depended on the other, and things messed up.


Cheers,
Wol

Re: [gentoo-user] Genlop wonky again

[Wols Lists]

On 06/01/2024 00:54, John Blinka wrote:
I’ve often found that it gives one estimate when multiple packages are 
being built, then a much longer estimate for still-in-progress builds 
once some of the builds have finished.


That result defies common sense. Less remaining work has to take less, 
not more (much more), time.


Common sense isn't common and, well, often doesn't make sense.

If there's a bunch of small builds skewing the "time per build" estimate 
down, as they drop off the list the estimated time per build will go up, 
and if the skew is serious enough it can even make the total estimated 
time go up ...


Cheers,
Wol

Re: [gentoo-user] Virtualbox 7

[Wols Lists]

On 21/12/2023 05:48, the...@sys-concept.com wrote:
I run Windows 7 in VirualBox-6 and it runs relatively fast but after 
upgrade to VirtualBox-7 (7.0.10) I noticed
users weren't able to access Windows-7 server; the network got almost 
non responsive.  I've notice same with earlier VirtualBox-7.0.8 (something)

I've found many post complaining about similar problems with VirtualBox-7

What changes did they introduced in VirtualBox-7 that make it unusable?


If it ran reasonably fast, well, think yourself unusual.

I run XP, and although I've got a Win7 box, I never run it, it always 
felt dreadfully slow - and that's going back years. I doubt they've 
changed anything, it wouldn't surprise me if it's a default config. I 
don't tend to dig in to that, so I wouldn't know what to change.


Plus, of course, changing things tends to break Windows' authentication :-(

Cheers,
Wol44

Re: [gentoo-user] New installation - not booting

[Wols Lists]

On 06/12/2023 14:56, Peter Humphreey wrote:

The idea is that you may want to install another system later, which may want
to install its own code in /efi. By all means shrink it if you think that's
unlikely and you need the space. Gparted on SysRescCD is ideal for this.


I had the opposite problem - Windows created a tiny EFI partition and I 
couldn't install linux ...


Cheers,
Wol

Re: [gentoo-user] Boot and EFI partitions

[Wols Lists]

On 06/12/2023 16:36, Jack Ostroff wrote:
The way I think of it is that the UEFI firmware needs to find the 
.efi loader, and it can only read FAT32 formatted partitions 
labelled as type esp. That .efi loader then needs to find your 
kernel and related files, but as it is specific for that type of kernel 
(linux) it can know about more partition formatting options. I suspect 
that many (most?/all?) existing linux utilities still expect the boot 
dir to be at /boot, but perhaps the docs are late to change describing 
that it no longer needs to be a separate partition, or perhaps one or 
more of those utilities still requires a partition.


It's not that far off base ... note that Macs use UEFI, but don't have 
FAT32 partitions ...


On the raid wiki I say that you need *some* sort of boot partition, and 
say that if you're using legacy (bios) boot you should really put /boot 
on its own partition so if you decide to go UEFI, you re-purpose that 
partition. It certainly seems weird that systemd-boot would want its own 
partition on top of the efi partition :-)


The UEFI spec basically requires - as a minimum - that the UEFI software 
in rom must be able to read fat32. There is no requirement that that is 
the only format it can read, hence Macs reading the Mac format. If you 
want your own boot rom that can read ext*, that's fine (so long as ext32 
is also available).


Cheers,
Wol

Re: [gentoo-user] ffmpeg: WARNING: One or more updates/rebuilds have been skipped due to a dependency conflict

[Wols Lists]

On 04/12/2023 08:28, Dale wrote:

Oh, I see the little  pointing up there but in Konsole, they never
point up to the right place.  If it has a clue, I wouldn't be able to
get help from it.  Also, I have some options in make.conf for emerge so
I'm taking this from emerge.log to show the complete command that is
used.  Keep in mind, I increased backtrack and added changed-deps.


2>&1 tee emerge.log

That should give you the output in a file (emerge.log), and in an editor 
it should all line up (provided the editor is set not to wrap).


Cheers,
Wol

Re: [gentoo-user] [OT] Very slow POST process

[Wols Lists]

On 29/11/2023 00:16, Michael wrote:

Thanks Dan, will do.  I was planning to take it apart soon to replace the HDD
with an SSD, so this would be the first thing to check.  I expect finding a
replacement unit will be difficult.  Every Lenovo RTC battery seems to have a
different part number.


I know laptops are different from desktops, but I think every desktop 
mobo I've come across uses a 3032 battery.


See if you can find out what the standard definition of the lenovo 
battery is, hopefully they just use internal part numbers for a totally 
standard item.


Cheers,
Wol

Re: [gentoo-user] Re: Portage reports preserved libs, but won't rebuild

[Wols Lists]

On 24/11/2023 10:27, Nuno Silva wrote:

On 2023-11-24, Arve Barsnes wrote:


On Fri, 24 Nov 2023 at 04:07, Jack  wrote:


May or may not help, but have you tried revdep-rebuild?


Also, you can try just one-shotting the reported packages, such as
(for the last one in your list):

emerge -1 sys-libs/zlib

Regards,
Arve


Shouldn't it be the other way around? I mean, isn't it freetype,
harfbuzz, glib and libpng which need to be rebuilt so that these
libraries from sys-libs/zlib and bzip2, libpcre2, graphite2, ..., are
not needed and get removed?


I regularly get "nothing to update".

I strongly suspect it's because libfreetype and harfbuzz are circular 
dependencies.


When I've built a couple of systems I regularly get "unable to install 
harfbuzz because it needs libfreetype which it can't install because it 
needs harfbuzz ..." You need to force-build one, and then it's happy.


So that's probably messed up @revdep-rebuild.

Cheers,
Wol

Re: [gentoo-user] Emerge -K ignoring new packages

[Wols Lists]

On 20/11/2023 17:12, Vitaliy Perekhovy wrote:

On Mon, Nov 20, 2023 at 05:07:45PM +, Peter Humphrey wrote:

Hello list,

Now that I have my NFS set up (with help - thanks) the next problem is that,
having new packages built by my workstation over NFS, emerge on the tiny box
is ignoring all those new packages. And yes, I have checked that they do
exist, and in the right place: /var/cache/packages/ .

The man page says that any new package will cause a remerge, so what has
tripped me up this time?

--
Regards,
Peter.


Default location for binary packages is /var/cache/binpkgs/

Can't remember what I did, but the first thing to check is you're using 
the same make flags (unless of course, you're sharing /etc/portage).


Then I seem to remember using -bK or something like that. So the command 
I'm giving emerge is "use a binary if you can find it, otherwise build it".


Because I might emerge packages on either machine, that worked great for 
me. And I actually usually emerged stuff on the slower machine, because 
it was more reliable ... :-)


Cheers,
Wol

Re: [gentoo-user] Re: OFF TOPIC Need Ubuntu network help: boot loader info

[Wols Lists]

On 19/10/2023 12:55, Neil Bothwick wrote:

On Wed, 18 Oct 2023 23:49:25 -0500, Dale wrote:


That config kinda reminds me of the old grub.  A title line, location of
kernel and then options.  Sounds easy enough.  The new grub config is
almost impossible to config by hand.  They had to make a tool to do it.
That says a lot there.  ;-)


GRUB2 was designed to be able to create a config for anything
automatically, such as from an installer. It does that very well, but is
total overkill for Gentoo-like people that like to stay in control.



Such a shame it doesn't work. :-)

I tried to install SUSE dual boot, and it broke the installer - NOTHING 
would boot. I needed a rescue disk to fix the mess ...


Cheers,
Wol

Re: [gentoo-user] OFF TOPIC Need Ubuntu network help

[Wols Lists]

On 16/10/2023 08:51, Dale wrote:
Anyone here have ideas?  Keep in mind, that thing uses systemd.  I 
thought I hated that before.  I truly hate that thing now.  Trying to 
figure out how to restart something is like pulling teeth with no pain 
meds.


systemctl restart servicename?

I like systemd, but given my battles with other stuff, I feel your pain. 
Having had to WRITE a service file, though, oh I'm so glad I wasn't 
messing with SystemV or stuff like that!


Just be warned - I feel about apt stuff just like you feel about systemd ...


But anyways. Does your hard disk kernel have the appropriate module for 
the network card loaded? I can't remember the name of the systemd 
networking service, but did you "systemctl enable" it?


Oh, and I think it fires up DHCP by default so you don;'t need to enable 
any of that stuff.


Hopefully those tips will get you somewhere - this is what I remember 
from enabling systemd on gentoo...


Cheers,
Wol

Re: [gentoo-user] world updates blocked by Qt

[Wols Lists]

On 11/10/2023 17:44, Philip Webb wrote:

231011 Alan McKinnon wrote:

Today a sync and emerge world produces a huge list of blockers.
qt 5.15.10 is currently installed and qt 5.15.11 is new in the tree and
being blocked.
All the visible blockers are Qt itself so --verbose-conflicts is needed.


My experience for some time has been that Qt pkgs block one another,
st the only way out is to unmerge them all, then remerge them all.
If anyone knows a better method, please let us know

I haven't had that in a long time. If I get blocks like that (rare) 
--backtrack=100 (or whatever it is) unusually unblocks it.


The other thing is, I don't have any explicit perl code on my system, 
but on at least one occasion running perl-cleaner --all unbunged a 
problem ...


There's a whole bunch of incantations that are rarely needed but need to 
be remembered for when they are ...


Cheers,
Wol

Re: [gentoo-user] Password questions, looking for opinions. cryptsetup question too.

[Wols Lists]

On 19/09/2023 10:13, Dale wrote:

That's a interesting way to come up with passwords tho.  I've seen that
is a few whodunit type shows.  Way back in the old days, they had some
interesting ways of coding messages.  Passwords are sort of similar.


Back when we were busy conquering India ...

The story goes of a General trying to send a message back of his latest 
conquest, but he didn't want to use codes because he had a suspicion the 
Indians could read them if his messenger was captured.


It appears the story is apocryphal, but the message he sent read "peccavi".

https://www.ft.com/content/49036e66-ac48-11e8-94bd-cba20d67390c

Cheers,
Wol

Re: [gentoo-user] Password questions, looking for opinions. cryptsetup question too.

[Wols Lists]

On 20/09/2023 19:05, Frank Steinmetzger wrote:

In principle, a repeated space character in your passphrase could help
reduce the computational burden of an offline brute force attack, by e.g.
helping an attacker to identify the number of individual words in a
passphrase.



Due to the rotation, the Enigma encoded each subsequent letter differently,
even if the same one repeated, which was (one of) the big strengths of the
Enigma cipher. The flaws were elsewhere, for example that a character could
never be encrypted onto itself due to the internal wiring and certain
message parts were always the same, like message headers and greetings.


And, as always, one of the biggest weaknesses was the operator.

Enigma had three (or in later versions four) rotors. The code book 
specified the INITIAL "settings of the day" for those rotors. What was 
*supposed* to happen was the operator was supposed to select a random 
three/four character string, transmit that string twice, then reset the 
rotors to that string before carrying on. So literally no two messages 
were supposed to have the same settings beyond the first six characters.


Except that a lot of operators re-used the same characters time and time 
again. So if you got a message from an operator you recognised, you 
might well know his "seventh character reset". That saved a lot of grief 
trying to crack which of the several rotors were "the rotors of the day".


And given that, for a large chunk of the war, the radio operators were 
"chatty", you generally got a lot of six-character strings for which you 
had a damn good idea what the plain text was.


So even where some of the operators were seriously crypto-aware and 
careful, once you'd cracked the rotors and initial settings from the 
careless, you could read every message sent by everyone (using those 
settings) that day.


Along with other things like RDF giving subs positions away (although 
I'm not quite sure how much we had good RDF and how much it was a cover 
for us reading their location in status reports), it certainly helped us 
loads hunting them down.


Cheers,
Wol

Re: [gentoo-user] Password questions, looking for opinions. cryptsetup question too.

[Wols Lists]

On 19/09/2023 10:10, Jude DaShiell wrote:

Once the set spots got figured
five dice got used for letters add the total and subtract 4 for the
particular letter.


Which actually isn't random. It's a bell curve peaking probably between 
J and M. Think, if you throw 2 dice, there are 36 possible combinations. 
Only one of them generates 2, only one generates 12, but 6 combinations 
can generate 7.


Cheers,
Wol

Re: [gentoo-user] Re: How to move ext4 partition

[Wols Lists]

On 20/09/2023 23:39, Grant Edwards wrote:

Assuming GParted is smart enough to do overlapping moves, is it smart
enough to only copy filesystem data and not copy "empty" sectors?
According to various forum posts, it is not: moving a partion copies
every sector. [That's certainly the obvious, safe thing to do.]


Seeing as it knows nothing about filesystems, and everything about 
partitions, it will treat the partition as an opaque blob and move it as 
a single object ...


The partition in question is 200GB, but only 7GB is used, so I think
backup/restore is the way to go...


You would think so :-)

I use ext4, and make heavy use of hard links. Last time I tried a 
straight copy (not backup/restore) I think the copied partition would 
have been three times the size of the original - that is if it hadn't 
run out of space first :-)


But it sounds like that would work well for you.

Cheers,
Wol

Re: [gentoo-user] Computer case for new build

[Wols Lists]

On 18/09/2023 11:13, Frank Steinmetzger wrote:

With so many drives, you should also include a pricey power supply. And/or a
server board which supports staggered spin-up. Also, drives of the home NAS
category (and consumer drives anyways) are only certified for operation in
groups of up to 8-ish. Anything above and you sail in grey warranty waters.
Higher-tier drives are specced for the vibrations of so many drives (at
least I hope, because that’s what they™ tell us).


Have you seen the article where somebody tests that? And yes, it's true. 
The more drives you have, the more you need damping. If all the drives 
move their heads together, the harder it is for them to home in on the 
correct track, to the point where you get the "perfect storm" of 
vibration causing them all to reset, go back to park, try again, and 
they are shaking so much none of them can find what they're looking for.



To be honest, I kinda like the Fractal Design Define 7
XL right now despite the higher cost.  I could make a NAS/backup box
with it and I doubt I'd run out of drive space even if I started using
RAID and mirrored everything, at a minimum.

With 12 drives, I would go for parity RAID with two parity drives per six
drives, not for a mirror. That way you get 2/3 storage efficiency vs. 1/2
and more robustness; in parity, any two drives may fail, but in a cluster of
mirrors, only specific drives may fail (not two of the same mirror). If the
drives are huge, nine drives with three parity drives may be even better
(because rebuilds get scarier the bigger the drives get).

One of my projects in my copious (not) free time was to try and 
implement raid-61. Like raid-10, you could spread it across any number 
of drives (subject to a minimum). You could lose any 4 drives which 
gives you a minimum of five (although with that few that would be the 
equivalent of a five-times mirror).


Hey ho, I don't think that's going to happen now.

Cheers,
Wol

Re: [gentoo-user] Computer case for new build

[Wols Lists]

On 18/09/2023 12:16, Rich Freeman wrote:

This is part of why I like storage implementations that have more
robustness built into the software.  Granted, it is still only as good
as your clients, but with distributed storage I really don't want to
be paying for ECC on all of my nodes.  If the client calculates a
checksum and it remains independent of the data, then any RAM
corruption should be detectable as a mismatch (that of course assumes
the checksum is preserved and not re-calculated at any point).


Which is why I run raid-5 over dm-integrity. I'm not sure it's that 
stable :-( :-( but it means any disk corruption will get picked up at 
the integrity level, and raid-5 will just get a read error which will 
trigger a parity recalc without data loss.


Cheers,
Wol

Re: [gentoo-user] Computer case for new build

[Wols Lists]

On 19/09/2023 00:40, Dale wrote:

I get it when you wanna do it your way because it always worked™ (which is
not wrong — don’t misunderstand me) and perhaps you had some bad experience
in the past. OTOH it’s a pricey component usually only needed by gamers and
number crunchers. On-board graphics are just fine for Desktop and even
(very) light gaming and they lower power draw considerably. Give it a swirl,
maybe you like it.  Both Intel and AMD work just fine with the kernel
drivers.

Well, for one, I usually upgrade the video card several times before I
upgrade the mobo.  When it is built in, not a option.  I think I'm on my
third in this rig.  I also need multiple outputs, two at least.  One for
monitor and one for TV.  My little NAS box I'm currently using is a Dell
something.  The video works but it has no GUI.  At times during the boot
up process, things don't scroll up the screen.  I may be missing a
setting somewhere but when it blanks out, it comes back with a different
resolution and font size.  I figure it is blanking during the switch.
My Gentoo box doesn't do that.  I can see the screen from BIOS all the
way to when it finishes booting and the GUI comes up.  I'm one of those
who watches.  

Well, in my case I've only recently upgraded to a system where AGPUs are 
available :-)


Plus, although I haven't got it working, I want multi-seat (at present, 
my system won't boot with two video cards). You can run multi-head off 
integrated graphics, but as far as I know linux requires one video card 
per seat.


Oh, and to the best of my knowledge, you can combine a video card and an 
AGPU.


Cheers,
Wol

Re: [gentoo-user] What is the point of baloo?

[Wols Lists]

On 17/09/2023 19:37, Michael wrote:

However, unlike locate, baloo is meant to index not just file names, but also
metadata tags and relationships relevant to files, emails and contacts. Its
devs would argue it has a small footprint.  So it is meant to be*more*  than a
simple file name indexer.


But what is the POINT of said index? If there's no point it's just a 
total and complete waste of time and space!


So, far the only point I'm aware of is it is supposed to make kmail run 
faster - an application I've never used.


Cheers,
Wol

Re: [gentoo-user] What is the point of baloo?

[Wols Lists]

On 17/09/2023 19:35, Peter Böhm wrote:

Am Sonntag, 17. September 2023, 19:46:05 CEST schrieb Wols Lists:


It always annoys me, but baloo seems to be being an absolute nightmare
at the moment.



I tried to kill it and it appears to have just restarted. Is there a use
flag I can use to just get rid of it completely?


Do you mean use-flag "semantic-desktop" ?

(I have disabled it in my make.conf)

I guess I do. I've just disabled indexing as per Mark, and it's reduced 
my load average from 12 "just like that". It's just an absolute pain in 
the arse given that about the only KDE app I actually use is Konqueror.


But yes, I'll set -semantic-desktop in make.conf.

Why does all this crap default to "I'll waste as much of your computer 
time as I can, and I won't tell you what I'm doing, or how to benefit 
from it"?


In turning off indexing, I notice there's also plasma search. But I 
haven't got a clue what all those widgets do. So I click on the "info" 
button and I just get the description AGAIN. What's the point of all 
this crap, if they can't be arsed to tell you what it DOES!?!?


Cheers,
Wol

[gentoo-user] What is the point of baloo?

[Wols Lists]

It always annoys me, but baloo seems to be being an absolute nightmare 
at the moment.


Iirc, it's "the file indexer for KDE" - in other words it knackers your 
response time reading all the files, wastes disk space building an 
index, and all for what?


So that programs you never use can a bit faster? What the hell is the 
point of shaving 10% of a run time of no seconds at all?


I tried to kill it and it appears to have just restarted. Is there a use 
flag I can use to just get rid of it completely?


What I find really frustrating is it claims to have been "built for 
speed". If it's streaming the contents of disk into ram so it can index 
it, it's going to completely knacker your system response whatever 
(especially if a program I WANT running is trying to do the same thing!)


Cheers,
Wol

Re: [gentoo-user] long compiles

[Wols Lists]

On 13/09/2023 12:28, Peter Humphrey wrote:

A thought on compiling, which I hope some devs will read: I was tempted to
push the system hard at first, with load average and jobs as high as I thought
I could set them. I've come to believe, though, that job control by portage
and /usr/bin/make is weak at very high loads, because I would usually find that
a few packages had failed to compile; also that some complex programs were
sometimes unstable. Therefore I've had to throttle the system to be sure(r) of
correctness. Seems a waste. Thus:


Bear in mind a lot of systems are thermally limited and can't run at 
full pelt anyway ...


You might find it's actually better (and more efficient) to run at lower 
loading. Certainly following the kernel lists you get the impression 
that the CPU regularly goes into thermal throttling under heavy load, 
and also that using a couple of cores lightly is more efficient than 
using one core heavily.


It's so difficult to know what's best ... (because too many people make 
decisions based on their interests, and then when you come along their 
decisions may conflict with each other and certainly conflict with you ...)


Cheers,
Wol

Re: [gentoo-user] VPN newbie questions

[Wols Lists]

On 20/08/2023 03:34, Walter Dnes wrote:

   I've been on Gentoo for years and years, but I've never used a VPN, so
consider me an absolute newbie.  Canadian big news media has
successfully lobbied our government to implement a link tax.  Google has
decided to avoid the tax by not linking to it in Google search.  This
morning I tried to find some news about the Russian invasion from my
desktop PC, and the results were brutal.  No links from any Canadian
sources (as expected) or US (CNN) or UK (BBC).  Talk about draconian.  I
need a VPN to make me "self-identify" as being in the USA, or wherever,
so that Google doesn't censor news on me.


Lets hope it goes the way of the Belgian link tax. Shortly after 
enacting it, the local news media etc etc found traffic (and ad revenue) 
went through the floor, and begged the search engines to add them back.


I don't know the deal struck, but it sounds like a private contract 
between the news agency(s) and the search engines "you index us, and we 
won't charge you a penny".


Hopefully your law doesn't outlaw those sorts of deals - or the news 
media might soon be begging the government to repeal the law :-)


Cheers,
Wol

Re: [gentoo-user] Need some help with location of git clone when bisecting with 9999-ebuilds

[Wols Lists]

On 11/08/2023 01:23, Yixun Lan wrote:

1) Where should I store the git clone of the repository and how do I tell
emerge to read the source files from there, when emerging the  ebuild,
instead of downloading them from the Internet as usual?



how about have a local clone of wine repository? and feed it to live ebuild
and you can do the bisect of this local repo..


I have a dedicated ~/gitstuff for most of my stuff, but the obvious 
other place is /usr/src (/wine).


Cheers,
Wol

Re: [gentoo-user] Email clients

[Wols Lists]

On 31/07/2023 16:55, Peter Humphrey wrote:

On Monday, 31 July 2023 15:19:22 BST Wols Lists wrote:


The big question that needs answering is "Are you storing your emails in
dovecot, or in kmail?"


In KMail.

My server has fetchmail -> postfix -> dovecot. Fetchmail collects POP3 emails
from my ISP and forwards it to postfix, and dovecot serves IMAP4 to my
workstation.


So if Dovecot is serving IMAP4 to your workstation, the emails should be 
in dovecot, and cached on your workstation. For my setup, they're stored 
in dovecot, and cached in thunderbird ...


My backup method is simple: I archive KMail's emails daily to a local disk,
then shut the system down on a Sunday to back up the entire system to an
external USB-3 disk.


If kmail is caching them, chances are they're stashed away in the .kmail 
directory or wherever, in some standard format, and you can just back 
that up.


The server is taken down on a Saturday for complete system backup, to another
USB-3 disk.


Perfect, they're now backed up all over the place :-)

Cheers,
Wol

Re: [gentoo-user] Email clients

[Wols Lists]

On 31/07/2023 13:33, Peter Humphrey wrote:

On Monday, 31 July 2023 08:34:05 BST Wols Lists wrote:

On 31/07/2023 00:11, Neil Bothwick wrote:

On Sun, 30 Jul 2023 23:53:39 +0100, Peter Humphrey wrote:

... I use Claws 99% of the time, but occasionally run Thunderbird.
Neither program cares that I have also used the other to read my
mail.


Ah, but you're using IMAP4 and leaving your emails on the server. I
don't want to do that.


But you're running the IMAP server locally, so what difference does it
make?


It's just the way things have 'just growed'. I could start again with the
server keeping the mails itself, but it's a good deal of work.


My server IS my workstation. And if *you* don't want to leave your mail
"centrally", why are you running a dovecot server?


Because KMail is horribly buggy with POP3 and my ISP doesn't offer IMAP4.


I'm trying to get my head round your setup then.

My setup is simple. I couldn't get postfix/fetchmail to behave, so my 
workstation/server runs dovecot.


Thunderbird (on my server) has an account pointing at my ISP, that 
retrieves all my mail and moves it into dovecot. Am I right you've got 
postfix/fetchmail working correctly? All you need to do is make it chuck 
it into dovecot on your server (or not even that).


But the point is, if you have a working instance of dovecot, and you are 
using kmail/imap4 to read your emails FROM DOVECOT, just point claws at 
dovecot as well.


Or are you using kmail/pop3 to pull your emails from dovecot into your 
local kmail instance?


The big question that needs answering is "Are you storing your emails in 
dovecot, or in kmail?"


Cheers,
Wol

Re: [gentoo-user] Email clients

[Wols Lists]

On 31/07/2023 00:11, Neil Bothwick wrote:

On Sun, 30 Jul 2023 23:53:39 +0100, Peter Humphrey wrote:


... I use Claws 99% of the time, but occasionally run Thunderbird.
Neither program cares that I have also used the other to read my
mail.


Ah, but you're using IMAP4 and leaving your emails on the server. I
don't want to do that.


But you're running the IMAP server locally, so what difference does it
make?

My server IS my workstation. And if *you* don't want to leave your mail 
"centrally", why are you running a dovecot server?


Cheers,
Wol

Re: [gentoo-user] Email clients

[Wols Lists]

On 29/07/2023 15:50, Arsen Arsenović wrote:

But I DO have to care about postfix/main.cf. This makes the fundamental blunder
of mixing distro defaults and local config in the SAME FILE. So yes it does
offer me etc-update. But if I MISS THAT, I've just trashed my local config and
have to rebuild it.

If portage trashes the local file, something went wrong.  That is the
only thing that I'm trying to get to the bottom of in this thread.
Application design is irrelevant to that.

You say that the opportunity to etc-update is offered?  If so, portage
worked as it should and I'm satisfied, but I'm still confused about how
the contents got trashed.

Because - with dovecot - I initially made the mistake of editing the 
global file. etc-update over-wrote it.


With postfix, I cannot see any way of NOT editing the global file.

If you go back to what started all this, it was me advising the OP to 
make sure he edited the dovecot local file, not the global one.


And yes, portage is working as it should, but it is working to mitigate 
breakage in the upstream application, namely postfix. Stuff it should 
not need to do.


Cheers,
Wol

Re: [gentoo-user] Email clients

[Wols Lists]

On 29/07/2023 14:54, Arsen Arsenović wrote:

Again, it shouldn't be able to do that.  Please check CONFIG_PROTECT
using: portageq envvar CONFIG_PROTECT

It should, normally, contain /etc, set by profiles/base/make.defaults.


And here is the root of the mis-understanding between us. And also why 
Dovecot does it right, and Postfix does it wrong.


WHY SHOULD I HAVE TO USE DISPATCH-CONF? (Or in my case, etc-update.)

The point is I don't (have to) care whether dovecot.conf is updated or 
not. I never change it from the distro defaults, so it never offers me 
etc-update, and it never does any damage.


But I DO have to care about postfix/main.cf. This makes the fundamental 
blunder of mixing distro defaults and local config in the SAME FILE. So 
yes it does offer me etc-update. But if I MISS THAT, I've just trashed 
my local config and have to rebuild it.


At the end of the day, if you can't keep distro and local config 
separate, that's a fault of the upstream application. etc-update and 
dispatch-conf are gentoo's way of working round the breakage. IFF you 
use dovecot/local.conf, it's a sign of good design by the upstream 
application, and etc-update or dispatch-conf are completely UNNECESSARY.


Cheers,
Wol

Re: [gentoo-user] Email clients

[Wols Lists]

On 29/07/2023 12:01, Peter Humphrey wrote:

Hm. I already have Dovecot on my LAN server, because KMail is horribly buggy
with POP3, which is what my ISP offers. So fetchmail -> postfix -> dovecot
became necessary before I could use IMAP4 in KMail.

All incoming emails are transferred to my workstation because I like to have
everything in one place and one backup.

Maybe I'll stick with KMail a bit longer...


Well then, install Claws and try it - just point it at Dovecot. Okay, I 
use Thunderbird, but there's no reason I have to - I run about 4 
different instances of TB, all pointing at my Dovecot server, and all 
mail is visible on all my computers - the server/workstation, my old 
laptop, my new laptop, my wife's laptop when I borrow it, ...


Cheers,
Wol

Re: [gentoo-user] Email clients

[Wols Lists]

On 29/07/2023 11:13, Arsen Arsenović wrote:

Wols Lists  writes:


On 29/07/2023 03:37, Bryan Gardiner wrote:

User of Claws with a local maildir here.  One mail per file always
felt safer to me.  If you do want to keep using maildir,
net-mail/dovecot provides IMAP access to ~/.maildir out of the box,
and I've found this combination to be reliable.

Just a tip which bit me when I first installed dovecot ...

The master config file actually chain-loads a local config file, make sure you
use it. I edited the master file directly, so of course the first update
overwrote and trashed it ...



That should not happen.  Where's the master config file?  Is it under a
directory masked by CONFIG_PROTECT?


And then the dovecot maintainers update things, update the config file, 
and it breaks for all users because the config version no longer matches 
the program version ...


The master config file is in the obvious place - 
/etc/dovecot/dovecot.conf. Just like postfix breaks exactly the same way 
- /etc/postfix/main.cf.


Imho dovecot has got this (almost) exactly right. Just like systemd. You 
have your master file that is updated by the distro, and you have your 
local file that is updated by the sys admin.


dovecot.conf points to a file local.conf, which does not error if it 
doesn't exist, but over-rides dovecot.conf if it does. The proper way to 
do it!


Unlike postfix - where I can't find a place to split my local config 
away from the default config - so every time postfix is updated I have 
to make sure it doesn't try to update main.cf !!!


Cheers,
Wol

Re: [gentoo-user] Email clients

[Wols Lists]

On 29/07/2023 03:37, Bryan Gardiner wrote:

User of Claws with a local maildir here.  One mail per file always
felt safer to me.  If you do want to keep using maildir,
net-mail/dovecot provides IMAP access to ~/.maildir out of the box,
and I've found this combination to be reliable.


Just a tip which bit me when I first installed dovecot ...

The master config file actually chain-loads a local config file, make 
sure you use it. I edited the master file directly, so of course the 
first update overwrote and trashed it ...


Cheers,
Wol

Re: [gentoo-user] Simple installation on BTRFS

[Wols Lists]

On 27/07/2023 17:18, Michael wrote:

Although I've been using btrfs for the best part of 10 years I have not really
done justice to it, because I have neither explored nor used enough most of
its features.  I am now thinking of installing Gentoo on btrfs again, but this
time I want to optimise the structure of btrfs subvolumes, to simplify
snapshots and backups.


Okay, I've chosen to run ext over lvm over raid, so my only experience 
of btrfs is SUSE using it as default, but ...


I see Ubuntu and derivates install the OS root fs under btrfs subvolume "@"
and /home under subvolume "@home".  This makes storing snapshots of the two
subvolumes under the btrfs top-volume, which remains unmounted, cleaner and
reduces the chance of mixing up the fs you may end up in and operate on (live,
or snapshot).

I have 3 partitions for /boot(ESP), / and /home, but have not yet created
additional partitions for general data storage and backups.

What's your recommended approach and subvolume structure for the deployment of
btrfs on Gentoo for a personal PC, if the primary objective is simplicity in
maintenance, combined with ease of fs recovery?


I did investigate btrfs ...


Any gotchas I should be mindful of?


If you can run two disks and raid, that's always a good idea. SMART is 
supposed to catch disk problems, but they still do die without warning.


btrfs raid is (still) full of gotchas, as far as I know.

Don't use anything higher than raid-1. Parity raid isn't reliable last I 
knew ...


Make sure you know what you're getting. FS people seem to concentrate on 
protecting the file system, not the data. I believe btrfs will raid-1 
the metadata without asking if it can, you need to actively ask for the 
data to be raided, and that's caught people out not realising btrfs 
treats data and metadata differently. (cf the ext3/4 debacle)


Your favoured snapshot/backup strategy?


Manual ... probably shouldn't be. I snapshot / every friday before I do 
an emerge on Saturday. /home I ought to snapshot more than I do.


WATCH YOUR FREE DISK. I think it's all sorted now, but whatever you're 
using it was always a good idea not to go over 90% full. For a very long 
time, a combination of snapshots and a full disk would wedgie the 
system, such that the only way to free up space was to reformat the 
entire disk! As I say, I think it's now fixed so you can delete 
snapshots, but >90% ain't a good idea anyway


Cheers,
Wol

Re: [gentoo-user] Loading modules prevents shutdown

[Wols Lists]

On 17/06/2023 12:57, dhk wrote:
Thanks for the tips.  After spending a lot of time on and off for a few 
weeks trying to keep /lib/modules on its own partition, it just did not 
work right; the system was scrapped and rebuilt per the trivial solution 
with /lib/modules on the root partition.  Now it works as expected.


A good explanation as to why /lib/modules cannot be a separate partition 
would be nice, but after learning learning the hard way again it stays 
on the root partition going forward.


The kernel needs to load modules to boot fully. If mount hasn't run by 
the time the kernel needs a module, you have a problem ...


Even worse, if mount needs the kernel to load a module, you're stuffed ...

Cheers,
Wol

Re: [gentoo-user] trying to get sd card reader to work

[Wols Lists]

On 13/06/2023 03:01, John Blinka wrote:

Good to know it all works, but if you're sticking a new card in an old
reader, they may not be compatible.


Don’t know what constitutes new/old, but these are <1 year old cards. 
Satisfied with empiric evidence that it all works. Have written mp3 
files to this card and played them via Arduino/attached mp3 board. 
Sufficient for my purposes. Amazed that it all works! (Pushing beyond my 
comfort level with card reader/Arduino/mp3 board/wiring all this stuff 
together.)


Basically, just a little bit of history ...

When these cards came out, they were true SD. With a max capacity of 4GB 
(4GB cards are actually rare as hens teeth ...)


As 2GB became cheap and common, the technology transitioned to SDHC, so 
your 4GB card is almost certainly SDHC, and will not work in a true SD 
reader (like my 2009-era satnav).


That had a limit of - iirc - 32GB, and as that became common the 
technology transitioned to SDXC. This is where my knowledge becomes 
rather hazy...


But anyways, everywhere the card is newer than the reader, you have the 
possibility of problems. It rarely happens, but I've been bitten twice 
trying to upgrade the chips in cameras ...


Cheers,
Wol

Re: [gentoo-user] Mouse pain

[Wols Lists]

On 15/05/2023 17:18, Michael wrote:

On Monday, 15 May 2023 17:11:45 BST Wols Lists wrote:

On 15/05/2023 03:51, William Kenworthy wrote:

Checked your menu? XFCE has a "mouse and touchpad" under settings with a
number of useful items including acceleration, double click timings etc.


Yes. As I remember, KDE USED to have such a menu ...

Cheers,
Wol


Still does:

Go to SystemSettings > Input Devices > Mouse

(also there are menus for Keyboard, Game Controller, Drawing Tablet, Touchpad,
Touchscreen, Virtual Keyboard).


The MENU is there. The settings ARE NOT.

You can set pointer speed, swap left and right mouse buttons, and that's 
about it. I've got a fancy gaming mouse and there is ABSOLUTELY NOTHING 
beyond the complete basics that is configurable :-(


No middle button settings. No double click speed. No extra button 
settings. Zilch. Nada. F.all.


Cheers,
Wol

Re: [gentoo-user] Mouse pain

[Wols Lists]

On 15/05/2023 03:51, William Kenworthy wrote:
Checked your menu? XFCE has a "mouse and touchpad" under settings with a 
number of useful items including acceleration, double click timings etc.


Yes. As I remember, KDE USED to have such a menu ...

Cheers,
Wol

[gentoo-user] Systemd query ...

[Wols Lists]

Nothing to do with but sparked by the Apache problem ...

One of the emails mentioned that the "ExecStop" section didn't appear to 
be working ... That's caused me considerable grief in a systemd config 
file I've written ...


Basically, somebody else added an ExecStop section - and all hell broke 
loose. It seemed to be firing on boot :-( And the service in question - 
ScarletDME - seemed to be killing processes at random, like DoveCot ...


Okay, accidentally killing processes it shouldn't is probably down the 
fork/exec code in ScarletDME, I haven't dug into it to know, but systemd 
should not be triggering the stop in the first place. Has anybody else 
encountered anything like this?


Sorry I'm not likely to respond quickly to say "solved", as I need to 
get "in the mood" to get back to debugging, but if anybody has any hints 
and tips, they'd be appreciated, and it might shed some light on that 
Apache problem :-)


Cheers,
Wol

[gentoo-user] Mouse pain

[Wols Lists]

I've been having grief with my mouse for a while, and all the help I can 
find is "how to adjust mouse speed", which is not my problem... and 
seems to be about the only thing that is adjustable ...


Basically even something as simple as left click doesn't work properly. 
I'm guessing it's timing related, but I can't find anywhere to adjust 
it. The symptoms are single clicks get interpreted as double or treble 
clicks, and when I try (especially in games) to do a "drag to select", 
the area the mouse drags over bears precious little resemblance to the 
are the mouse actually is. Combined with the "double click effect" it 
makes the mouse - randomly :-( ! - almost unusable.


It might well be tied into system lock-ups, as every now and then the 
system simply stops responding, as in either the mouse and keyboard just 
no longer work, or the mouse moves freely but the system just ignores it.


Basically I just don't know how - or where - to start debugging it

(systemd, kde system)

Cheers,
Wol

Re: [gentoo-user] file system for new machine

[Wols Lists]

On 27/04/2023 16:52, Neil Bothwick wrote:

On Thu, 27 Apr 2023 15:54:34 +0200, tastytea wrote:


btrfs and zfs have some useful features for normal use cases. the
transparent compression can save a lot of space and even increase speed
in some cases, the checksumming guarantees that you will never get a
corrupt file


That's only true if you use RAID, when there is a good copy to use. If
you have a single disk, they can only let you know a file is corrupt but
not restore it.



I run ext4, over lvm, over raid, over dm-integrity, over spinning rust.

Quite a lot ... dm-integrity in particular is (of necessity) lashed up a 
bit. But lvm over raid is normal.


https://raid.wiki.kernel.org/index.php/Linux_Raid

In particular, as a matter of course, I snapshot my root partition using 
lvm before I "emerge --update", though I've never had any trouble that 
warrants trying to recover. It'll be quite a shock if I do ...


Cheers,
Wol

Re: [gentoo-user] Re: Finally got a SSD drive to put my OS on

[Wols Lists]

On 18/04/2023 23:13, Frank Steinmetzger wrote:

/var/tmp/portage on tmpfs. And on every disk I allocate a swap partition
equal to twice the mobo's max memory. Three drives times 64GB times two is a
helluva lot of swap.



Uhm … why? The moniker of swap = 2×RAM comes from times when RAM was scarce.
What do you need so much swap for, especially with 32 GB RAM to begin with?
And if you really do have use cases which cause regular swapping, it’d be
less painful if you just added some more RAM.


Actually, if you know your history, it does NOT come from "times when 
RAM was scarce". It comes from the original Unix swap algorithm which 
NEEDED twice ram.


I've searched (unsuccessfully) on LWN for the story, but at some point 
(I think round about kernel 2.4.10) Linus ripped out all the ugly 
"optimisation" code, and anybody who ran the vanilla kernel with "swap 
but less than twice ram" found it crashed the instant the system touched 
swap. Linus was not sympathetic to people who hadn't read the release 
notes ...


Andrea Arcangeli and someone else (I've forgotten who) wrote two 
competing memory managers in classic "Linus managerial style" as he 
played them off against each other.


I've always allocated swap like that pretty much ever since. Maybe the 
new algorithm hasn't got the old wanting twice ram, maybe it has, I 
never found out, but I've not changed that habit.


(NB This system is pretty recent, my previous system had iirc 8GB (and a 
maxed out value of 16GB), not enough for a lot of the bigger programs.


Before that point, I gather it actually made a difference to the 
efficiency of the system as the optimisations kicked in, but everybody 
believed it was an old wives tale - until Linus did that ...


Cheers,
Wol

Re: [gentoo-user] Can some config files be automatically protected from etc-update?

[Wols Lists]

On 17/04/2023 19:26, Walter Dnes wrote:

   Now that the (no)multilib problem in my latest update has been solved,
I have a somewhat minor complaint.  Can I get etc-update to skip certain
files?  My latest emerge world wanted to "update"...

1) /etc/hosts (1)
2) /etc/inittab (1)
3) /etc/mtab (1)
4) /etc/conf.d/consolefont (1)
5) /etc/conf.d/hwclock (1)
6) /etc/default/grub (1)
7) /etc/ssh/sshd_config (1)

...hosts is critical for networking.  consolefont allows me tp use the
true text console with a readable font, etc, etc.  I have my reasons
for making certain settings, and keeping them that way.

I had it want to update grub. Which would have utterly borked my system 
the moment I updated my kernel.


Okay, the problem is where you mix user and system config in the same 
file, but this would have deleted lvm and mdadm from my boot config, 
rendering any kernel unbootable. :-(


Like it tried to update postfix many moons ago and would have destroued 
my mail config ...


Surely there's some way of fixing this ...

Cheers,
Wol

Re: [gentoo-user] updating /boot directory EFI

[Wols Lists]

On 17/04/2023 23:36, Peter Humphrey wrote:

On Monday, 17 April 2023 21:41:09 BST Wols Lists wrote:

On 17/04/2023 17:52, Mark Knecht wrote:

Later on a Kubuntu update found Windows, updated the EFI
stuff on the Windows drive and then, I see this morning,
erased everything out of the Kubuntu EFI partition but
left the partition there.


I had a similar problem trying to install SUSE to dual boot a laptop. I
made the mistake of letting Windows wipe the disk and install itself,
with the result I was left with a tiny EFI partition. I couldn't install
linux because there was no room.

My latest attempt (when I get gentoo video working) will be to *add*
Windows to a working system.


Can you not just resize the partition?


Not any more :-)

But I don't tend to do that sort of thing. Which is why my main (linux 
only) machine has pretty much all the disk in one huge raid partition 
with lvm on top ...


Cheers,
Wol

Re: [gentoo-user] updating /boot directory EFI

[Wols Lists]

On 17/04/2023 17:52, Mark Knecht wrote:

Later on a Kubuntu update found Windows, updated the EFI
stuff on the Windows drive and then, I see this morning,
erased everything out of the Kubuntu EFI partition but
left the partition there.


I had a similar problem trying to install SUSE to dual boot a laptop. I 
made the mistake of letting Windows wipe the disk and install itself, 
with the result I was left with a tiny EFI partition. I couldn't install 
linux because there was no room.


My latest attempt (when I get gentoo video working) will be to *add* 
Windows to a working system.


Cheers,
Wol

Re: [gentoo-user] Finally got a SSD drive to put my OS on

[Wols Lists]

On 17/04/2023 02:14, Dale wrote:

My current install is over a decade old.  My /boot partition is about
375MBs.  I should have made it larger but at the time, I booted CD/DVD
media when needed.  I didn't have USB sticks at the time.  This time, I
plan to make some changes.  If I put Knoppix and/or Gentoo LiveGUI in
/boot, it will be larger.  Much larger.  Mark's idea is best tho.  If I
can get Grub to work and boot it.


If you dd your boot partition across, you can copy it into a larger 
partition on the new drive, and then just expand the filesystem.


So changing partition sizes isn't a problem if you want to just copy 
your system drive onto a new disk.


Cheers,
Wol

Re: [gentoo-user] PCIe x1 or PCIe x4 SATA controller card

[Wols Lists]

On 27/03/2023 01:18, Dale wrote:

Thanks for any light you can shed on this.  Googling just leads to a ton
of confusion.  What's true 6 months ago is wrong today.  :/  It's hard
to tell what still applies.


Well, back in the days of the megahurtz wars, a higher clock speed 
allegedly meant a faster CPU. Now they all run about 5GHz, and anything 
faster would break the speed of light ... so how they do it nowadays I 
really don't know ...


Cheers,
Wol

Re: [gentoo-user] Setting a fixed nameserver for openvpn

[Wols Lists]

On 06/03/2023 11:08, Peter Humphrey wrote:

On Monday, 6 March 2023 10:56:37 GMT Wols Lists wrote:

On 06/03/2023 10:06, Michael wrote:



I suspect the behaviour you noticed is related to FF functionality like
TRR
(Trusted Recursive Resolver) farming all your DNS queries over to the
cloudfarce honeypot.

Have a look here if you want to disable it:

https://wiki.archlinux.org/title/Firefox/Privacy#Disable/
enforce_'Trusted_Recursive_Resolver'


Thanks. That led me to network.trr.allow-rfc1918, which provided your
name has a dot in it ! appears to resolve addresses from /etc/hosts. I
guess that actually means firefox uses your local resolver first, and if
it returns an rfc1918 address, will use it.

Surely that should be the default! It shouldn't break a PRIVATE network
in the name of security !!!


It is the default here, in www-client/firefox-110.0.1 .

I'm running amd not ~amd, and I've got FF 102esr. As soon as I changed 
it to allow rfc1918, it started working ...


Cheers,
Wol

Re: [gentoo-user] Setting a fixed nameserver for openvpn

[Wols Lists]

On 06/03/2023 10:06, Michael wrote:

On Monday, 6 March 2023 08:24:35 GMT Wols Lists wrote:

On 06/03/2023 08:08, Neil Bothwick wrote:

On Mon, 6 Mar 2023 07:54:51 +, Wols Lists wrote:

There's another file - can't remember its name - that tells your
resolver what to try in what order - the hosts file, dns, what dhcp
told you, etc etc, so your resolver might not be using dns the way you
think.


Do you mean /etc/nsswitch.conf?


Ah yes. Any idea why Firefox seems to ignore it? Whenever I try to
browse to local machines in /etc/hosts, firefox gives me a google search
page which is a bloody nuisance. If I type a VALID ADDRESS in the
ADDRESS BAR, that's where I expect to go! Not some damn random search page!

Cheers,
Wol


I suspect the behaviour you noticed is related to FF functionality like TRR
(Trusted Recursive Resolver) farming all your DNS queries over to the
cloudfarce honeypot.

Have a look here if you want to disable it:

https://wiki.archlinux.org/title/Firefox/Privacy#Disable/
enforce_'Trusted_Recursive_Resolver'


Thanks. That led me to network.trr.allow-rfc1918, which provided your 
name has a dot in it ! appears to resolve addresses from /etc/hosts. I 
guess that actually means firefox uses your local resolver first, and if 
it returns an rfc1918 address, will use it.


Surely that should be the default! It shouldn't break a PRIVATE network 
in the name of security !!!


Cheers,
Wol

Re: [gentoo-user] Setting a fixed nameserver for openvpn

[Wols Lists]

On 06/03/2023 08:08, Neil Bothwick wrote:

On Mon, 6 Mar 2023 07:54:51 +, Wols Lists wrote:


There's another file - can't remember its name - that tells your
resolver what to try in what order - the hosts file, dns, what dhcp
told you, etc etc, so your resolver might not be using dns the way you
think.


Do you mean /etc/nsswitch.conf?


Ah yes. Any idea why Firefox seems to ignore it? Whenever I try to 
browse to local machines in /etc/hosts, firefox gives me a google search 
page which is a bloody nuisance. If I type a VALID ADDRESS in the 
ADDRESS BAR, that's where I expect to go! Not some damn random search page!


Cheers,
Wol

Re: [gentoo-user] Setting a fixed nameserver for openvpn

[Wols Lists]

On 05/03/2023 18:41, Dale wrote:

I edited the file they say with kwrite.  Even after I restart openvpn,
the IP they want is there but it doesn't use it according to the site
they sent for me to check it with.  It shows other IP addresses.  I'm
sure I'm missing something, likely something simple, but I can't figure
out how to make it work.  I don't know if it is because I'm using openrc
or what.

Anyone have a idea on how to make this work?


resolv.conf tells DNS where to look. That's not openrc/systemd specific.

There's another file - can't remember its name - that tells your 
resolver what to try in what order - the hosts file, dns, what dhcp told 
you, etc etc, so your resolver might not be using dns the way you think.


I can't get that to work, either. I want my hosts file to take priority, 
but it's ignored.


And then, of course, to really screw you over your ISP might be 
hijacking dns.


Cheers,
Wol

Re: [gentoo-user] Thunderbird build failure .. [Gone away]

[Wols Lists]

On 01/01/2023 21:05, Wol wrote:

On 01/01/2023 20:08, cal wrote:

FWIW, Thunderbird builds fine with GCC on my machine -- I'm unsure of
your reasons for setting your Portage compiler to clang, but you may
wish to use a package.env override to build Thunderbird with GCC as a
workaround until the problem can be fixed upstream.


I don't know anything about clang ... it must be the default ...

I thought part of Firefox/Thunderbird was written in Rust, so I assumed 
it was built with llvm as a matter of course.


I'll just wait for it to sort itself out.

Just to say it's finally sorted itself out without me doing anything, 
all the while causing random failures in the build chain ie I think 
Firefox and LLVM all failed along the way ...


Cheers,
Wol

[gentoo-user] Thunderbird build failure ..

[Wols Lists]

I got the following build failure in my weekly emerge yesterday ...

* Messages for package mail-client/thunderbird-102.6.0:

 * ERROR: mail-client/thunderbird-102.6.0::gentoo failed (compile phase):
 *   (no error message)
 *
 * Call stack:
 * ebuild.sh, line 136:  Called src_compile
 *   environment, line 4782:  Called die
 * The specific snippet of code:
 *   ${virtx_cmd} ./mach build --verbose || die
 *
 * If you need support, post the output of `emerge --info 
'=mail-client/thunderbird-102.6.0::gentoo'`,
 * the complete build log and the output of `emerge -pqv 
'=mail-client/thunderbird-102.6.0::gentoo'`.
 * The complete build log is located at 
'/var/tmp/portage/mail-client/thunderbird-102.6.0/temp/build.log'.
 * The ebuild environment file is located at 
'/var/tmp/portage/mail-client/thunderbird-102.6.0/temp/environment'.
 * Working directory: 
'/var/tmp/portage/mail-client/thunderbird-102.6.0/work/thunderbird-102.6.0'
 * S: 
'/var/tmp/portage/mail-client/thunderbird-102.6.0/work/thunderbird-102.6.0'


I'm wondering whether this will simply clear itself next week, seeing as 
last week I got a very similar failure for both thunderbird and firefox.


Could it simply be a bit of the fallout from app-alternatives? Of 
course, it's blocking my depclean ...


Cheers,
Wol

Re: [gentoo-user] Scanner and weird streaking

[Wols Lists]

On 30/12/2022 13:42, Dale wrote:

Given I only use Linux, know of any good and commonly available flatbed
scanner models?  I'm fine with used ones as long as they work.  I like
the one I got since when I'm done, it stands upright in my closet and
takes up very little floor space.  It does a really good job. I've even
scanned in pictures and they looked good, when it wasn't doing the
streak thing.


What do you have by way of a printer? Would you be better with an MFP?

Okay, it's not cheap, but I like my HP MFP M477. Cost about £300 for the 
fdn model. Duplex printer, duplex scanner with document feed, cat-5 
(don't think it's wireless).


You can probably get similar cheaper ...

The thing I really like scanner wise is I just set up a Samba share on 
my computer, configure the thing, and then I can just scan from the 
printer front panel and it appears on my computer automagically.


Cheers,
Wol

Re: [gentoo-user] NAS and replacing with larger drives

[Wols Lists]

On 21/12/2022 02:47, Dale wrote:

I think if I can hold out a little while, something really nice is going
to come along.  It seems there is a good bit of interest in having a
Raspberry Pi NAS that gives really good performance.  I'm talking a NAS
that is about the same speed as a internal drive.  Plus the ability to
use RAID and such.  I'd like to have a 6 bay with 6 drives setup in
pairs for redundancy.  I can't recall what number RAID that is.
Basically, if one drive fails, another copy still exists.  Of course,
two independent NASs would be better in my opinion.  Still, any of this
is progress.


That's called either Raid-10 (linux), or Raid-1+0 (elsewhere). Note that 
1+0 is often called 10, but linux-10 is slightly different.


I'd personally be inclined to go for raid-6. That's 4 data drives, 2 
parity (so you could have an "any two" drive failure and still recover).


A two-copy 10 or 1+0 is vulnerable to a two-drive failure. A three-copy 
is vulnerable to a three-drive failure.


In other words, a two-copy raid-10 might be taken out by a failure that 
a raid-6 will survive. A three-copy raid-10 might be taken out by a 
failure that will take out a raid-6. Choose your poison :-)


Cheers,
Wol

Re: Living in NGL: was: [gentoo-user] NAS and replacing with larger drives

[Wols Lists]

On 19/12/2022 12:00, Rich Freeman wrote:

On Mon, Dec 19, 2022 at 12:11 AM Dale  wrote:

If I like these Raspberry things, may make a media box out of one.  I'd
like to have a remote tho.  



So, I've done that.  Honestly, these days a Roku is probably the
better option, or something like a Google Chromecast or the 47 other
variations on this them.


Where do you put that 2TB drive on your Roku or Chromecast?

I'm thinking of building a media server, not to drive the TV, but to 
record and store. I thought that was what a media server was!


Cheers,
Wol

Re: [gentoo-user] NAS and replacing with larger drives

[Wols Lists]

On 18/12/2022 22:11, Dale wrote:

Wol wrote:

On 18/12/2022 18:59, Dale wrote:

Since this is local, I just use rsync to do my backups.  I did have to
change the options a bit.  It seems TrueNAS doesn't like some of the
permissions or something.


Are you running the rsync daemon on the NAS? I'm probably teaching
grandma to suck eggs, but that massively reduces the need for network
traffic.

Cheers,
Wol





I mount the NAS on my Gentoo rig.  I mount it under /mnt.  Then I run
rsync and copy from the source to the mount point for the NAS.  I may
could go the other way but never thought about doing it that way.  Kinda
sounds backwards to me but I dunno. ;-)


Sounds to me like you're doing it all wrong either way ...

What is *supposed* to happen is that you have the daemon running on one 
machine and the client on the other - doesn't matter which.


Then the client tells the daemon what files are to be copied, THE TWO 
COMPARE CHECKSUMS, and only the stuff that fails the checksum is copied. 
So if you're doing an incremental backup, network usage and writes are 
kept to a minimum.


I tell people to an in-place backup if they're running on a snapshot 
setup, because again it only writes stuff that has actually changed.


Cheers,
Wol

Re: [gentoo-user] NAS and replacing with larger drives

[Wols Lists]

On 10/12/2022 16:19, Frank Steinmetzger wrote:

Where do we confuse those two? We specifically talked of codecs and
“contain”.


"I didn't know .ts could contain h264".

If .ts is the container, then surely the assumption is it can contain 
any codec? If not, why not?


(Yes I do get the impression I didn't read the OP properly. But then, 
the OP didn't make sense properly so I'm not surprised I got it wrong :-)


Cheers,
Wol

Re: [gentoo-user] NAS and replacing with larger drives

[Wols Lists]

On 09/12/2022 13:38, Frank Steinmetzger wrote:

Depending on the PVR make/model I've seen 1080p resolution recordings with
.m2ts and .ts file extensions, while the codecs inside them are the same.



I wasn’t aware that ts could contain h264. But then again—I never really
bothered with live TV recordings in recent years. These days, if I find
something interesting, I download the show form the TV channel’s website
(called Mediathek in Germany, a word play on Bibliothek, meaning library).
Interestingly though, the picture quality is noticably worse than what I
receive via DVB-T.


I think this is confusing CONTAINER and CODEC.

.ts is a container format, h264 is a codec. I don't understand it 
myself, either, but think of ts as your directory structure and h264 as 
your file structure.


Incidentally, sticking this stuff in a .tar is probably okay - that's 
just another container, but sticking it in a .tar.gz is not, the gz is 
your codec and will make the file BIGGER in all probability.


Cheers,
Wol

Re: [gentoo-user] NAS and replacing with larger drives

[Wols Lists]

On 08/12/2022 13:31, Mark Knecht wrote:



On Thu, Dec 8, 2022 at 5:38 AM Dale > wrote:

 >
 > Howdy,
 >
 > I've pretty much reached a limit on my backups.  I'm up to a 16TB hard
 > drive for one and even that won't last long.  Larger drives are much
 > more costly.  A must have NAS is quickly approaching.  I've been
 > searching around and find some things confusing.  I'm hoping someone can
 > clear up that confusion.  I'm also debating what path to travel down.
 > I'd also like to keep costs down as well.  That said, I don't mind
 > paying a little more for one that would offer a much better option.
 >
 > Path one, buy a NAS, possibly used, that has no drives.  If possible, I
 > may even replace the OS that comes on it or upgrade if I can.  I'm not
 > looking for fancy, or even RAID.  Just looking for a two bay NAS that
 > will work.  First, what is a DAS?  Is that totally different than a
 > NAS?  From what I've found, a DAS is not what I'm looking for since I
 > want a ethernet connection and the ability to control things over the
 > network.  It seems DAS lacks that feature but not real sure.  I'm not
 > sure I can upgrade the software/OS on a DAS either.
 >
 > Next thing.  Let's say a NAS comes with two 4TB drives for a total of
 > 8TB of capacity from the factory, using LVM or similar software I
 > assume.  Is that limited to that capacity or can I for example replace
 > one or both drives with for example 14TB drives for a total of 28TBs of
 > capacity?  If one does that, let's say it uses LVM, can I somehow move
 > data as well or is that beyond the abilities of a NAS?  Could it be done
 > inside my computer for example?  Does this vary by brand or even model?
 >
 > Path two, I've researched building a NAS using a Raspberry Pi 4 8GB as
 > another option.  They come as parts, cases too, but the newer and faster
 > models of Raspberry Pi 4 with more ram seem to work pretty well.  The
 > old slower models with small amounts of ram don't fair as well.  While I
 > want a descent speed, I'm not looking for or expecting it to be
 > blazingly fast.  I just wonder, if from a upgrade and expansion point of
 > view, if building a NAS would be better.  I've also noticed, it seems
 > all Raspberry things come with a display port.  That means I could hook
 > up a monitor and mouse/keyboard when needed.  That could be a bonus.
 > Heck, I may can even put some sort of Gentoo on that thing.  :-D
 >
 > One reason I'm wanting to go this route, I'm trying to keep it small and
 > able to fit inside my fire safe.  I plan to buy a media type safe that
 > is larger but right now, it needs to fit inside my current safe.  Most
 > of the 2 bay NAS or a Raspberry Pi based NAS are fairly small.  They not
 > much bigger than the three external hard drives and a couple bare drives
 > that currently occupy my safe.
 >
 > One thing I'd like to have no matter what path I go down, the ability to
 > encrypt the data.  My current backup drives are encrypted and I'd like
 > to keep it that way.  If that is possible to do.  I suspect the
 > Raspberry option would since I'd control the OS/software placed on it.
 > I could be wrong tho.
 >
 > One last thing.  Are there any NAS type boxes that I should absolutely
 > avoid if I go that route?  Maybe it is a model that has serious
 > limitations or has other problems.  I think the DAS thing may be one for
 > me to avoid but I'm not for sure what limits it has.  Google didn't help
 > a lot. It also could be as simple as, avoid any model that says this in
 > the description or uses some type of software that is bad or limits
 > options.
 >
 > Thoughts?  Info to share?  Ideas on a best path forward?  Buy already
 > built or build?
 >
 > Thanks.
 >
 > Dale
 >
 > :-)  :-)

DAS is direct-attached-storage. I don't think you want that.


Depends. If it fits in the safe, and can be connected using one of these 
eSATA thingy connectors, it might be a very good choice.


Synology (sp?) is sort of a big name in home & small office NAS boxes. 
You can buy the boxes with or without drives. I suspect you won't like 
the prices.


I've been looking :-) I think the empty box costs more than the drives 
you're going to put in it ...


I wonder if you might consider what data on your backups needs to be 
immediately available and which doesn't. Possibly buy an 8TB USB drive, 
take a bunch of the lower priority data off of your current backup thus 
system freeing space and move on from there?


I built my NAS devices using old computers ala Wol's suggestion to me 
maybe a year ago. They work for me but don't have the fastest network 
interfaces.


I get the impression Dale isn't actually PLANNING his disk storage. It's 
just a case of "help I'm downloading all this stuff where do I put it!!!"


How much storage do you have in your actual computer? How much space do 
you need IN ONE PARTITION? Can you get an external disk caddy that you 
just slot bare drives in?


I've no doubt