Re: realpath quoting

[DdB]

Am 03.05.2024 um 21:11 schrieb David Christensen:
> I can obviously add an extra step to the process to convert the new file
> name to something acceptable before processing. However, my question was
> how to avoid that extra step by getting fully quoted filenames to process.

Today, on linux, i am using rename (perl based) to change spaces (in
filenames) into something else, or such...
HTH, DdB

Re: realpath quoting

[DdB]

Am 03.05.2024 um 21:11 schrieb David Christensen:
> I can obviously add an extra step to the process to convert the new file
> name to something acceptable before processing. However, my question was
> how to avoid that extra step by getting fully quoted filenames to process.

Not sure, if i get it right ...
Several years back, when i had to import huge amounts of files from an
outdated windows system, i found convmv to be somewhat helpful, as it is
able to adjust character sets in filenames only, without touching their
content.

gl, DdB

Re: realpath quoting

[DdB]

Am 03.05.2024 um 00:59 schrieb jeremy ardley:
> I have a need  to get the full path of a file that has spaces in its
> name to use as a program argument
> 
> e.g.
> 
> jeremy@client:~$ ls -l name\ with\ spaces
> -rw-r--r-- 1 jeremy jeremy 0 May  3 06:51 'name with spaces'
> jeremy@client:~$ realpath name\ with\ spaces
> /home/jeremy/name with spaces
> 
> 
> The spaces without quotes cause problems with subsequent processing.
> 
> Can realpath or other utility return a quoted pathname?
> 
> 
Do you mean something along those lines:
> $ touch '/mnt/tmp/gibt es nicht'
> $ realpath -mz "../ln/tmpRAM/gibt es nicht" | xargs -0 ls 
> --quoting-style=escape
> /mnt/tmp/gibt\ es\ nicht

?

Re: Debian does not load zfs automatically at boot and strange messages displayed on the screen...

[DdB]

Am 30.04.2024 um 16:48 schrieb Mario Marietto:
> Probably this is not the proper method to do it ?
Done it in vm's and on bare metal many times. Never ran into your kind
of problems. :-(

Here is the guide, i suggest:
https://openzfs.github.io/openzfs-docs/Getting%20Started/Debian/index.html#installation

SOLVED (was: Re: using mbuffer: what am i doing wrong?)

[DdB]

Am 11.04.2024 um 15:49 schrieb Marc SCHAEFER:
> Hello,
> 
> On Tue, Apr 09, 2024 at 03:13:01PM +0200, DdB wrote:
>> from my research, the abbreviated takeaway is:
> 
> I never used mbuffer, I use buffer combined with netcat-traditional:
> 
># receiver (TCP server on port 8000)
>nc -l -p 8000 | buffer -S 1048576 -s 32768 -o /dev/null
> 
># sender (TCP client on ephemeral port)
>nc localhost 8000 < /dev/zero
> 
> I just installed mbuffer:
> 
>mbuffer -I 8000 -o /dev/null
> 
>mbuffer -i /dev/zero -O 127.0.0.1:8000
> 
> and it also works.
> 
>>> sudo netstat | grep $port
>> to return nothing
> 
> yes, but those work:
> 
>netstat -a | grep :8000
> 
>netstat --listen | grep :8000
> 
> Maybe it's just that by default netstat only shows sockets in the
> ESTABLISHED state and not in the LISTEN state.
> 
>> What am i doing wrong?
> 
> If there is a timeout, I would suggest to investigate firewalls
> on the server side.
> 
> 

Thank you for your (and everyone else)'s hints.
It is working now. The lessons learned:
- in fact netstat is somewhat outdated, ss is the replacement (package
iproute2 instead of net-tools)
- yes, and my attempt at checking just failed, as you kindly pointed
out, due to missing out on listening ports. LOL
- when using mbuffer from a pipe, it is important to say "-i - " in the
arguments (missed that at first)
- the resulting transfer is way faster than say ... ssh.

Re: using mbuffer: what am i doing wrong?

[DdB]

Am 09.04.2024 um 15:30 schrieb Arno Lehmann:

> I'd propose to use
> 
> ss -f inet -lpn
> 
> ss instead of netstat... I try to catch up with changing times :-)
(...)
> 
> Arno
> 
> 

Thank you so much! Your suggestion did help big time, and the transfer
is working now as desired.

Great relief and a good lesson: not to install net-tools, but iproute2
instead. :-)

Have a nice day!
DdB

using mbuffer: what am i doing wrong?

[DdB]

Hello list,

from my research, the abbreviated takeaway is:

> port=8000 # just an example
> filename=test.bin # created before
> 
> # Start the receiver first, like:
> mbuffer -I $port -o $filename
> 
> # Then start the sender like:
> mbuffer -i $filename -O ${receiverIP}:$port

On my LAN (all virtual, at the moment), this fails, because the sender
cannot connect to the receiver, so times out.

What left me puzzled is this:
Just starting the receiver, and then checking, if it is listening at the
$port, i found:
> sudo netstat | grep $port
to return nothing

At that point, i decided, i would appreciate some hint from people more
experienced than i am.

What am i doing wrong?
DdB

Re: Why LVM (was: HDD long-term data storage with ensured integrity)

[DdB]

Am 08.04.2024 um 23:08 schrieb Stefan Monnier:
> David Christensen [2024-04-08 11:28:04] wrote:
>> Why LVM?
> 
> Personally, I've been using LVM everywhere I can (i.e. everywhere
> except on my OpenWRT router, tho I've also used LVM there back when my
> router had an HDD.  I also use LVM on my 2GB USB rescue image).
> 
> To me the question is rather the reverse: why not?
> I basically see it as a more flexible form of partitioning.

As an LVM-newbie (never used it before, i am more familar with ZFS), i
did already collect quite a bit of misconceptions of mine/design
problems with lvm. Therefore i would rather renew the question: Why?

Just one example:
In order to be able to use thin snapshots on my root partition, i did
every thing i could, to have it inside a thinpool... until i noticed
some weird problems booting from it (attributed to grub), so i setup a
/boot outside, but the problems stayed (due to lvm's limitations).

I came to use it to gain some flexibility (although it is an experiment)
and found myself setting up zfs for its data integrity + flexibility,
just to have a quality backup of the lvm-volume(s) on a zfs pool.

> 
> Even in the worst cases where I have a single LV volume, I appreciate
> the fact that it forces me to name things, isolating me from issue
> linked to predicting the name of the device and the issues that plague
> UUIDs (the fact they're hard to remember, and that they're a bit too
> magical/hidden for my taste, so they sometimes change when I don't want
> them to and vice versa).

Even GPT brings you the chance to name hings (like part_label), only it
does not force you. But i have been using that for 10+ years as a routine.

DdB

Re: making Debian secure by default

[DdB]

Am 27.03.2024 um 22:30 schrieb Lee:
> oof.  Are there instructions somewhere on how to make Debian secure by 
> default?

To be honest: I did not read this thread, as my spidey senses got
tingling. IMHO Even the idea/concept, that such a thing would be
possible, is broken.

Sounds like: Get me a car, that never kills anybody, no matter how
stupid the driver is... (impossible).

But to slow it down a bit:
As a former software engeneer, i learned quite a few things, among the
important ones were the lessons around "Cleanliness" (donno, if this
word even exists).

>From time to time, even experienced devs come across a choice:
Either, they work on very clean contracts between software parts, which
in an ideal world lead to good interfaces, but this makes some difficult
tasks impossible, as it may be impossible to agree upon such a contract
with everyone involved (or the whole world).
Or, they lean more towards what is feasible, preferring tricking or
evading from contracts, like what most hackers do.
My own recipe in such cases, was to feel my inner work ethic and to
trust it more than anything. But i do understand, that such a solution
is bound to be changing and thus is not really a good solution in general.

But that leaves us with:
There is no way, a secure system will ever come to be. We need to take
responsability for what we are doing, as even a hammer can be used to
kill. Sorry, that means we have to use our brains, learn from mistakes
and do the best we can without falling asleep.

Just my 2 cents
DdB

SOLVED (was: Re: help needed to get a bookworm install to succeed)

[DdB]

Am 01.04.2024 um 18:52 schrieb David Christensen:
> A bad USB flash drive would explain why you cannot boot the Debian
> installer.  Please buy a good quality USB 3.0+ flash drive and try again.

A friend of mine just let me use an external CD-Drive with the netboot
image. This is already the third time, i am restarting the installation
process, due to my false assumptions about the intelligence within the
installer.

The last time, i was quite happy until i came to notice, that partitions
were not aligned with physical sector boundaries, which i assumed would
be elementary best practice.

But apart from losing some of my illusions the hard way, all is well.
A big thank you to all the crowd offering suggestions and encouragement.

so long, DdB

Re: help needed to get a bookworm install to succeed

[DdB]

Am 01.04.2024 um 07:44 schrieb David Christensen:
> 
> 
> A computer with a 6-core processor, 64 GB memory, and 9 drive bays/
> ports that cannot boot USB?  That does not make sense.

Why not?

> 
> 
> Please post a console session that identifies the ISO you are using,
> verifies the checksum, burns the ISO to a USB flash drive, and compares
> the ISO against the flash drive.

Ok, in the meantime, i came to similar conclusions and found that the
USB-stick i was using, had consistent read errors at the first 2
gigabytes after having been used for years as memory extension in my
router. Fixed that and will replace the stick.

> 
> 
> Then insert the USB flash drive into a USB port on the the target
> computer, power up and enter Setup, reset the settings to factory
> defaults, enable USB booting, set the USB flash drive as the first boot
> device, save, and exit.  The Debian installer should then boot.

*should* is the correct word. The board being over 10 years old, it does
not offer USB booting, no way. It is an early server board that supports
that much ECC, which is great for zfs.
> 
> 
> David

But i received many hints and ideas and just have to wait for a friend
of mine to overcome my physical handicap to see some progress. :-)

Tx 2 everyone
DdB

help needed to get a bookworm install to succeed

[DdB]

Hello list,

i intend to create a huge backup server from some oldish hardware.
Hardware has been partly refurbished and offers 1 SSD + 8 HDD on a 6core
Intel with 64 GB RAM.
Already before assembling the hardware, grub was working from the SSD,
which got lvm partitioning and is basically empty. As i have no working
CD drive nor can this old machine boot from USB, i put an ISO for
bookworm onto an lvm-LV. Using grub, i can manually boot from that ISO
and see the first installer screens. But after asking some questions,
the installer wants to mount the external media (ISO), and does not find
it on sd[a-z], then aborts.
By switching to Desktop 4, i can see the attempt to search for the
"CD"-drive, which is bound to fail.
I am not familiar with the very restricted shell, that is available from
the installer (busybox) and have not yet found an approach to circumvent
my problems. i would like to use the installer, as debootstrapping would
necessitate alot more knowledge than mine.

Suggestions are welcome :-)
DdB

Re: partition reporting full, but not

[DdB]

Am 19.02.2024 um 04:20 schrieb Keith Bainbridge:
> I am convinced that the missing space is used by btrfs snapshot process.

First off: I am not a btrfs user (and will never be, i might add).
I am using zfs since many years, and - although i read an awful lot of
documentation beforehand, and played around with it quite a bit, i had
to learn a couple of lessons "the hard way".

To me, what i am inferring from your mails, are a couple of conceptual
misundestandings of what a COW-filesystem is, and how to best make use
of it. Especially the time-machine using snapshots and hardlinks seems
very kinky to me.

I am using zfs somewhat in a similar way, but it took me some years to
set it up in a good way. Apart from a warning and the usual RTFM, i do
not have much to offer. Except maybe for this: Snapshots hold space,
they are not like hardlinks. (although one can snapshot hardlinks ;-) )

Once, some older filesystem state is snapshotted, its space is taken
until the snapshot gets destroyed/removed. This can lead to situations,
where a device is full, but in order to free some space, deleting files
will NOT help, because in order to do so, a change in the directory
needs to be made, but there may not be the room to create a copy of it
in the first place.
And deleting files from a snapshot is not feasable (at least not in zfs).

To my eyes, your handling of the system looks somewhat risky and not
well planned. You may go on this way, as long as you consider the whole
thing as being part of a longish learning session. But do not trust
important data being safe this way!

just my 2 cents

Re: Copy from Firefox and paste into Terminal with Vim

[DdB]

Am 06.02.2024 um 00:14 schrieb David Christensen:
> Comments or suggestions?
This may be unrelated, but ...
I can copy/paste using the mouse, or - if i use the keyboard - i need to
copy paste using CTRL-Shift-C and CTRL-Shift-V (when in the terminal
emulator like gnome-terminal or terminator)

Re: Request about Boot Repair Disk

[DdB]

Am 27.01.2024 um 19:03 schrieb fran...@libero.it:
> (it is a long report)
This is interesting ... BUT i lack the required experience:
Neither do i use Windows nor a notebook, but i would guess, that the
last paragraph from your report contains the root cause of your problem:

> Please do not forget to make your UEFI firmware boot on the Debian GNU/Linux 
> 12 (bookworm) entry (sda1/efi//shim.efi ( will be updated in the 
> final message) file) !
> If your computer reboots directly into Windows, try to change the boot order 
> in your UEFI firmware.
>  
> If your UEFI firmware does not allow to change the boot order, change the 
> default boot entry of the Windows bootloader.
> For example you can boot into Windows, then type the following command in an 
> admin command prompt:
> bcdedit /set {bootmgr} path \EFI\\shim.efi ( will be updated in 
> the final message) > 

As you already found by yourself, both systems are setup to be bootable
in uefi mode, and uefi by default has some way of making a choice
(either at boot time of through changing the boot order permanently)
GRUB is not strictly necessary for this to happen, but it can be used in
the mix. What would be very good to know, is, how is your firmware
manufactured and what does it allow? How does it cooperate?

If you want to get a better understanding of the UEFI boot process (with
or without grub), i recommend reading the documentation at
https://www.rodsbooks.com/linux-uefi/

good luck

Re: How to compare contents of two folders against third one?

[DdB]

Am 02.11.2023 um 08:36 schrieb Alexander V. Makartsev:
> Rsync wasn't helpful in my case. It refused to recognize file names
> properly, complaining about charset encoding, not even with --iconv
> parameters.
> Probably because of locale differences between my system and mounted
> filesystems and who knows what else.

In that case, i suggest to have a look at fdupes. I am using it for the
same purpose with success since long time.

cheers, DdB

Re: Unattended upgrade of grub failed

[DdB]

Am 08.10.2023 um 12:46 schrieb DdB:
> OMG, this looks wrong to me.

Errata:
I apologize for spreading nonsense. My mistake was, that, although most
of my vm's are actually using UEFI, the one, i was testing was NOT.
Sorry for my confusion. Disregard my earlier statements ...

Re: Unattended upgrade of grub failed

[DdB]

Am 08.10.2023 um 11:33 schrieb Marco M.:
> Specifying a block device for boot loader installation is only needed
> for BIOS boot and not for UEFI boot, so in case of an UEFI system,
> grub-pc isn't needed.
> 

OMG, this looks wrong to me. So i went to check in practice, using a
virtual machine, that is using UEFI booting.
If i try to remove grub-pc, it is issuing several warnings, among them
the question, if i REALLY want to remove grub2 from /boot/grub, which is
strange, because grub exists in the efisys partition (/dev/sda1 in my
case). But the list of dependencies contains - among other things -
every single kernel ever installed on the system, indicating, that
removing it would necessitate another boot manager to be installed
afterwards.
As mentionned, the VM uses UEFI, thus it is set up properly using grub2.
But apparently, both are present and somehow intertwined. I did refrain
from completing the purge.
Yes, my understanding also is, that grub-pc was the old way, but i did a
clean install of a buster system on UEFI and grub-pc got there without
me asking for it.
My understanding is, that it might be dangerous to uninstall grub-pc and
its dependencies without further precautions.

Re: Unattended upgrade of grub failed

[DdB]

Am 08.10.2023 um 11:09 schrieb Jesper Dybdal:
> This has worked fine since then, with unattended-upgrades succeeding in
> keeping it up-to-date, including kernel upgrades and reboots.
> 
> But this morning, unattended-upgrades failed:

Hi, i was having the same problem on several virtual machines, that i am
running. On issuing the update interactively, i could see, that grub was
prompting for the place to write itself to, which needed interaction.
After giving the info (like /dev/sda in my case), the upgrade succeded.
Later, there came more updates, but this time, they could indeed install
themselves automatically.
That is why i suggest updating interactively this one time for grub and
relateds to succeed...
HTH, DdB

Re: Need help with PGP signature verification

[DdB]

Am 08.10.2023 um 01:16 schrieb Tom Browder:
> I'm willing to trust published PGP key fingerprints for signers of
> Rakudo downloadable files.
> 
> Question:  How can I get the fingerprint from the downloads? 
> 
> The products I download are (1) the file of interest, (2) a PGP signed
> checksums file with various shaX hashes for the file, and (3) a separate
> file containing a PGP signature.
> 
> Thanks so much.
> 
> -Tom
> 
> 
> 
There is more than just one way to archieve this, first result from
G**-search returns:
https://superuser.com/questions/1297670/how-do-i-check-gpg-signature-given-only-the-fingerprint-and-key-id
which also contains security related warnings and hints.
HTH, DdB

Re: Letting Windows go: scanning

[DdB]

Am 20.09.2023 um 19:06 schrieb Tom Browder:
> So how can I get my Debuian host to see and use the scanner part?

No answer from me. Just saying: My situation is similar, but YMMV.
I was using a scanner from MS Windows since almost 20 years. I am on
debian since about 5 years, but never got the scanner to work.

The workaround - for me - is to keep a Windows VM with outdated Win7,
scanner software and configuration, which i use to scan, whereas i can
print to the scanner/printer directly from debian. Certainly a bit
weird, especially the "virtual USB configuration" that is supported
inside my router for this to work, but it does since a long time and
needed adjustments only once in a blue moon.

So i am keeping this very old (scanner-) hardware functional, which has
a laser printer (no inkjet) as long as i still have toner cartridges (no
longer available from producer).

good luck with your endeavor.
DdB

Re: [a bit OT] Automate a (G o o g l e) search from a list of strings

[DdB]

Am 19.09.2023 um 16:41 schrieb steve:
> I guess I could code a Python script to do that but if something already
> exists I'd rather use it.
Really, your wording is very vague and thus, i donno, if i even
understand, what your trying to accomplish.

But from what my phantasies entice, i would maybe use wget to search one
site, eventually modify the output with awk, and parallelize it all for
several websites by the means of GNU parallel, expecting to create a
one-liner for it.

just my 2 cents
DdB

Re: Using the bash shell: determine if the root user used 'sudo -i'

[DdB]

Am 26.08.2023 um 16:25 schrieb Tom Browder:
> Is there a way to distinguish whether 'sudo -i' was used or not?
> 
Sorry, i am not an expert on this. But ... since years i am using this
to check for it:

> # if `echo $HOME` is not "/root" or the working dir (pwd) is not "/root", 
> then this was not executed with "sudo -i"
>   assert "echo $HOME" /root "nicht mit sudo -i aufgerufen"
>   assert pwd /root "nicht mit sudo -i aufgerufen"

hope, this will give you a clue ;-)
DdB

Re: Fwd: Problem verifying iso file

[DdB]

Am 24.06.2023 um 00:09 schrieb Thomas George:
> I tried md5sum SHA512SUMS.txt debian-12.0.0-amd64-DVD-1.iso
> 
> The outputs do not match

Seriously?

i would have tried
sha512sum -c ShA512SUMS.txt
in the folder, where the iso can be found.

gl next time
DdB

Re: VirtualBox key is store in deprecated legacy keyring

[DdB]

Am 20.06.2023 um 04:58 schrieb Rick Thomas:
> PS: As an aside,  it appears that the VirtualBox developers at Oracle waited 
> until Bookworm was officially released before they started working on getting 
> a bookworm version of their software, so I'm still using the Bullseye version 
> -- which seems to work fine.  Presumably, fixing this problem would be one of 
> the things they might want to do before releasing a new version...  (One can 
> hope, anyway...)  Would it be worth filing a bug-report to Oracle?  If so, 
> does anyone know how to do that?

I used to use their forum first in order to get confirmation for the bug
existing, before actually filing a bug on their public bugtracker
https://www.virtualbox.org/wiki/Bugtracker

Re: package managers problem

[DdB]

Am 17.06.2023 um 14:38 schrieb Greg Wooledge:
> On Fri, Jun 16, 2023 at 11:03:59PM -0400, pa...@quillandmouse.com wrote:
>> Why isn't there a ONE WAY for packages to be managed?
> 
> Because each user has a different preference.  Just read this thread
> for example, and see all the differing opinions about how we like
> our packages to be managed.
> 
> 
Back in the days, i have been using synaptic myself. And after reading
this thread, i was curious about its appearance today. Thus, in my
virtualised buster install, i did log out of wayland and logged in as
"system X11 default" (under the wheel on login screen). Then, i was able
to install synaptic and use it, although the gnome-software mentionned
its source being debian-old-oldstable-main.

I seem to have left it behind since stretch, when i moved to commandline
tools instead. When things got really hairy, i recall having used
aptitude on few occasions, but only after thorough experimentation in
virtualized world, before applying the steps on bare metal.

But since most of you are on bullseye, with a huge crowd even already
landing in bookworm, this report from an outdated software will most
certainly be neglected. FWIW, i chose to let you know anyway.

DdB

Re: Please help with not booting from USB so to install Debian

[DdB]

Am 08.06.2023 um 20:36 schrieb Rodolfo Medina:
> I want to install Debian on a new machine but don't manage to boot from USB
> stick.  (I can do so regularly with another machine, so the USB stick is ok 
> and
> so is the Debian netinst I burned onto it.)  At the boot I press F9 and a menu
> appears where I can choose to boot from USB stick; but then it doesn't so at 
> all
> booting instead into Windows 11.  In BIOS I enabled the CSM protocol but
> nothing.  Please help as I don't know what to do: thanks.
> 
> Rodolfo
> 
> 
Sorry, i have no experience with this concrete issue. Last time, i
experienced USB boot issues, it did help to just use another USB-port
instead. ;-)

But honestly, i see many hints and advices are coming, but still the
cause of the problem has not yet been identified, which would be my
first attempt in any case.

Since you confirm, that booting from the stick is possible on another
machine, it cannot be, that the stick itself doesnt work. But does that
other machine also have the same processor architecture (celeron ...)?

IIRC it is possible to boot a 32-bit OS on a 64 bit machine (not
recommended, but possible), whereas the other way round would just fail.

Sure, in here Windooze is regarded as a nuisance, but even windows can
inspect its boot config and determine, how the machine is coming up, if
the disk has GPT or MBR (probably the first), and you might be able to
find out, if your setup allows ANY booting from USB (like non flash
storage).

Even without aspiring for completeness, i was suggesting different areas
being possibly of concern, like
- hardware (usb, processor)
- bios (setup)
- version mismatch (partition tables, OS-architecture)
- there may be more (like secret holes leading to hidden switches)

Once the cause will be identified, it is going to be much easier (or
just plain impossible) to remedy the problem. And please report back,
what you found.

Re: how to change boot screen image?

[DdB]

Am 09.06.2023 um 09:05 schrieb hlyg:
> 
> On 6/9/23 09:01, DdB wrote:
>>
>> STFW: https://www.youtube.com/watch?v=xAi2UfUkp_A
>>   +
>> https://unix.stackexchange.com/questions/349234/how-to-show-kernel-boot-messages-by-modifying-grub-config-files
>>
>>
>> .
> 
> Thank DdB! i have some success
> 
> /usr/share/desktop-base/homeworld-theme/grub/
> 
> boot screen can be changed by replacing file in location above
> 
> i haven't been able to change login background
> 
> 
Very strange: if i use my favorite search engine, i find dozens of
places, describing a solution. And especially nowadays, where AI
specializes on giving useful hints and explanations to remedy computer
configuration problems, i am wondering as to why you keep asking here?

I have neither lxde nor lightdm, so cannot even verify those solutions,
but you should ... beginning with perhaps:
https://www.marksayson.com/blog/lxde-customizing-the-log-in-background/

Re: how to change boot screen image?

[DdB]

Am 09.06.2023 um 01:45 schrieb hlyg:
> i've installed deb11 for i386 from live lxde CD
> 
> i want to change boot screen image
> 
> i browse /boot/grub, can't get clue
> 
> lxde has many menus to change appearance, but i can't find menu to
> change boot screen
> 
> btw is it possible to show kernel msg during boot?
> 
> 
STFW: https://www.youtube.com/watch?v=xAi2UfUkp_A
 +
https://unix.stackexchange.com/questions/349234/how-to-show-kernel-boot-messages-by-modifying-grub-config-files

Re: UUIDS (addenum)

[DdB]

Am 28.05.2023 um 01:37 schrieb mick.crane:
> (...)
> and gives intramfs prompt.
> Am I supposed to be able to sort it out from there?
> like how?
> mick

oh the initramfs prompt... that is not the place to fix it easily.
much easier to stop while booting grub (like asking for the menuitem,
you are used to boot into, read that and next get a grub prompt (c) and
boot the system by hand. Of course, i did play with grub before in order
to be accustomed to do that in case of emergency. the grub help command
is just barely enough to be reminded of what you already know, not more.

And you should know, thet the idea behind UUID#s (Unique Universal
Identifiers) is, that they should be unique (at least on one system).
maybe just removing one of the duplicates can be enough to circumvent
the error?

Re: UUIDS

[DdB]

Am 28.05.2023 um 01:37 schrieb mick.crane:
> I'm sure it used to be that you could swap linux discs between PCs and
> it would sort itself out but I try swapping disks about and booting and
> they complain
> "Cannot find UUID..lots of identifying numbers"
> and gives intramfs prompt.
> Am I supposed to be able to sort it out from there?
> like how?
> mick
> 
> 
I am juggling with disks and partitions frequently. And would like you
to be awarem that there are different UUID#s involved. One resides in th
filesystem ext2/3/4 itself, and can be modified with tune2fs -U (while
fs is at rest/unmounted)

The other one site inside GPT, but you can change that also ((s)gdisk is
your friend).

But all that is meaningless, if you dont find the other end of the
equation (like fstab, or grub scripts) that reference the said UUID's.

But step by step you will get there, just be prepared to reboot several
times and work your way up the chain.

Re: GPT backup table, was Re: virtualisation

[DdB]

Am 26.05.2023 um 11:47 schrieb Thomas Schmitt:
> (...) 
> (And as mick.crane already noticed, it is a bit awkward to create an
> extended partiton 2 only to fill it nearly up with logical partition 5.
> I wonder what entity decided to do so.)
> 
About one year ago, i was helping some neighbor at setting up an old
notebook to use linux, XUbuntu in that case.

I was quite surprised to find out, that the installer had done exactly
that: Setup MBR, with extended partition, where only swap was in.
After making sure, that notebook could handle UEFI booting properly, i
allowed myself to find out, if it was possible to change partitioning to
GPT and booting to use EFI. - Yes, i did not need to backup/restore the
partition contents, just carefully moving it to make room for EFISYS
partition and such, and installing the EFI support was enough and the
machine is working happily ever since.

There may be other entities, but apparently in 2022 the Ubuntu installer
still decided to go very old-fashionned about its job. ;-)

Re: os-prober Just a Rant

[DdB]

Am 25.05.2023 um 19:28 schrieb Andrea Borgia:
> 
> updates cause my edits to be overwritten... that sucks
> 
> 
> Ah, ok, I wasn't seeing ghosts, then!
>  

I am using os-prober sometimes, but i am aware, that it fails almost on
a regular basis on machines making use of ZFS. That has to be expected,
because ZFS is not very much interested in "partitions", not even in
"disks" but its main unit (a pool) is built out of one or more vdevs
(virtual devices) which can be anything from a regular file to a bunch
of disks. Certainly neither grub nor os-prober know, how to deal with
that properly, which is why i have been fixing grub.cfg after each
grub-update, and i set up my system to do so automatically using some
script, that does split the output from grub-mkconfig into pieces, sed
some parts of them and reassembles the resulting files into a working
grub.cfg.

Maybe i should not call os-prober broken (or grub, for that matter) but
people working on it certainly do not consider the wide range of
workflows making use of grub's features, but i consider it insufficient
for my needs big time.

FWIW - you are not alone.

DdB

Re: What does "freeze" mean in Debian?

[DdB]

Am 25.05.2023 um 10:06 schrieb Hans:
> Hi folks, 
> 
> just a little thing, I am somehow confused about.
> 
> I read that debian/testing is now in state "freeze" as the next release is 
> shortly to come.
> 
> As I running "bookworm" now, I am wondering, that debian/testing (aka 
> bookworm), still gets a lot of changed packages last days. 
> 
> Obviously I seem to misunderstand the meaning of "freeze". 
> 
> Does "freeze" mean "No new packages" od does it mean "actual packages in 
> testing will not be changed any more till next release". 
> 
> What did I not understand? For me "freeze" means "stay at actual status and 
> do 
> only necessary changes for security or breaking reasons".
> 
> There are almost a hundered packages I got untill "freeze" and my change to 
> bookworm (aka testing).
> 
>  
> 
> 
> 

https://release.debian.org/bookworm/freeze_policy.html
-- 

Liebe ist ...
Datakanja

Re: Ok so Now which backup should I use

[DdB]

Am 16.05.2023 um 02:17 schrieb Maureen L Thomas:
> I have everything I need including a third HDD.  There are so many
> backup programs I have to wonder which one will work for my needs.  I
> just need to make a backup of my home directory so if I do something
> stupid like play with /var and have no idea how to fix it.  Is there
> something else I need to back up besides /home?  I appreciate your help.
> 
> Moe
> 

TBH, I have always preferred to use my own hand-scripted backup using
standard tools, bacause i am willing to tailor it to special use cases i
might encounter. And a basic backup has to offer the feature to restore
in any way, i want.
But, just recently, i decided to create a backup for my neighbor on
their laptop, and did not wwant to write everything from scratch. After
searching a bit, i opted for fsarchiver, a simple tool, that is already
on some ubuntu live cd. Learning its commandline is not so difficult,
the biggest problem AFAICT is, that there is only one (french) developer
and thus the tool has not seen a huge amount of public testing. But i
pucked it anyway, had several backups and - who would have thought:
several restore operations carried out since. And i am happy with the
choice. You might take a look as well: https://www.fsarchiver.org/

just my 2 cents, DdB

Re: sudoers question

[DdB]

Am 13.05.2023 um 00:03 schrieb Lee:
> On 5/12/23, Stefan Monnier  wrote:
>>> Or configure sudo to disable tty_tickets, so that the timeout (10
>>> minutes by default IIRC) applies to all terminals.
>>
>> `sudo bash` anyone?
> 
> me!  me!  but I also have
(...)
> %adm  ALL = (root) NOPASSWD: ADM_COMMANDS

Of course, there are ways to allow any/all sudo commands without
password. And i also have to cast a warning here:

The kind of mistakes, any user (including yourself) can initiate, grows
considerably, if he can use any commands without even thinking.

To my eye, as there is a huge responsability involved with using
elevated powers, i would not want "my little brother" to accidentally
sit in front of my computer while just trying commands at a console,
that he may have heard of somewhere.

Even worse: When i found out, how to prevent sudo from asking a pwd, i
in fact did cause a couple of bad mistakes, that the system would
otherwise have prevented from happening (including making it
unbootable). And it took my quite some time in order to get used to some
kind of a routine, that keps me from having to reinstall everything from
scratch after each mishap.

So, after some time, i have become way more cautious at allowing too
many powers to myself without thinking. And especially the OP did reveal
some contradictory habits:
He was asking, how to allow any sudo command without being asked for a
password ( which means: without being controlled by the system ). On one
hand, this could make sense under certain premises.
OTOH, he was failing to display any kind of responsible attitude for the
job (like as if reading logfiles was hs only interest ...).

Just simply asking for help in this regard let me wonder, as i had been
able to find out all this without even knowing about his group,
including the relevance of sudoedit in this regard (which no one even
mentioned).

You can't have your cake and eat it too!

If we (as a community) would support such a behavior, wouldn't we be
responsible for the effecs, this entails?
Would you hand out a loaded weapon to a child? (I certainly did not.)

just saying ...
good luck
DdB

shame on me (was: Re: No fool like an old fool (debian installation probs))

[DdB]

Am 02.05.2023 um 05:50 schrieb David Wright:
> On Tue 02 May 2023 at 02:21:20 (+0200), DdB wrote:
>> Am 01.05.2023 um 21:38 schrieb David Wright:
>>> And PARTLABELs aren't interfered with even by the installer.
>>
>> This at least i can contradict for a fact.
> 
> So is this post the rebuttal, or are you posting the evidence elsewhere?
> 
>> VM is functional with known
>> and documented partlabels, then the installer handles partitions
>> (reformat is permitted) and the UUID's AND the PARTUUIDS were changed,
>> which was a huge surprise.

I cannot recall, how my impression was formed, as i did more than a
dozen trial installations in the last 3 days, not all of which i kept.

But a recent verification proves me wrong:

from original host:
> sudo blkid | grep nvm
> /dev/nvme0n1p1: UUID="F497-DC20" TYPE="vfat" PARTLABEL="EFISYS" 
> PARTUUID="1b3d99f5-b568-4445-8a09-854bb0ed1583"
> /dev/nvme0n1p2: UUID="1a4db22f-65e4-476a-b7e3-8f9f91cf0b8b" TYPE="ext4" 
> PARTLABEL="buster-main" PARTUUID="69941f2a-6d75-4662-b6ae-3e98425ff463"
> /dev/nvme0n1p3: UUID="ec317602-8175-46b1-bced-7be335f59a01" TYPE="ext4" 
> PARTLABEL="SOS" PARTUUID="5f04ff28-c5e4-47c4-b9a5-c4820bb01f6a"
> /dev/nvme0n1p4: LABEL="bpool" UUID="9675743913905685606" 
> UUID_SUB="6515248399914718640" TYPE="zfs_member" PARTLABEL="bpool" 
> PARTUUID="c391b3ac-4460-4dd6-92e7-3ff8def7b07b"
> /dev/nvme0n1p5: LABEL="rpool" UUID="2471356229651970105" 
> UUID_SUB="13336611508289702781" TYPE="zfs_member" PARTLABEL="rpool" 
> PARTUUID="de2e5e4e-382c-4c5b-b2fd-60fcc2a71229"
> /dev/nvme0n1p6: UUID="dd0919fe-8575-483c-b90a-20c7ff65e43a" TYPE="swap" 
> PARTLABEL="swap" PARTUUID="da10bfac-a3e4-4847-a53f-7d0547801b91"
> /dev/nvme0n1: PTUUID="9c76aed7-574b-48b6-9309-62d74e5303cb" PTTYPE="gpt"

from the simulating vm (ancient snapshot):
> sudo blkid | grep sda
> /dev/sda1: UUID="F497-DC20" TYPE="vfat" PARTLABEL="EFISYS" 
> PARTUUID="1b3d99f5-b568-4445-8a09-854bb0ed1583"
> /dev/sda2: UUID="1a4db22f-65e4-476a-b7e3-8f9f91cf0b8b" TYPE="ext4" 
> PARTLABEL="buster-main" PARTUUID="69941f2a-6d75-4662-b6ae-3e98425ff463"
> /dev/sda3: UUID="b9548515-9271-4d3f-91f0-d06b47375107" TYPE="ext4" 
> PARTLABEL="SOS" PARTUUID="5f04ff28-c5e4-47c4-b9a5-c4820bb01f6a"
> /dev/sda4: PARTLABEL="bpool" PARTUUID="c391b3ac-4460-4dd6-92e7-3ff8def7b07b"
> /dev/sda5: PARTLABEL="rpool" PARTUUID="de2e5e4e-382c-4c5b-b2fd-60fcc2a71229"
> /dev/sda6: UUID="1fcd07ac-9764-4970-99dd-5d133b95ec67" TYPE="swap" 
> PARTLABEL="swap" PARTUUID="da10bfac-a3e4-4847-a53f-7d0547801b91"

from the current vm (with bullseye in it):
> /dev/sda1: UUID="F497-DC20" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="EFISYS" 
> PARTUUID="1b3d99f5-b568-4445-8a09-854bb0ed1583"
> /dev/sda2: UUID="017cf209-aff6-45ec-a45e-bd293a34c49c" BLOCK_SIZE="4096" 
> TYPE="ext4" PARTLABEL="buster-main" 
> PARTUUID="69941f2a-6d75-4662-b6ae-3e98425ff463"
> /dev/sda3: UUID="b9548515-9271-4d3f-91f0-d06b47375107" BLOCK_SIZE="4096" 
> TYPE="ext4" PARTLABEL="SOS" PARTUUID="5f04ff28-c5e4-47c4-b9a5-c4820bb01f6a"
> /dev/sda4: PARTLABEL="bpool" PARTUUID="c391b3ac-4460-4dd6-92e7-3ff8def7b07b"
> /dev/sda5: PARTLABEL="rpool" PARTUUID="de2e5e4e-382c-4c5b-b2fd-60fcc2a71229"
> /dev/sda6: UUID="a2739581-96b5-45e6-91a0-d63695b61770" TYPE="swap" 
> PARTLABEL="swap" PARTUUID="da10bfac-a3e4-4847-a53f-7d0547801b91"

I notice, that, even though i did change the name of the sda2 partition,
the PARTLABEl remained unchanged.
This proves my previous post wrong and i am sorry for the confusion,
this may have caused. You were correct all along.
DdB

Re: No fool like an old fool (debian installation probs)

[DdB]

Am 01.05.2023 um 21:38 schrieb David Wright:
> And PARTLABELs aren't interfered with even by the installer.

This at least i can contradict for a fact. VM is functional with known
and documented partlabels, then the installer handles partitions
(reformat is permitted) and the UUID's AND the PARTUUIDS were changed,
which was a huge surprise.

Just knowing about this, i can deal with the situation, but i think,
this is based on a false understanding of how a persistent
identification should be dealt with.

my 2 cents
DdB

Re: No fool like an old fool (debian installation probs)

[DdB]

Am 01.05.2023 um 19:46 schrieb David Christensen:
> Reading the above plus your previous post "Looking for
> inspiration/advice/best practices on system upgrade", it seems that you
> are making things too complicated.
> 
> 
> I would pick one machine, disable/ disconnect/ uninstall all of the
> drives except optical, install a zeroed 2.5" SATA SSD, make a decision
> regarding BIOS/MBR (legacy) vs. UEFI/GPT, run the Setup utility and
> configure CMOS/NVRAM settings accordingly, boot the Debian installer,
> pick "Install", partition the SSD manually with a 1 GB ESP (if doing
> EUFI/GPT), a 1 GB boot partition (ext4), 1 GB or larger swap partition,
> and a 12 GB root partition (ext4), at the "Choose software" page select
> "SSH server", select "standard system utilities", and deselect
> everything else.  After reboot, you should have a working Debian instance.

I am sorry, i do not really understand, what you want to point to.
Your suggestions are coming without even knowing, what my needs and
options are, and involve alternatives that look unnecessary in my view.

Just to give you an idea: my OS partitions are 20 GB and that seems to
be a little too small, i had to delete stuff in order to keep the
systems in good shape.
I did abandon MBR years ago, and look with some bemusement at the many
places, whare that old technology still lingers.
Physical un-/plugging at the machines always involves getting external
help, due to a handicap.
Maybe your feeling is correct and i am in fact complicating things, i
don't know. OTOH, i am dealing with 2 dozen disks, server grade HW,
DUAL-CPU EPYC, 128 GB ECC RAM, nvme-ssd, aso.
My understanding is, that precautions are in order to keep the system(s)
up and running and useable for someoone, who is bad at typing.

I have installed many debian VM's before, all of them with a desktop,
and many of them are in use while i am investigating, how to
replace/upgrade the host OS. Do you get the idea?

But thanks for voicing your concerns, somthing i am trying to understand
and consider. Just keep in mind, that if a problem arises, i might not
be able too deal with, i will be cut off from everything, no paper based
calendar even exists. So i am careful.

DdB

Re: No fool like an old fool (debian installation probs)

[DdB]

Am 01.05.2023 um 10:33 schrieb Michel Verdier:
> Le 1 mai 2023 DdB a écrit :
> 
>> Any suggestions/questions/hints from the power-users in here?
>> ... would be wildly appreciated ...
> 
> When installing you have to stop on your first problem. The others could
> be created from it. It's longer but easier to cope with. So give us full
> details on your first problem or choice you don't understand.
> 
> 
Thank you for being this clear.
In the meantime i changed my plan somewhat (as stated earlier, i am now
willing to defer the resolution of the PARTUUID topic till later, in
order to see, if the upgraded GNOME will be able to serve me well, just
as the buster one did). Also, i wanted to be certain, that i will be
able to deal with the zfs upgrade.

Today, i found, that GNOME extensions are causing quite a bit of
trouble. The cause being, that the one extension, i have been using very
extensively ("Quick-Toggler") has been orphaned and is way out of sync
with current gnome-shell. That really hurts, as it had already been a
replacement for another extension, that had been orphaned in jessie
IIRC. - This is already the second time, that i have to find adjustments
in my workflow after an upgrade, due to changing functionality and lack
of compatibility. ... makes me somewhat unhappy, as i have been involved
 quite a bit in software rollout processes and we cared very much for
(paying) customer satisfaction. Ofc, this could be different in a
project based on the workforce from volunteers.

But it got even worse:
The site extensions.gnome.org let me know, that it cannot sync
extensions with my gnome-shell, because of a version mismatch. So i read
up a bit to understand, what is going on, and found the most concice
description here: (1)
Funny little detail: Ubuntu is suffering way less, because their
distribution is based on more recent version than debian stable is.

At least i saw some guy posting a video on youtube showing, how he
upgraded gnome-shell to experimental thereby creating a frankendebian,
something, i thought was an absolute no-go!

So i could not resolve the issue (You need to update
gnome-browser-connector to at least version 42.0), because the version
from debian is something like 10.1 or so.

Once again, i am happy to just playing around, not touching the
workhorse, that is still on buster, until i find a safe way forward.

zfs seems to be working fine on bullseye, the dev from the Quick-Toggler
published, that he abandonned the software due to his switch to KDE a
few years back.

Now, i am considering, if i should investigate the other desktops before
making a decision?

(1)
https://bugs.launchpad.net/ubuntu/+source/chrome-gnome-shell/+bug/1983851

Very sad, that the upgrade turns out to be rather difficult for a human
being like me, that is more or less isolated in his basement with no
friends sharing the passion for computing, or debian - for that matter.

So this is my todays report. Confused and undecided, i will head over to
get some rest now.
Ofc, i am curious to learn, how the crowd of "standard users" is dealing
with gnome extensions atm.

No fool like an old fool (debian installation probs)

[DdB]

Am 01.05.2023 um 10:23 schrieb Michel Verdier:
> Le 1 mai 2023 DdB a écrit :
> 
>> But omitting GNOME from the list lead to a system failing to boot with
>> tons of messages stating the absense of all kind of gnome parts.
> 
> To install without gnome I select the task ssh server then after I
> manually select what I want, and a WM if needed.
> 
> 
Thank you for encouraging me. A good night sleep did help as well.
Following your advice, i retried a fresh install. This time round, i did
permit the reformatting of everything, and just noted a new todo to
resolve that later. But ...

The install succeeded, an sshd is running ON a machine with GNOME and
all its (un-) pleasant surprises like: firefox, openoffice, and much
more. But i know for sure, that i deselected it and only left Desktop +
ssh server. I guess, that means, that the installer chose gnome as its
preferred desktop.

Ok, i will go on from here, as i certainly want a DE + GNOME, only the
standard tools, i wouldnt have all of them.

==

One explanation of my choice to stick with PARTUUID's:
In my overall stategy of using my computer, i am sometimes copying (dd)
whole partitions into a backup, a secondary partition or a VM, which can
easily lead to difficulties, if the same LABELS were used, which can be
changed, but reside INSIDE the partition itself, whereas PARTUUID's
reside in the GPT, which is not affected on partition copy operations.
Thus my choice was related to convenience, avoiding some adjustments
after partition copy operations. Only the installer interferes with the
so called "persistent" PARTUUID's, which i regard as being offensive (or
ignorant - for that matter).

I am going to fix that later (on my machine), returning to my preferred
fstab format of using PARTUUID whereever appropriate.

Bonne journée
DdB

No fool like an old fool (debian installation probs)

[DdB]

Hello list,

after receiving so much good advice, i finally made up my mind and began
playing through the installation (debian bullseye, current stable) in my
simulation-VM in order to learn about the pitfalls to avoid. After more
than a dozen tries, i am running out of fuel, because i am continuously
running into problems, that i would have preferred to circumvent.

At first, i picked a debian (debian-11.6.0-amd64-DVD-1.iso), as the 11.7
release was not yet out), and installed from there, choosing the Expert
Install. The main reason was, that i was expecting to be invited into
customizing the system picking software to install by hand.

But at first, i ran into a multitude of problems with the partitioning
tool, even though the system was already properly configured (GPT +
EFISYS + OS + Swap + several other partitions booting in UEFI style).

For example, when i allow the use of the swap partition, it gets
reformatted and another PARTUUID is assigned to it, leading to failures
booting other systems from that disk. OTOH Disallowing its use for the
time being leads to install failures due to the 4GB of RAM apparently
not being enough for that VM.

Another problem, similar but distinctly different happens around the OS
partition. Whenever i gave it to the installer, it reformatted it while
changing the PARTUUID, again leading to failures in the boot process of
the other partitions... Until i discovered this workaround: Assign the
partition to ext4 and the mountpoint / but not allowing to delete its
content (as that would induce the reformatting). Instead, after
finishing partitioning, i asked for a shell and from there manually
deleted ONLY THE DATA from the partition(s) used. Then installation
progressed further.

BTW: Apparently, this was the only way to keep the PARTUUID, as deleting
the data by overwriting with zeroes lead to the installer overwriting
the fs structures making another mke2fs necessary.

But the worst of all, was the software selection: Even though i asked
for debconf priority low, i got only a few checkboxes to pick from. And
choosing GNOME did automatically install software i neither need nor want.

But omitting GNOME from the list lead to a system failing to boot with
tons of messages stating the absense of all kind of gnome parts.

So, just in order to make some progress, i had to willingly and
temporarily kill the bootability of other partitions in order to have
the install terminate without failures.

At that point, i decided to try the netinstall, as it is now available
with release 11.7 and i did expect there to be more choices. And when
even that one failed to boot without GNOME, i even did check the
sha512sum to make sure, i was using the proper one. - I did!

Now, i am confused. Many years ago, the expert installer was very
difficult to use and did provide very many choices (and liberties to
spoil the fun). Now, i dont seem to be able at coming to the stage,
where i can seriously test more complicated things (like multi-boot,
zfs, and other things) that make up my current host in its buster version.

It is quite obvious, that i am lacking SOME kind of understanding,
because what i am trying to accomplish should really be quite easy, but
i am only experiencing troubles.

Oh, as a side note: As a nicety of my setup, i can use bash scripts,
that work well from my main OS, or from the simulation VM which can be
used to test functionality before it is rolled out to the host. That is
why, i am relying/insisting on absolutely identical PARTUUIDs inside
that VM.

Now, i am really curious as to how to get to a proper system without
having to go through the baby-steps beginning with debootstrap, and all
the millions of configuration steps, that i did not even bother to list.

I am really interested to understand, in which way i am creating all
those problems for myself. Could it be related to the fact, that i am
accessing the net fom a VPN? - I don't think so.

Any suggestions/questions/hints from the power-users in here?
... would be wildly appreciated ...
DdB

Re: Looking for inspiration/advice/best practices on system upgrade

[DdB]

Am 25.04.2023 um 15:43 schrieb David Wright:
> On Tue 25 Apr 2023 at 09:11:23 (+0200), DdB wrote:
(...)
> 
> The problem lies with the user accounts 101–999 (and releated groups),
> which are system accounts created in a somewhat random manner as
> packages are installed on each system. Those ≤100 are fixed by Debian
> (IIRC in package base-passwd), and those ≥1000 are easily kept
> consistent by good administration practice.
> 
(...)
> Just an observation (I know nothing about ZFS):
> 
> AFAICT there are two occurrences of "permission" in this thread,
> both in your own text, whereas your problem seems to be one of
> ownerships.
> 
> Decades ago, I made the mistake of transferring a file from one
> system to another (probably passwd.client), and ended up with a
> broken email system as the file was now owned by lpadmin or some
> such. But only the ownerships were wrong, not the permission bits.
> 
> I have toyed with the idea of keeping my other PCs in sync with
> a master PC by preemptively creating users/groups as packages
> are installed on the master. This would only be practical with
> fresh installs of a new release.
> 
(...)
> 
> Cheers,
> David.
> 
> 

Hey! Excellent noticing there!
Yes, you are entirely correct, my issues are/were ownership related. And
my wording was largely due to me misunderstanding the meaning of the
word "permission", as in german, that identifies the larger concept
which can be dealt with by many different implementations (more or
less), whereas you identified the implementational detail called
"permission bits", which my vague, imprecise description accidentally
did not exclude.

Thank you anyhow for pointing this out, as it also clarifies my
thinking, and possibly the things, i am going to try to fix the issue.

As i said: Excellent noticing.

Thank you, DdB

Re: Looking for inspiration/advice/best practices on system upgrade

[DdB]

Am 25.04.2023 um 02:18 schrieb David Christensen:
> I have a SOHO network with FreeBSD servers and Debian, Windows, macOS,
> and iOS clients.  The hardware is anywhere from new to 16 years old.
> Where possible, I install a 2.5" SATA 6 Gbps trayless mobile racks in
> the computers and use 2.5" SATA 6 Gbps SSD for system drives.  With FOSS
> OS, I try to install the system such that it will boot and run in
> several computers.
> 
> 
> For each system, I keep notes, console sessions, system configuration
> files, etc., in a version control system (VCS):
> 
> * RCS was my first FOSS VCS.  Learning and using RCS was worthwhile. The
> key advantage of RCS is that it works when the network is down.  The key
> disadvantage is that RCS works on files individually.
> 
> * CVS works on directories of files, has a central repository, and
> allows network access to the repository.  The repository can be accessed
> from the local machine when the network is down.  CVS meets my software
> development and system administration needs.
> 
> * Git is even more powerful, and popular with FOSS projects.
> 
> 
> Configuration management systems, such as Ansible and Puppet, are beyond
> my needs.
> 
> 
> When an OS new major version is released, I typically wait a year or two
> for it to stabilize (or for the previous version to be EOL'd).  I then
> use another computer, insert a zeroed SSD, do a fresh install, configure
> the system (manually), and migrate the data.  Before, during, and after,
> I document my work, check in files, validate data, back up, archive, and
> image.  One advantage of this approach is that you are certain that
> there is no leftover cruft wasting space or waiting to bite you.
> 
> 
> debian-11.6.0-amd64-netinst assigns UID/GID in the range 0 through 999
> for system accounts, plus 1000 for the default user account, plus 65534
> for the "nobody" account.  That leaves 1001 through 65533 for everything
> else.  I made a list of the accounts I add to systems many years ago and
> use those values on all of my systems.  LDAP and related are services
> that centralize this kind of information.
> 
> 
> I would not attempt to "Straightening out" passwd(5) or group(5).  I
> would a make a list of accounts (as above), do the fresh build (as
> above), and translate the data during migration -- e.g. rsync(1) to new
> directory names, rename(1) to new file names, chown(1) to new UID/GID,
> etc..  Doing the migration/translation by hand is tedious, but may be
> feasible if the data is organized by username.  If you can write
> scripts, correct scripts can save time and reduce errors (incorrect
> scripts are another matter).  STFW might turn up suitable power tools,
> such as backup/restore tools with username/groupname/UID/GID translation.
> 
> 
> On machines with correct passwd(5) and group(5) but
> username/groupname/UID/GID that do not match the new account list, I
> would create new accounts per the new account list, move/translate the
> data, and then delete the obsolete accounts.
> 
> 
> Practicing on a VM is a reasonable idea.
> 
> 
> David

Thank you so much for chiming in.

Quite a relief to hear, that it may make sense to avoid hassling with
passwd/group and to just straighten out permissions from the files on
the zfs pool(s) AFTER a fresh install.

Furthermore, i am taking away, that it may be a bit too early for the
final move to bookworm yet (in my case).

But most of all, i am taking the suggestion to use Version control for
the documentation. I am already using git for the scripts, that i wrote,
which doesnt necessarily need a network connection. But to include
system configuration directly (i guess, you mean /etc + ssh-sessions +
individual command history + hand-written jotter/notebooks/logfiles)
could be a sensible move going forward. Up to this point, i did include
only a selected few config files, by hard-linking them from my git
workspace.

What you are doing in terms of standardisation for your machines
corresponds to how i am using VM's, which allows myself to trust some
user/directories/files to exist and be respected. This allows to clone +
reconfigure some of them instead of recreating each one individually.

I am going to pursue my goal along those lines and start playing with
the (Simulation-) VM first. Thank you again for those valuable hints.

And finally: Since i tend to only have minimal configuration on the VM
host, and specialise mostly in the vm workspaces, the reinstall will
most likely not be all that hard.

Looking for inspiration/advice/best practices on system upgrade

[DdB]

Am 24.04.2023 um 23:04 schrieb Timothy M Butterworth:
> https://www.cyberciti.biz/faq/update-upgrade-debian-10-to-debian-11-bullseye/ 
> 
> Usually this is the safest way. But why don't you upgrade sooner to
> bullseye ? Do you have some problematic packages ?

thank you for the link suggested. I will use that (almost like a
playbook) to see, what is going to be ahead ...

I have been upholding the upgrade for several reasons, some of which are
not coming from the problem field. But i just checked: the packages i am
actively holding back consist of:

- ZFS
- virtualbox
- firefox
- thunderbird

Obviously, upgrading any of those would stop my current workflow from
being functional. That let me stick with the saying, to "Never change a
running system." ;-)

As i see it, there are plenty of things to do now, beginning with
another attempt at straightening out the file permission issue, followed
by attempting the upgrade in a vm first.

Thanks again for the linked article, which will come in handily.
Cheers, DdB

Looking for inspiration/advice/best practices on system upgrade

[DdB]

Thank you for providing your take on this.

Am 24.04.2023 um 19:46 schrieb Dan Ritter:
> Upgrade buster to bullseye, reboot, upgrade to bookworm.
> 
> Solve the final set of problems, not all the intermediates which
> may have been fixed.
interesting consideration there. :-)
> (...) If you have owner/group problems now, solve them before any of the
> upgrades.
That is, what i was thinking too. Unfortunately, i am afraid of this
step. That is why i had been builing a kind of playground first.
> 
> -dsr-
> 
> 

Looking for inspiration/advice/best practices on system upgrade

[DdB]

Hi list,

while still on debian buster (old-old-stable soon), i am approaching the
point, where i will be ready to upgrade. I do have backup(s) from
different points in time, and can carry out restore to a VM, that is
almost identical to my main system, that i can use as a playground for
playing through the whole process.
Also i can automatically set up a running duplicate of my system on bare
metal in another partition, in order to allow investigation of the
process without risking my current working environment.

But i could not yet decide upon the right way to progress, 3
alternatives coming to mind:
1. Upgrade from current configuration using upgrade path tools (apt) and
plan only one step (going to bullseye) at a time, eventually having to
upgrade a second time later.
2. Install a fresh bullseye and try to manually make it as similar as
can be to my current workhorse
3. Skip bullseye altogether and jump to bookworm right away.

>From my past experiences, i recall one problem, that i am still
suffering from: the file group and user owners, that reside on a
different disk set (on a ZFS pool, to be more precise). In order to
alleviate the problem, i did at one point fetch an older /etc/group and
passwd file from a backup and created a mess. I'd love to straigthen
this out now.
At the time, i did investigate (configuration management) tools, but
found the effort was much too high for my liking, as i am the only real
user on my system and also retired, making such a professional solution
going way too far.

I can be patient, but i would like to find the most reasonable path to
get a more up-to-date system soon. Would anyone with more experience,
than i have, give me their point-of-view/recommendations as
food-for-thought, please?

Thank you in advance
DdB

Re: efi problem

[DdB]

Am 22.04.2023 um 21:01 schrieb mick.crane:
> Any idea what the correct manifestation  would  be to get the PC to boot
> the debian installer CD?

Without checking your CPU and general hardware, it would be dangerous to
speculate as of what you need. But to give at least one hint to you,
i'll copy from MY grub.d/40_custom file:

> menuentry "Test debian-live " {
> # Hier sind die Kernel und Initrd-Versionen (erstmal) Hardcoded!
>   insmod part_gpt
>   insmod ext2
>   search --no-floppy --fs-uuid --set=root 1a4db22f-65e4-476a-b7e3-8f9f91cf0b8b
>   set IsoDatei="/mnt/Debian-bulls-gnome.iso"
>   set cmdline="noprompt noeject toram noswap locales=de_DE.UTF-8 
> keyboard-layouts=de timezone=Europe/Berlin --"
>   loopback loop \$IsoDatei
>   linux (loop)/live/vmlinuz-5.10.0-20-amd64 boot=live config 
> findiso=\$IsoDatei \$cmdline
>   initrd (loop)/live/initrd.img-5.10.0-20-amd64
> }

but please note, that this includes hardcoded details not applying in
your case.

Re: efi problem

[DdB]

Am 22.04.2023 um 19:30 schrieb mick.crane:
> I suspect the GPU is suspect because there are small blocks of pixels
> appearing where they aren't wanted.
> I'd like to re-install the OS as I've got copies of everything I think.
> Just to see if it's maybe a driver issue.
> Thing is I can't get the f12 options screen of the PC to let me boot
> from a CD in this EFI mode what's grub done.
> How can I boot from CD in this EFI mode?
> 
> mick
> 
> 

My neighbor has an old laptop without a CD drive. But there is a linux
running (ubuntu). So what i did is this:
i did download (and check) an ISO-file to boot from (debian- or ubuntu-
live versions) and googled for the appropriate stanza for grub to handle
booting from that ISO file. Then i did modify the /etc/grub.d/41_custom
file, to make sure, grub will include that stanza on update-grub
automatically.
But if you only want to boot it only once, you could - on boot, while
seeing the grub menu, enter "c" to open a command line and enter the
necessary commands by hand. (I liked to learn, what grub has to offer
for cases of emergency. Can come in handily, if you - or a friend - gets
stuck on boot.)

BTW: IIRC, grub is able to start an efi-shell, if needed, to force boot
options that way. But my neighbor did not have uefi. Whereas I do, and -
although i am lazy and prefer automation, i had to boot up manually via
UEFI several times.

What i am saying is: it is possible, but not exactly trivial. Learning
is involved and desirable.

Have fun
DdB

Re: gitification (was Re: /etc/fstab question (problem)?

[DdB]

Am 22.04.2023 um 08:33 schrieb Max Nikulin:
> On 21/04/2023 00:43, songbird wrote:
>> Max Nikulin wrote:
>>> On 20/04/2023 19:10, songbird wrote:
>>>>     one of the worst design decisions i've come across in
>>>> the modern era was the lack of git respecting file metadata.
>>
>> i know what all you've written below but
>> it does not apply to what i want or how i use those tools
>> and i consider git broken that it caters to broken tools
>> and intentionally then has to screw up information which i
>> consider both useful and critical to how i do things.
> 
> Then I have no idea what you were trying to achieve by your original
> message.
> 
> It is perfectly valid to warn people that git is not an appropriate tool
> when file attributes audit is involved. I can understand if somebody is
> pushing you toward git. At the same time I see nothing bad in tracking
> config files in git.
> 
> If you are looking for a backup tool that keeps metadata then it is
> better to ask it explicitly to to get suggestions like rdiff-backup.
> 
> Ignorance may be an excuse, but you said it is not the case. For me it
> is too much to blame developers with harsh statements concerning design
> decisions just because a tool was created for different tasks.
> 
> Git appeared as a tool for linux kernel, a project that relies on make.
> Frequent incremental rebuilds are must have for developers. Git has some
> weak sides, but there is no point to attack its features. Git is a great
> step forward in comparison to CVS and SVN. Experiments with version
> control systems and build tools have not seized, likely we will see
> better ones.
> 
> Precise tracking of file attributes can cause troubles for VCS. Various
> file systems have different set of attributes, incompatible time
> precision. There is no point to track UIDs at all.
> 
> I admit, for reproducible builds that include unprocessed files from
> repository, git behavior is not perfect.
> 
> Do not confuse a conscious design choice (even if it is a trade-off) and
> wrong selection of a tool that is inappropriate for specific purpose.
> 
>>> Some build systems make decisions based on file hashes, not their
>>> modification times. It may require a daemon watching file changes to
>>> avoid recalculation of all hashes on each build. So such approach is a
>>> kind of trade-off.
>>
>>    not a choice i agree with and so i have to work around
>> it for my purposes.
> 
> File hash approach is for developers relying on incremental rebuilds and
> caching of build results. It is a way to avoid changing of mtime on
> checkout.
> 
> P.S. Old version of git FAQ explains taken mtime approach in the same
> way. Build tools relies of modification time comparison:
> https://archive.kernel.org/oldwiki/git.wiki.kernel.org/index.php/Git_FAQ.html#Why_isn.27t_Git_preserving_modification_time_on_files.3F
> 
> (New one is rather brief https://git-scm.com/docs/gitfaq)
> 
> 
Thank you, Mr. Nikulin, for writing this long message. I am totally with
you in many regards, and have been following this thread for some time,
mostly wondering and finding myself unable to put into words, what was
bugging me with it. It is quite a challenge to see it the way you (and
i) do, while still being friendly and helpful to the OP. but you managed
that, and i admire the way, you are doing it. Very nice and fine
distinctions like (i'll give one example only: )

>> Then I have no idea what you were trying to achieve by your original
>> message.
>> 
>> It is perfectly valid to warn people that git is not an appropriate tool
>> when file attributes audit is involved. I can understand if somebody is
>> pushing you toward git. At the same time I see nothing bad in tracking
>> config files in git.
>> 
>> If you are looking for a backup tool that keeps metadata then it is
>> better to ask it explicitly to to get suggestions like rdiff-backup.

Just saying: I do very much appreciate your post. Thank you for that.
DdB

Re: /etc/fstab question (problem)?

[DdB]

You got your plan mapped out. and i agree, except for one little detail:
see below. -

Am 19.04.2023 um 22:06 schrieb Default User:
>> I think, it is the case when reboot is safer. Open file descriptors
>> remain on the original partition. However I do not expect that single
>> user mode or booting from live image is required. Just restore
>> original
>> /etc/fstab and reboot.
>>
>> Perhaps update-initramfs is necessary after restoring of /etc/fstab
>> in
>> any chosen approach.
>>
>>
>
>
> Well, now I am totally confused.
>
> I had hoped for, and really expected, an easy, obvious, intuitive
> solution.  But I guess that may be a distant memory of the good old
> days, before [insert string of four-letter words here] like dbus,
> systemd, and Gnome 3. And when partitions were named /dev/hda5, not
> 6a105a72-f5d5-441b-b926-1e405151ee84.
>
> Sigh.
>
> Anyway, here is where I am at:
>
> I have two Clonezilla backups.
> 1) a full disk backup.
> 2) a "partitions" backup.
> So, if things really go bad, I can theoretically revert to the setup as
> of 2023-04-18, when this thread was started.
>
> I also have a backup of the current /tmp directory (from under the /
> directory).
> And I have a backup of the old tmp partition.
>
> Both of these tmp backups were made using a Debian Stable 11.6
> Live/install usb thumb drive, as root user.
>
> All of these backups are on an external usb hdd.
>
> Here is what was in the (root) tmp directory:
>
> _root_partition/tmp
> total 32K
> 88473604 drwxr-xr-t 8 [user] [user] 4.0K Apr 19 14:18 ./
> 88473602 drwxr-xr-x 3 [user] [user] 4.0K Apr 19 14:18 ../
> 88473608 drwxr-xr-t 2 [user] [user] 4.0K Apr 19 14:18 .font-unix/
> 88473606 drwxr-xr-t 2 [user] [user] 4.0K Apr 19 14:18 .ICE-unix/
> 88473609 drwxr-xr-t 2 [user] [user] 4.0K Apr 19 14:18 .Test-unix/
> 88473610 drwx-- 2 [user] [user] 4.0K Apr 19 14:18 tracker-extract-
> files.116/
> 88473605 drwxr-xr-t 2 [user] [user] 4.0K Apr 19 14:18 .X11-unix/
> 88473607 drwxr-xr-t 2 [user] [user] 4.0K Apr 19 14:18 .XIM-unix/
>
> And here is what was in the old tmp partition:
>
> total 48K
> 88473611 drwxr-xr-t 10 rootroot4.0K Apr 19 14:20 ./
> 88473603 drwxr-xr-x  3 [user] [user] 4.0K Apr 19 14:20 ../
> 88473618 drwxr-xr-t  2 rootroot4.0K Apr 19 14:20 .font-unix/
> 88473615 drwxr-xr-t  2 rootroot4.0K Apr 19 14:20 .ICE-unix/
> 88473620 drwx--  2 rootroot4.0K Apr 19 14:20 lost+found/
> 88473619 drwxr-xr-t  2 rootroot4.0K Apr 19 14:20 .Test-unix/
> 88473624 drwx--  2 rootroot4.0K Apr 19 14:20 tracker-
> extract-files.1000/
> 88473623 drwx--  2 rootroot4.0K Apr 19 14:20 tracker-
> extract-files.116/
> 88473621 -r--r--r--  1 rootroot  11 Apr 19 14:20 .X1024-lock
> 88473622 -r--r--r--  1 rootroot  11 Apr 19 14:20 .X1025-lock
> 88473612 drwxr-xr-t  2 rootroot4.0K Apr 19 14:20 .X11-unix/
> 88473617 drwxr-xr-t  2 rootroot4.0K Apr 19 14:20 .XIM-unix/
>
> As far as I can tell, there is nothing crucial in either tmp backup.
>
> BTW, I know nothing about bind or mount --bind. I looked them up
> briefly, and decided that they are too difficult and maybe dangerous to
> try to learn and use under the current circumstances.
>
> So here is what I am thinking of doing:
>
> While running from within the Debian Stable 11.6 Live/install usb thumb
> drive, as root user:
>
> 1) On the computer's internal ssd, delete the /tmp directory and its
> contents.
Do NOT delete the directory itself, only its content, as it will be used
as the mountpoint for your /tmp drive.

>
> 2) On the computer's internal ssd, delete the contents of the old tmp
> partition, but not the partition itself.
>
> 3) On the computer's internal ssd, replace /etc/fstab with
> /etc/fstab.original, renaming it /etc/fstab. I have already made a copy
> of the current /etc/fstab as /etc/fstab.as-of-2023-04-19.
>
> The UUIDs of all partitions on computer's internal ssd seem to be the
> same as in /etc/fstab.original.
>
> (Note: in /etc/fstab.original, it states "Please run 'systemctl daemon-
> reload' after making changes here." Since I am doing all this from a
> live usb, I do not think that applies, so I would skip that.)
>
> Then I would shut down, remove the usb thumb drive, and boot into the
> Debian system on the computer's internal ssd.
>
> I hope that from then on, the system would mount the old tmp partition
> on the computer's internal ssd as /tmp, re-populating it automatically,
> and use it as such from then on.
>
> Does that seem reasonable?
>
> Or am I missing something, obvious or not.

Please report your success, will you?

Re: tmp on tmpfs

[DdB]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Am 19.04.2023 um 08:34 schrieb to...@tuxteam.de:
> On Wed, Apr 19, 2023 at 01:15:01PM +0700, Max Nikulin wrote:
>> On 19/04/2023 11:30, to...@tuxteam.de wrote:

> Of course, I wouldn't propose to set it as a default for a distro.
> 
> Cheers
> 

Neither would i.

my 2 cents:
I did put /tmp on tmpfs some time ago to minimize SSD churn without
really understanding the consequences. Later, i ran into several
problems, which turned out to be caused by /tmp running out of space.
After some investigation, i found the sweet spot, where i did not need
to remount /tmp with bigger size any longer, and all of my task were
happy. (And i was happy, that i could provide so much RAM temporarily,
tmpfs doesnt eat up more than what is actually required.)

For my application, that sweet spot turned out to be at size=45G. This
config is still in my fstab and the system behaves fine since. Just
got to confess, that i am still on debian buster (=old-old-stable very
soon).

just saying
DdB
-BEGIN PGP SIGNATURE-

iQIzBAEBCAAdFiEEumgd33HMGU/Wk4ZRe3aiXLdoWD0FAmQ/lFoACgkQe3aiXLdo
WD0Y3g/9Gb4bFB+JkO0OUrKLKS9a6jEjMaSYmySh6qw4LB6+O+Zsodnkhq79me36
AuaTE1imt8BHqDhBtZyf26/iCfszdpzhVH/Ayl/AbO7axZMhMjXFqNH49gkr9NLI
BNVAP1SQ4OmE43W2gscqIUzN8Hcw+fbHpFbmzcUL3zluR7Bzi84n4lqKrqwxcQUO
0g64F0xjyq0h2MEqmnJN96Lu2D/2ES7JI4XO2YNqpotagfusF9opycCC+oP+xKwm
mPlTAiwfMjRpNWMWSRl7lXAjoah4+HqVyGoNkQOYapw56CXUWYQkwk74bkL9RTmQ
xl1t768LVS/O22OBXidqUUEausRkOe7yhWLL/Rv46ZkN1xn1IVsNomcq9abeUkBg
OTSKcvNiBQDEG58sSf2iSnfyQ9O+1dQ45D2dDHOqxg4PS1B0GIijetucpg2izLSg
6RLuk+Fn92xxKlcPV2791QQpJBh0dAwNdXuRmSVQ9VwxF7Rj2tOE9oK5lC/QbXnw
dF7bNmHCEJ82GRtSVe6o+ujj8Hmbi+nyg1ci7wxy4a81seOb4pM9oVpaS5GA5x1l
Yqj2AWXBBlx4N3NNTZb/X1KAaJrob/jNo76LeRyCyiygyOlrQZixhZMJvXNh6SXM
PzP1J0Sz/OneYI1GT/uUoYe6VsqVpgSUfrRJKJh3eX7O7DQkYb4=
=5YWO
-END PGP SIGNATURE-

/etc/fstab question (problem)?

[DdB]

Am 18.04.2023 um 16:59 schrieb Default User:
> Hey, I have a strange situation! 
Wow! Am I misunderstanding something?
You seem to be well in control of your system, thus i am i bit surprised
as of the simplicity of your question.
If it was me, i would just find out, where exactly tmp resides now, then
shutdown and boot from an ISO in order to have a "cold" system on disk.
Then mount both tmp places to check, if there is anything worth
keeping/consolidating. Otherwise clear the mountpoint and uncomment the
/tmp line from your fstab. Rebooting should then just mount it as you want.

Questions, i could not answer:
What if you wanted to make the change without downtime?
What if you want to make use of systemd services to mount /tmp?
* I assume, that one gets created automatically, if a tmpfs is used for
/tmp, but i dont know, when exactly, it would run.
Is there any relevance? I guess not.




-- 

Liebe ist ...
Datakanja

Re: Which Diff tool could I use for visually comparing two text files where Word Wrap is possible?

[DdB]

Am 01.04.2023 um 05:37 schrieb Susmita/Rajib:
> Dear Mr. DdB:
> I fondly remember my interaction with you some time during May 2022.
> Perhaps you have overlooked that I needed text wrapping for diff. I
> have checked the synaptic screenshot for meld, have installed and
> tried it. But it too suffers from the lack of text wrapping function.
> For huge text files it is thus problematic. Is a text wrap option
> available? Am I missing something?

Thank you, Sir, for your exquisite feedback. From what i read, your
problem found a resolution suiting your needs, which i refrain from
reading as an aprils fool prank. ;-)

In fact, unfortunately, i did not understand the necessity to wrap the
output, as i am happily using the synchronised scrollbar (inside meld)
in such cases, but ofc, that may not fit your use case. Sorry for that.


Happy diffing
DdB

Re: Which Diff tool could I use for visually comparing two text files where Word Wrap is possible?

[DdB]

Am 31.03.2023 um 18:13 schrieb Susmita/Rajib:
> My dear illustrious leaders and senior debian-user list-members,
> 
> I tried diffuse, but it appears to me that it suffers from a
> limitation so far as my need is concerned. It compares files by lines
> and line numbers, so I can't use word-wrap to have the differences
> between two files within the program window without venturing out to
> the right within the two file windows.
> 
> Is there a way to Word Wrap? Am I making a mistake here? Which program
> would be the best suited for my work for comparing text files?
> 
> There is a package called diffoscope but it has to install a long list
> of dependent packages in my present Debian system installed from
> "Official Debian GNU/Linux Live 11.6.0 lxde 2022-12-17T11:46"
> 
> Eagerly awaiting your advice.
> 
> Best wishes,
> Rajib B
> Etc.
> 
> 
I may be misunderstanding your needs:
In many cases, i replace diff with meld, an interactive diff UI

Re: youtube-dl → yt-dlp, was Re: OT: Detecting ISP throttling (was: Re: Potentially OT. Videos lagging & buffering in any browser but Google Chrome.)

[DdB]

Am 31.03.2023 um 00:28 schrieb l0f...@tuta.io:
> How do you get that URL? Via your browser resource/code inspector?

Hi, i had been asking not having to answer that question, because i
myself do not really understand, how it works.

I stumbled across an explanation while skimming through open issues of
yt-dlp on github. But unfortunately i did not save the link. => Happy
searching ;-)

Re: youtube-dl → yt-dlp, was Re: OT: Detecting ISP throttling (was: Re: Potentially OT. Videos lagging & buffering in any browser but Google Chrome.)

[DdB]

Am 30.03.2023 um 20:20 schrieb Bret Busby:
> On 31/3/23 02:08, David Wright wrote:
>> On Fri 31 Mar 2023 at 01:41:04 (+0800), Bret Busby wrote:
>>> On 31/3/23 00:40, David Wright wrote:
 On Thu 30 Mar 2023 at 19:31:21 (+0800), Bret Busby wrote:

> I had previously been able to use youtube-dl, to download videos from
> youtube, but, it no longer works with youtube.

 AIUI youtube-dl is now obsolete, and its new spelling is yt-dlp.

 You can download it from bullseye-backports. If you're a backports
 user, just install it and check out the CLI options you normally
 use, in case they've been tweaked, with:

     $ yt-dlp -help | less

> And, I had been able to download videos from some streaming hosts,
> but, find that I cannot download videos from other streaming hosts.
>
> By download, I mean saving to my computer - so that I can get
> continuity, and, replay the streamed videos at slower speeds, and,
> manipulate the sound, so that I (having a hearing disability that
> affects my ability to hear what is spoken) can better hear what is
> being said, which is important for educational videos, such as at
> Rootstech 2023, where I cannot download significant videos.

 The yt-dlp dependencies are all unversioned, so rather than put
 backports into my sources.list, I just checked that I had them all,
 downloaded yt-dlp….deb from the Packages page, and installed it with
 the    apt-get install /full-path-to/yt-dlp….deb    syntax.
 (The backports youtube-dl package used to get very long in the tooth
 at times, so I've done this frequently in the past.) Whenever
 downloading fails, I check out whether there's a more up-to-date
 version with the   yt-dlp -U   option.

 Apparently there's a bug in yt-dlp at the moment, but I don't think
 either of us uses mpv /with/ the downloader, but only /after/
 downloading has completed.

>>> I had downloaded and installed yt-dlp, but, that did not work, either.
>>
>> Writing "did not work" just doesn't cut it on this list:
>> we need some specifics. Here's an example I ran this
>> morning on a reference given by David Christensen:
>>
>> $ gy bKzonnwoR2I
>> /usr/bin/yt-dlp http://www.youtube.com/watch?v=bKzonnwoR2I
>> [youtube] Extracting URL: http://www.youtube.com/watch?v=bKzonnwoR2I
>> [youtube] bKzonnwoR2I: Downloading webpage
>> [youtube] bKzonnwoR2I: Downloading android player API JSON
>> [info] bKzonnwoR2I: Downloading 1 format(s): 248+251
>> [download] Destination: Unix Pipeline (Brian Kernighan) -
>> Computerphile [bKzonnwoR2I].f248.webm
>> [download] 100% of   50.64MiB in 00:00:05 at 8.45MiB/s
>> [download] Destination: Unix Pipeline (Brian Kernighan) -
>> Computerphile [bKzonnwoR2I].f251.webm
>> [download] 100% of    3.85MiB in 00:00:00 at 6.28MiB/s
>> [Merger] Merging formats into "Unix Pipeline (Brian Kernighan) -
>> Computerphile [bKzonnwoR2I].webm"
>> Remember to remove any unmerged files as appropriate.
>> $
>>
>> Whether you can download that would determine where you go from here.
>>
>> Cheers,
>> David.
>>
> Fri Mar 31 02:18:31 bret@bret-Precision-Tower-5810:~$yt-dlp
> https://www.familysearch.org/rootstech/session/expanding-your-family-tree-with-sideview-and-more-innovations-from-ancestrydna?lang=eng
> 
> [generic]
> expanding-your-family-tree-with-sideview-and-more-innovations-from-ancestrydna?lang=eng:
> Requesting header
> WARNING: [generic] Falling back on generic information extractor.
> [generic]
> expanding-your-family-tree-with-sideview-and-more-innovations-from-ancestrydna?lang=eng:
> Downloading webpage
> [generic]
> expanding-your-family-tree-with-sideview-and-more-innovations-from-ancestrydna?lang=eng:
> Extracting information
> ERROR: Unsupported URL:
> https://www.familysearch.org/rootstech/session/expanding-your-family-tree-with-sideview-and-more-innovations-from-ancestrydna?lang=eng
> 
> Fri Mar 31 02:19:05 bret@bret-Precision-Tower-5810:~$
> 
> 
> ..
> Bret Busby
> Armadale
> West Australia
> (UTC+0800)
> ..
> 
> 


I was successful at downloading the video with:
> yt-dlp --verbose -k --ignore-config -c 
> https://manifest.prod.boltdns.net/manifest/v1/hls/v4/clear/1241706627001/83ddeca4-2e3a-4149-840f-0ca907c2cb59/10s/master.m3u8?fastly_token=NjQyNjYyOWZfOTIzNjUyM2MwN2JlZTI3NjdhNDcwMjM4ZjhmNmY1MmRiZTQxMzM5ZTllOTYyM2E3YTkxZWNhOGQxYmY1YTU0OA%3D%3D

after finishing download, i found the file
> master-master.mp4

Dont ask me, how i did this, because i dont know, just happened to work.

Re: should CLI have a nice UI today?

[DdB]

Am 24.03.2023 um 12:32 schrieb cor...@free.fr:
> Hello,
> 
> Should CLI (command line interface) have a nice UI library?
> today web dev has so many libraries that make web pages with
> rich/colorful interactive views.
> But CLI is still in dull mode. That should be improved in these days.
> for example, run "df -h" we got the statistics with plain text. But web
> statistics for cloud storage (GCP,AWS etc) are chart like, which give
> people more intuitive feeling.
> 
> Thanks
> Corey H.
Well, how do you call messages, that provoke troll replies?

In other words: Just the way, this was written, let me hide away.
But since so many people seem to take this seriously, i got to say:
FWIW: If i was searching for GUI niceties, i would take a look at
Windows. Because it is easier to use for simple tasks, and requires less
understanding - at first.

For me, it is exactly the other way round: It is exactly because of the
many things, i could not do in Windows, that i came to linux and after
some years of "playing" with it, i would never want to go back... One of
the reasons being the power and flexibility of the command line.

Sometimes, doing something at the command prompt for the first time, may
be daunting, but then the history is my friend and helps to collect the
raw steps and to generate a script for future use, which is empowering
even more.

And over time, it seems to me as if my thinking changes into searching
for the most generic way to do things instead of operating on single
entities. Just being able to compose a specific "find ... -print0"
command and pipe it into xargs -0 (or parallel) makes so many tasks
straight forward and complete in themselves, that i have a clear
understanding about the difficulties/impossibility to create a GUI with
identical powers.

Even if i am assisting some neighbor at using their linux computer, i
find myself losing my patience at times and just opening up a terminal
window to execute some job faster than pointing and clicking could provide.

I LOVE my command line!
just my 2 cents
DdB

artifiial intelligence (was: Re: question about net address)

[DdB]

Wow!
Great hint there!
I just tested it in a couple of areas and found it to be quite useful,
by far more up-to-date and i did enjoy the experience.
Thank you for sharing it.

Am 19.03.2023 um 12:01 schrieb Yassine Chaouche:
> In contrast,
> a tool like perplexity.ai is an answer-questionning tool.
> Is is a search engine.
> It cites its sources,
> so you can check for yourself whether it's talking crap,
> or if it's backed by facts.

Re: Whole-disk RAID and GPT/UEFI

[DdB]

Am 22.02.2023 um 17:07 schrieb Nicolas George:
> Unfortunately, that puts the partition table
> and EFI partition outside the RAID: if you have to add/replace a disk,
> you need to partition and reinstall GRUB, that makes a few more
> manipulations on top of syncing the RAID.

Yes, i get it. AFAIK, you cannot really avoid that, because a disk
replacement comes with new disk UUID's anyway and slightly different
sizes mostly. Maybe the best thing to do is to document, and maybe
script that case with your knowledge from today. that way, you can go to
sleeo until the case happens, and then read the doc (in your script) and
execute it. That is what i do (because i keep forgetting things faster
and faster ;-))

And usually, the script gets better over time after several runs ...

Sorry, that i have no better suggestion.
Good luck
DdB

Re: Whole-disk RAID and GPT/UEFI

[DdB]

Am 22.02.2023 um 16:30 schrieb Nicolas George:
> Is there something I have missed?

I cannot say yes or no. But, of the top of my head, i know, that gdisk
does support a strange thing called hybrid between MBR and GPT, where a
clean GPT formatted drive contains (instead of a protective MBR) an MBR
with pointers to up to four partitons, in order to make a system
bootable without UEFI. I understand, that your use case is somewhat
different. But maybe you can use the idea anyway. The hybrid formatting
has some restrictions, like you should not have standard MBR tools mess
this configuration up (not use them at all) and unfortunately that
applies to standard GPT tools as well, but the dual bootability can
solve some problems.

The only issue, i have had a look at, was the problem to have a raid,
that is bootable no matter which one of the drives initially fails, a
problem, that can be solved by having at least 2 grub installs (on the
two mirrors).

Certainly, there is a solution to your problem, even if might not
exactly fulfill your premises.

Re: Test ECC memory

[DdB]

Am 20.02.2023 um 23:48 schrieb krys...@ibse.cz:
> I am sorry, it was little missleading - not that they can not support them, 
> but there is no official document that would state so. The only official 
> specsheet I saw that explicitely mentions ECC support is this one: 
> https://www.amd.com/en/products/specifications/embedded

Lucky me, i just looked up my hardware (Dual CPU on server MB):

> https://versus.com/en/amd-epyc-7282
> Supports ECC memory

I conclude: at least the hw *should* be able to ... if i manage to set
it up properly ;-)

Re: Test ECC memory

[DdB]

Am 20.02.2023 um 18:42 schrieb krys...@ibse.cz:
> Dear Debian community,
> we recently started using AMD Ryzen CPUs, ASRock Rack motherboards and 
> Kingston unbuffered ECC DIMMs for our small bussiness servers. All the 
> servers are running on ZFS for which ECC memory is recommended. So I naively 
> tried to test it actually works. I read EVERY disscussion on EVERY forum I 
> was able to find (and there is a lot of them, believe me), but I did not find 
> a satisfying answer. According to the legendary tweet from AMD (for which is 
> link in every discussion), the Ryzen CPUs should support ECC memory, but it 
> is not tested feature since they are consumer CPUs. Funny thing is, that 
> according to their spec sheets even EPYC class CPUs do not support them (only 
> CPUs with stated ECC support I found are Ryzen Embedded ones - for example 
> the V1605B in UDOO Bolt). Nevertheless system reports it works -  dmidecode, 
> lshw, kernel loads driver and EDAC MC is present in 
> /sys/devices/system/edac/mc, even memtest86+ v6.0 and above reports ECC 
> memory. In forum discussions Intel guys are saying that correctable ECC 
> errors are relatively common - stated counts vary, but I got the impression 
> that at least one in a week should appear. And our virtual hypervisor running 
> over half a year with more than 80% memory utilization has not a single one, 
> niether in sysfs nor in EUFI event log. I understand that the errror count 
> rises with height above mean sea level due to solar radiation and we are in 
> 246m altitude, but at least one error would be nice.
> The only thing I had success with was memory overclocking - I lowered timing 
> as low as possible for system to POST and when Debian was running, it 
> reported corectable errors from different memory regions (13 during 30 
> minutes). Rising memory frequency did not work. But all this was done on Asus 
> motherboard, with same memory and CPU however. When I change any memory 
> related setting on ASRock Rack motherboard, it will not POST.
> In kernel documentation is described that Intel CPUs have ability to inject 
> errors for driver testing but I did not find anything like it for AMD. Does 
> anyone know any way to test that ECC works without breaking the system 
> before? Thank you for your answers.
> 
> PS: Some commercial memtests should allegedly be able to inject ECC errors 
> (for example the one from passmark), have anyone tried those?
> 
> Best regards,
> Kryštof
> 
just saying:

i am on the same ship ... (ZFS + AMD (2 EPYCs in my case) + ECC + not
verified behavior)
Previously, i was using Intel, where i got edac to work somehow, and it
even caught some correctable errors. But since i learned, that edac went
out of business and dmidecode shall be used to get info from hardware
interrupt caused by ECC memory, i have never seen one, and as a less
than experienced debian user, i got stuck on other problems, thus forgot
to pursue this issue somehow. Now, i am very much interested in the
hints/replies you may get, in order to finally test/straighten my
infrastructure.

Did you really read, that epycs cannot support ECC?
At least i can say, that my pools did not report any faults (which ofc
would be several layers above ecc) either in 3 years, which did help in
falling asleep. ;-)

Anyone experiencing some wind in his sails while sailing along similar
paths?

... Would be welcome ...
DdB

Re: Partitioning an SSD?

[DdB]

Am 16.02.2023 um 13:30 schrieb DdB:
> Unfortunately, the
> data set related to this, i could gather personally is not large
> enough to be telling.

https://www.servethehome.com/ssd-alignment-quickly-benchmark-ssd/

Re: Partitioning an SSD?

[DdB]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Am 16.02.2023 um 13:00 schrieb The Wanderer:
> This being the very first time I can remember having encountered
> even the suggestion that there's no need to be concerned about
> erase-block sizes when dealing with SSDs et cetera, I hope it's
> understandable that I'd want to get things clarified.

I second that. To me, it sounds like 2 different mind sets (a.k.a.
belief systems) suggesting different behaviors. Unfortunately, the
data set related to this, i could gather personally is not large
enough to be telling. And my belief was - just as yours - supported by
my inner pictures of what i think is going on inside the SSD. I did
not conduct performance tests with and without alignment myself.

But i recall reading about them, from users/admins of large zfs sites
(on zfs.discuss mailing list) and also from a hardware store (was it
backblaze, or something?), and later never questionned my belief.

But who am i to speak about this? - I just gave my honest best
opinion, that has been guiding my own efforts since years. But i dont
care, if some people prefer disregarding such advice. Everyone does,
what he thinks is best for them.

-BEGIN PGP SIGNATURE-

iQIzBAEBCAAdFiEEumgd33HMGU/Wk4ZRe3aiXLdoWD0FAmPuIdQACgkQe3aiXLdo
WD3X0Q/+MoTuqcbwovYXBiw4CanXDMiSAy71krXPrMHdADBWHP4Rl7rPHSGs3mRf
r2WM0AdmUBDVbK+EsSqbw33BdW3HMDy0tZ7z9GTwOff9wD7RJbsoSMwtxI8yrgaX
g/7N4YqsocKAzainWPXqFJTbZzTI3M2PjBoyuqBJ1JF+LSuLzUi6DK5Z0uySG6HD
KMiGswVzqYpOtR8XYPl9Q7x4/3Zt3buCRRtaISkW+0MwrjRbDJdy/h7hKV7l27C7
6f1clZ8Glqv43/ZCCympORdrJT4ddxGNipd8/gEAC/DlgDfdt9wSTOy7eiQcnGIZ
e4HrvteSqVX731ojjcF9pt1ujIqDzW28FxT1KnUozc/sc4JbrrbHzaDzH1PdqWEi
TcXGcDxVzY2iQ9Q5/217F2UDWLE+5mWAgOhvmYHxCv5tImSfTaKPYKec5NvlhfOo
UaYHEEGAVFpThGPlJXF/NbFQFo+Oyp/E6bYMSGTl2/nIU7K2s5QidF5EG9IY3Tcj
DnMigUSWgpnWAbd8Z4muluCuEkPqJcYYTx/2YS13u2VgNPm8gbUS6CM+CIu607qv
Rkl7wCa86hxegoslk+2klGDm3CGBku89ii5M6iIwPLTZjAPY3W7e6Olz3f9avogW
l6gWv+wIHGLwE1o3ncq5Au5EJA037CnU3fnlbzUPOxmCq2qqA3E=
=ELkf
-END PGP SIGNATURE-

Re: Partitioning an SSD?

[DdB]

Am 16.02.2023 um 09:31 schrieb Felix Miata:
> None of the 25 or so SSDs/NVMEs I have have 4k sectors. e.g.

Wow, they must be rather old, then. ;-)

I know, i am not the only one ...
https://serverfault.com/questions/1113068/how-to-find-page-size-of-my-ssd

Re: Partitioning an SSD?

[DdB]

Am 16.02.2023 um 08:22 schrieb Felix Miata:
> What physical boundaries do SSDs have to report? All I know about that are 
> exposed
> are sector size and sector count. I have yet to find one where 
> logical/physical
> were not 512B/512B.

That is what i meant: nowadays SSD's at least are AF Advanced Format =
4KB sector size), but even much more than that. Do not trust the values
reported but check documentation properly. 512B is only virtual
(sometimes called 512e - for emulated).

Re: Partitioning an SSD?

[DdB]

Am 16.02.2023 um 07:44 schrieb DdB:
> I do use (NVMe-) SSD, and i did partition it.
> I did it to make sure, pages/partitions start on PHYSICAL boundaries,
> not the logical ones reported to satisfy Windooze. Not every model
> reports correct hardware parameters to the OS.
> 
> What i would recommend, is to use GPT/UEFI if possible, to avoid future
> hassle and limitations (gdisk being your friend) and set alignment
> manually to the physical page size. That is going to avoid eventual
> read/write cycles to accomodate emulating 512e virtual sectors and also
> wasting space on the drive.
> 
> Apart from that, i do not think, you would be limited to do anything you
> want with the installation. My personal preference has always been to
> have room for 3 OS partitions (20GB max each), one for real, one for
> fast cloning/backup and one for wild experimentation. Any serious backup
> belonging onto outside media/computer anyway.
> 
> just my 2 cents
> 
> But everybody has different needs and habits, so do, whatever suits you.
> 
> 

I forgot to mention, that i prefer using real RAM to swapping,
especially on SSD's, as those fast use patitions (swap space and the
like) eat those up literally and wear them out quickly. Enough RAM makes
any system fast, so for me swap space doesnt make sense.

Re: Partitioning an SSD?

[DdB]

Am 15.02.2023 um 23:58 schrieb PMA:
> Dear Debian,
> 
> I'm preparing to install Debian 11.5.0 on a new computer.
> Its drives are SSDs, not the HDDs I've been accustomed
> to and have always fastidiously *partitioned*.
> 
> With my file groupings already well differentiated c/o
> directory-tree layout, is there any further advantage
> to be had in partitioning *these* drives?
> 
> (I do understand somewhat the difference between the
> drive types -- e.g., that SSDs don't assign functional
> space.  I'm just not sure what other issue may apply.)
> 
> Thanks in advance for your time!
> 
> Best regards,
> Peter Armstrong
> 
> 
I do use (NVMe-) SSD, and i did partition it.
I did it to make sure, pages/partitions start on PHYSICAL boundaries,
not the logical ones reported to satisfy Windooze. Not every model
reports correct hardware parameters to the OS.

What i would recommend, is to use GPT/UEFI if possible, to avoid future
hassle and limitations (gdisk being your friend) and set alignment
manually to the physical page size. That is going to avoid eventual
read/write cycles to accomodate emulating 512e virtual sectors and also
wasting space on the drive.

Apart from that, i do not think, you would be limited to do anything you
want with the installation. My personal preference has always been to
have room for 3 OS partitions (20GB max each), one for real, one for
fast cloning/backup and one for wild experimentation. Any serious backup
belonging onto outside media/computer anyway.

just my 2 cents

But everybody has different needs and habits, so do, whatever suits you.

Re: awk not just using the Field separator as such. it is using the blank space as well ...

[DdB]

Am 15.02.2023 um 08:21 schrieb DdB:
> $ awk --version
> GNU Awk 4.2.1, API: 2.0 (GNU MPFR 4.0.2, GNU MP 6.1.2)
> Copyright © 1989, 1991-2018 Free Software Foundation.

even mawk would. see:
$ mawk -W version

compiled limits:
max NF 32767
sprintf buffer  2040

$ echo "Adams, Fred, and Ken Aizawa \"The Bounds of Cognition\"" | mawk
-F'\"' '{for (i=1; i<=NF; i++) print $i;}'
Adams, Fred, and Ken Aizawa
The Bounds of Cognition

Re: awk not just using the Field separator as such. it is using the blank space as well ...

[DdB]

Am 15.02.2023 um 07:25 schrieb Albretch Mueller:
> $ _L="Adams, Fred, and Ken Aizawa \"The Bounds of Cognition\""
> echo "// __ \$_L: |${_L}|"
> _AR=($(echo "${_L}" | awk -F'\"' '{for (i=1; i<=NF; i++) print $i}' ))
> _AR_L=${#_AR[@]}
> echo "// __ \$_AR_L: |${_AR_L}|"
> for(( _IX=0; _IX<${_AR_L}; _IX++ )); do
>  echo "// __ [$_IX/$_AR_L): |${_AR[$_IX]}|"
> done
what awk are you using? gnu awk works fine. see:

$ echo "Adams, Fred, and Ken Aizawa \"The Bounds of Cognition\"" | awk
-F'\"' '{for (i=1; i<=NF; i++) print $i;}'
Adams, Fred, and Ken Aizawa
The Bounds of Cognition

$ awk --version
GNU Awk 4.2.1, API: 2.0 (GNU MPFR 4.0.2, GNU MP 6.1.2)
Copyright © 1989, 1991-2018 Free Software Foundation.

Dieses Programm ist Freie Software. Sie können es unter den Bedingungen
der von der Free Software Foundation veröffentlichten GNU
General Public License weitergeben und/oder ändern.
Es gilt Version 2 dieser Lizenz oder (nach Ihrer Wahl) irgendeine
spätere Version.

Dieses Programm wird weitergegeben in der Hoffnung, dass es nützlich ist,
aber OHNE JEDE GEWÄHRLEISTUNG; nicht einmal mit der impliziten Gewähr-
leistung einer HANDELBARKEIT oder der EIGNUNG FÜR EINEN BESTIMMTEN ZWECK.
Sehen Sie bitte die GNU General Public License für weitere Details.
Sie sollten eine Kopie der GNU General Publice License zusammen mit
diesem Programm erhalten haben. Wenn nicht, lesen Sie bitte
http://www.gnu.org/licenses/.

How to use bridge-utils to enable connection sharing?

[DdB]

Am 05.02.2023 um 18:30 schrieb Dan Ritter:
> (...)
> RJ45 is a physical connector with 8 pairs of twisted wires. The
> twisted pairs carry the same signal but with the polarity
> reversed, + on one is - on the other. Together with the
> twisting, that makes the signal being carried resistant to
> interference. Ethernet is a low level protocol often carried by
> RJ45-connected twisted pair cables.
> 
> Bridging extends a network segment. Routing makes decisions
> about where IP packets should go.
> 
> The router that connects to the internet for you is probably
> doing several jobs:
> 
> - it is bridging from a long-haul network like xDSL, DOCSIS
> (cable modem), or fiber, to your local ethernet
That is correct, DOCSIS in my case.
> 
> - it is acquiring an IP address from the provider and using it
> as your public address
> 
> - it is probably using network address translation (NAT) to
> allow multiple internal addresses to masquerade as the single
> public address
> 
> - it is probably providing those internal addresses via DHCP
I can confirm all of the above. And i'd have to add, that matters got
more complicated with dozens of VM's running on the host, and atm 5
networks serving different purposes, which would really need some
unwinding to happen for progress to be even possible. I am afraid, that
i did create all this mess over time due to my understanding being so
poor...
> 
> It would be reasonably cheap to put an ethernet switch between
> the router and your existing host and new backup host. In the
> US, an 8 port ethernet switch is $28:
> https://www.newegg.com/netgear-gs308-300pas-8-x-rj45/p/N82E16833222029?Item=N82E16833222029
> 
> and you would need at least two more ethernet cables, which are
> also cheap.
> 
> $7.50 for preterminated 25 foot cable
> https://www.newegg.com/yellow-belkin-25-ft-accessories-cable-management/p/N82E16812107315?Item=9SIB897JGS5925
> 
> I presume that there are similar companies in Germany which will
> sell you basically the same things.
> 
> You unplug the host from the router, plug the router into the
> switch, and the two hosts into the switch. No configuration
> changes, most likely. Everything is likely to work.
> 
> ---
> If you insist on plugging your second host directly into your first,
> you can do that instead:
> 
> - configure an ethernet interface on both sides with an RFC1918
> network address
> 
> - plug them into each other
> 
> - you can now ping each from the other
> 
> - now you configure routing and NAT masquerading on your primary
> host to hide the secondary host behind it
> 
> I don't recommend this. Buy a switch and some cables.
> 
> -dsr-
> 

I believe, this would work, because i had someone lend me a switch for
some time and it just worked (somehow).
For some reason, i am still interested to get bridge-utils to work, and
i am taking your hint, to use static ip configuration as a starting
point. The difficulty in that case: the backup-server does not have any
OS at this point, which makes it necessary to switch keyboard and
monitor several times in the process, and my handicap forces me to ask
someone for physical help in order to do that.
Very inconvenient to say the least. :-(

How to use bridge-utils to enable connection sharing?

[DdB]

Am 05.02.2023 um 18:30 schrieb Dan Ritter:
> (...)
> RJ45 is a physical connector with 8 pairs of twisted wires. The
> twisted pairs carry the same signal but with the polarity
> reversed, + on one is - on the other. Together with the
> twisting, that makes the signal being carried resistant to
> interference. Ethernet is a low level protocol often carried by
> RJ45-connected twisted pair cables.
> 
> Bridging extends a network segment. Routing makes decisions
> about where IP packets should go.
> 
> The router that connects to the internet for you is probably
> doing several jobs:
> 
> - it is bridging from a long-haul network like xDSL, DOCSIS
> (cable modem), or fiber, to your local ethernet
That is correct, DOCSIS in my case.
> 
> - it is acquiring an IP address from the provider and using it
> as your public address
> 
> - it is probably using network address translation (NAT) to
> allow multiple internal addresses to masquerade as the single
> public address
> 
> - it is probably providing those internal addresses via DHCP
I can confirm all of the above. And i'd have to add, that matters got
more complicated with dozens of VM's running on the host, and atm 5
networks serving different purposes, which would really need some
unwinding to happen for progress to be even possible. I am afraid, that
i did create all this mess over time due to my understanding being so
poor...
> 
> It would be reasonably cheap to put an ethernet switch between
> the router and your existing host and new backup host. In the
> US, an 8 port ethernet switch is $28:
> https://www.newegg.com/netgear-gs308-300pas-8-x-rj45/p/N82E16833222029?Item=N82E16833222029
> 
> and you would need at least two more ethernet cables, which are
> also cheap.
> 
> $7.50 for preterminated 25 foot cable
> https://www.newegg.com/yellow-belkin-25-ft-accessories-cable-management/p/N82E16812107315?Item=9SIB897JGS5925
> 
> I presume that there are similar companies in Germany which will
> sell you basically the same things.
> 
> You unplug the host from the router, plug the router into the
> switch, and the two hosts into the switch. No configuration
> changes, most likely. Everything is likely to work.
> 
> ---
> If you insist on plugging your second host directly into your first,
> you can do that instead:
> 
> - configure an ethernet interface on both sides with an RFC1918
> network address
> 
> - plug them into each other
> 
> - you can now ping each from the other
> 
> - now you configure routing and NAT masquerading on your primary
> host to hide the secondary host behind it
> 
> I don't recommend this. Buy a switch and some cables.
> 
> -dsr-
> 

I believe, this would work, because i had someone lend me a switch for
some time and it just worked (somehow).
For some reason, i am still interested to get bridge-utils to work, and
i am taking your hint, to use static ip configuration as a starting
point. The difficulty in that case: the backup-server does not have any
OS at this point, which makes it necessary to switch keyboard and
monitor several times in the process, and my handicap forces me to ask
someone for physical help in order to do that.
Very inconvenient to say the least. :-(

-- 

Liebe ist ...
Datakanja

How to use bridge-utils to enable connection sharing?

[DdB]

Hello list,

Among all things in software business, networking is the area i know the
least about.

What i am trying to setup is to have an older unused pc act as a
backup-server. The first thing to do would be to install a current
debian (disks are already pluged in). And the installation absolutely
wants an internet connection.

Due to physical restrictions, my router has only one connection (used by
my main host), but there are more networking cards in it, so i am
intending to allow a connection from the host to the backuup server,
later that net is going to be used for the backing up of data. But now,
i would like to avoid any additional hardware (like bridge, router,
twisted pair, or such) and just have the backup server connect with a
normal network cable (rj-45?) to a secondary network card of the host,
and have the latter allow internet access thru it. (dhcp or fixed
address, whatever is easier)

In order to achieve that, i installed on the host (debian, buster for
the time being)

apt install bridge-utils

and looked up the interfaces in question (enp0s3 and enp0s8)

brctl addbr br0
brctl addif br0 enp0s3 enp0s8

but, even after rebooting, the bridge does not work

ip addr show

1: lo:  mtu 65536 qdisc noqueue state UNKNOWN
group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
   valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
   valid_lft forever preferred_lft forever
2: enp0s3:  mtu 1500 qdisc noop master br0 state
DOWN group default qlen 1000
link/ether 08:00:27:e1:93:46 brd ff:ff:ff:ff:ff:ff
3: enp0s8:  mtu 1500 qdisc noop master br0 state
DOWN group default qlen 1000
link/ether 08:00:27:7e:c0:bb brd ff:ff:ff:ff:ff:ff
4: br0:  mtu 1500 qdisc noop state DOWN group
default qlen 1000
link/ether 08:00:27:7e:c0:bb brd ff:ff:ff:ff:ff:ff

But now, i am at a loss: even after rebooting, the interfaces are not
coming up and thus i cannot install debian on the backup-server.

Can anyone point me to the misunderstanding of my ways, and get me going
again?

Re: Cloning a disk: partclone?

[DdB]

Am 19.01.2023 um 19:49 schrieb Tom Browder:
> On my main PC, I would like to clone my boot drive onto another disk for
> 2 reasons:
> 
> 1. Use a larger disk for the main drive
> 2. Create an emergency recovery disk
> 
> A new Debian package to me is "partclone". Questions:
> 
> + Can that be used for both purposes?
> 
> + Can it do a complete clone on an active disk? Or do I need a live CD
> or USB stick?
> 
> Thanks.
> 
> -Tom
When i set up a similar job on my neighbor's home comp, i chose
http://www.fsarchiver.org/
Straightforward, yet flexible, can deal with resizing partition(s) and
such. I use it from a live DVD, that is available thru grub menu, no
media involved.

Re: OpenSolaris Boot Environements equivalent

[DdB]

Am 19.01.2023 um 12:18 schrieb Yassine Chaouche:

> If nothing like it is avaiable for linux,
> what would be the necessary steps to achieve something similar?
> (even manually)
> 
Sorry for being late (i missed the post):
The ZoL gurus did work with the ubuntu team in order to provide all this
with their zfs on root install. (using lua to generate the grub scripts
from the snapshots) and some extension for ubuntu to keep the boot
requirements intact.

I myself did abandon ubuntu way before that, but i had a look and it
seems to work quite well. Only i am not returning to their derivative
just for this benefit. Maybe you want to take a look at it for yourself?

DdB

Re: reportbug: don't know: bug in apt [list] or in grep

[DdB]

Am 19.01.2023 um 13:13 schrieb Greg Wooledge:
> The fact that this *appears* to work is what causes so much confusion.
> It will "work" some of the time, but not all of the time, and you'll
> get different results depending on which directory you're in, on which
> computer.
> 
> Bash has two other settings for handling unmatched globs.  The first one
> is called "nullglob", and if it's turned on, an unmatched glob is simply
> discarded from the command argument list.

I was really curious, how Greg would put words to this one. And i gotta
applaud: Such unambiguous explanations, and so circumspect at the same
time. Even understanding the basis for confusion, i could learn
something new from this (other settings ...).

Excellent post there!
Thank you
DdB

Re: reportbug: don't know: bug in apt [list] or in grep

[DdB]

Am 19.01.2023 um 09:10 schrieb js-p...@online.de:
> Hello together,
> listing packages in apt with ”sudo“ in the title returns different output 
> (bash commands at the end of the email). I would fill a bug report, but I'm 
> not sure whether to address it to grep or apt. How do you see this?
> 
> Kind regards
> Julian Schreck
> --
> $ apt list sudo*   vs.  $ apt list | grep "^sudo[a-z-]"
> $ apt list *sudo   vs.  $ apt list | grep "[a-z-]sudo/"
> $ apt list *sudo*  vs.  $ apt list | grep "sudo"
> 
> 

I do not understand the issue. Nor do not see the "bug".
I do get different outputs from different commands though, as they are
not identical.

Re: Passwords

[DdB]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

thx everyone for sharing your point of view(s).
I am enjoying that food-for-thought and reconsidering ...

Am 17.01.2023 um 15:05 schrieb to...@tuxteam.de:
>> chroot can be tricky for newcommers…
> That's why passwd is nice to us and has the -R option :)
Whoa! nice one, much easier, in fact.
Much appreciated hint. Thank you
DdB

-BEGIN PGP SIGNATURE-

iQIzBAEBCAAdFiEEumgd33HMGU/Wk4ZRe3aiXLdoWD0FAmPGu1wACgkQe3aiXLdo
WD1vkBAAifiu2NoiTxD0nWV+zrX0KW8bIcop3VTAyajepFHY8T9FPh78Ol5D7fp8
2WpJYtX5FTchQz/i/Qy5n+yc1gkHP4A0MVpz/3OYJ/dXJ29l6b7rSNY9KDkAa7gB
VkO1COHKe6Oo459IwmsBNhmT49ZzlYj91d0LU3nB71pWKOw/iYVbCjtTZqmpVaGd
amQXGYr/xolDkYrj5RkzE9ReN5q1f8hLGQ4kDSYQqLVKj8+9Qz353KoYcxI1TTFu
NQcybsLQ/KAEHYcEV9o0LxiuPXWPC+tczgXicO6TGcP2LNQ+tvS9fb8IrmD1T49w
zwOAZoePr/eqfXUkfA6yKycTB1OHO2dltQ5CDoVPusIm3MuwfNhSDTJtSzYGdcde
4suGr1r3axHM2dTr/ivXQrxDPvcYcRWWKjXPilxS4MPrLIsAP6BK5xOpFSll7Rka
BSgw67dGLbyGT+ugVjqLm2lzPrc72KWEaFKTm6LLgqfTPynAikiHb4XOY9T6K5iH
f0PznG6xdTn5QA+18dVXRoWr3lLiFroJWHplV6sb+uNlHDuoPazvHLyJZ9vAWiL8
xHaxuaPpgRXSDYjRt7i5EXJCurfAFzhTGSmpOGE/k9IXQ5qdMD9v5xAX6tP2CFEJ
vkrd5dvXkoCO+8dl2jL85WXffLlE44Qp7KM4k5Rx4xFFQpILWbI=
=eIKP
-END PGP SIGNATURE-

Re: Passwords

[DdB]

Am 17.01.2023 um 07:14 schrieb Stanislav Vlasov:
> вт, 17 янв. 2023 г. в 11:01, David :
>> Looking on the internet it says the passwords are stored in /etc/passwd
>> and /etc/shadow
> 
>  In /etc/shadow only password's hashes, some data, one-way calculated
> from password string.
> 
>> The password string in /etc/shadow looks as if it's encoded, how can I
>> read this string?
> 
> You can't.
Everyone (and their friend) seem to know, how to work around this, which
apparently is common debian knowledge (which is nice).

But somehow, i feel there could be more caring about avoiding to teach
future hackers by accident. Is this kind of lesson appropriate for a
users list? - I doubt it.

just my 2 cents
DdB

Re: How can I check (and run) if an *.exe is a DOS or a Windows program?

[DdB]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Am 07.01.2023 um 13:40 schrieb The Wanderer:
> On 2023-01-07 at 07:13, Timothy M Butterworth wrote:
> 
>> All .exe files are windows, linux does not use .exe!
> 
> While the latter statement is (largely) correct, the former is not.
> Some .exe files are DOS programs, not Windows programs.
> 
> The question being asked is how to tell which type a given EXE is,
> and assuming it turns out to be a DOS program, how to run it.
> 

OMG, i really have some difficulty to access my own knowing, as i used
to handle such problems almost one generation ago. ;-)

But my first research turned up some useful hints.

But mostly: i did not receive the OP's originel post (for whatever
reason) and am therefore lacking a pointer to the question, that
really woke up his interest.

This is my list of links:
> http://justsolve.archiveteam.org/wiki/Linear_Executable 
> https://www.techtarget.com/whatis/definition/executable-file-exe-file
>
> 
https://fileinfo.com/extension/exe
> https://en.wikipedia.org/wiki/Comparison_of_executable_file_formats
>
> 
http://justsolve.archiveteam.org/wiki/MS-DOS_EXE
> https://en.wikipedia.org/wiki/DOS_MZ_executable 
> https://en.wikibooks.org/wiki/X86_Disassembly/Windows_Executable_Files
-BEGIN PGP SIGNATURE-

iQIzBAEBCAAdFiEEumgd33HMGU/Wk4ZRe3aiXLdoWD0FAmO5ddQACgkQe3aiXLdo
WD3YqA//STrZTqCWgcvC9ddT0zT2J5dTuRr7V60S8Nahexn1MnepuJO92h93UUIB
MB0BI5RIw4WLtQOaUqWHtsIK4RB8W+40PUJkT4AQwXR2mcObvX19DLiNmpCp4gQR
O57JRdthgj+E7VWnvT1ez0ZBD26+CWoBmiYo6wKdmmVRlAie8K0wAWEufNBflFQ4
ydydph1ikl2RIbbNRbojC47eg7ZrymMc8XuSGbM6BH76Q6ukhW+fw/TGbxBtA6zl
HqgKIMJrlsY2dq+zOK0XJHLyagTLw5MYDvE3ZUavUeB2coXnio1cXgw/URSSWOYT
CN+bQjOFhhL+16LH+iKs410jS9wNmVQ3VDZefH5DXKhAPua8EIvkmv2+8w16AEAO
32ED5Z5MEaT97RkEGrX18upyBp6jltM42xSDeUYxizeJKIF8Vzc6WRE6fi4CINfY
6upiLHt9uBACM48wSiukB3n9d0JzWe/KCDBoztbeWqCywsMYTw43x7cE7zOrKE7L
N6/RXvfdiVXfH83L4ELztIfqSaT9RV8NX5EG6Un6yVe5bcAV0wNgNFpU9g9i+yuC
zeifomxcGnBdnI/qWkgLP2OBRUFOgylqlnjCn5QDjPOcwbIpbDb6v2FiDNSHh25Q
8Jrj8afA3YvnT17d7s/f7TDMtEqOK4aQWqhDk0INv24upZXWud4=
=v6UH
-END PGP SIGNATURE-

Re: request a replacement for Thunderbird + Enigmail

[DdB]

Am 03.01.2023 um 15:04 schrieb Jeffrey Walton:> Claws email with the GPG
plugin is popular. It may be a good fit for you, too.
>
> https://www.claws-mail.org/plugin.php?plugin=gpg
>
> Jeff

Thank you so much. I am going to check it out, test it and possibly
prepare the transition if all goes well. (Not a short term adventure,
but i do have the time ...)

Have a happy year 2023!
DdB

request a replacement for Thunderbird + Enigmail

[DdB]

Hello List,

i feel the time has come to find a more up-to-date replacement for my
email-solution, but ...

Up til now, i am using Thunderbird (52.9.1 (64-Bit) + Enigmail +
ToneQuilla + Virtual Identity and more ...) on stretch from inside a
Virtualbox-VM.

That allowed to correspond seamlessly with GPG users on more than 20
addresses (from 6 different providers), while i was acoustically
notified about the different levels of relevance of those mails, since
20+ years. (makes a huge database)

While keeping this stuff alive for an extended period of time, i have
been able to prevent the necessity to enter any passphrase manually, to
decrypt/encrypt automatically, and many more convenient solutions to
ease my life.

Of course, i knew, that this could come to an end, but i have not found
a valid replacement yet. The current Thunderbird fails big time at
solving the GPG part, even worse: They (Mozilla/NSA/???) want to have
control over private keys in their own keystore, built by people
obviously not skilled enough to create a(ny) secure piece of software.
That is why i refused to embark on the path of replacing the isolated,
but integrated solution Enigmail with something less trustworthy, less
flexible, and less stable.

But i feel, i will have to start building up an alternative, could not
yet make up my mind.

That is why i am asking for your communities experience and recommendations.

How are YOU dealing with encryption, with multiple providers, with
addresses created on-the-fly, with a huge email history, and so on?

Really interested to find a solution, that can last for many years to come.

regards, DdB

Re: Logout at apt upgrade

[DdB]

Am 29.11.2022 um 23:35 schrieb Loïc Grenié:
>     when I apt upgrade my system, I often (one every three, more
>   or less) find myself brutally logged out of the window system,
>   with systemd services painfully restarting (or failing to restart).
>   The only way I can recover is usually to reboot.

Yes, i see similar problem, amd AFAICT it is somewhat related to my use
of GNOME3. They insist on restarting the system for almost all package
updates, but sometimes, i refuse to and keep the machine running without
updating it. I am aware, this could cause troubles on the security-side.
That is why i am keeping a long list of system-backups.

But whenever i get logged out by the system (funnily, usually there are
applications running fine on other desktops), i could never sort out the
cause and had to reboot. But i do not complain, since i am still on
oldstable and thus not in touch with the most recent and best of debian.

Re: FIREFOX is killing the whole PC Part III

[DdB]

Am 20.11.2022 um 12:06 schrieb Schwibinger Michael:
> 
> 2 Questions:
> 
> What does it do?

The script, that you included, seems to search for a couple of browser
processes to limit their usage of CPU resources.

I do not think, this would be such a great idea.

In order to avoid, that a browser changes your system in any way, you
could do, what i do:

I did locate the places, the browser uses (not just as cache, but also
for storing defaults, history, and such). Then i use a tmpfs to create
an overlay filesystem and allow the browser to change those filesystems
only temporarily.
works as intended, but created problems, if you WANT to change something
(like f.i. download a file).

But this method is a bit difficult to implement for someone, who is
incomfortable with commandline and scripts.

Re: FIREFOX is killing the whole PC

[DdB]

Am 19.11.2022 um 15:04 schrieb Schwibinger Michael:
> Hello
> Thank You.
> I tried
> also with CPU Limit
> bute it did catch the whole CPU:
> 
> Where can I delete by terminal the whole cache?
> 
> Regards
> Sophie

Hello Sophie!

I suggest you to relax a bit, otherwise your aggressive attempts to
remedy a sim0le problem might cause further havor that can be difficult
to fix later.

You situation is not dramatic, and erasing the cache is possible, but
not meaningful.

Myself, i am using Firefox differently from you, it never reopens
anything due to my settings. But i just tried it and saw, that safe-mode
was in fact not helping. (Sorry for that!)

And i found a proper description of how to fix the problem here
<https://support.mozilla.org/en-US/questions/870805>

I hope your english will be good enough to understand the approach, and
your computer skill will suffice to execute this remedy.

Otherwise ask again on the list and people will be able to explain it
differently.

best regards, DdB

Re: FIREFOX is killing the whole PC

[DdB]

Am 19.11.2022 um 10:34 schrieb Schwibinger Michael:
> Hello
> 
> Any idea?
> 
> What did happen?
> FF did open a page with bad PC,
> so it needs 5 minutes to open it.
> We killed the tab.
> When we now try to open FF
> whole PC is blocked.
> How can we clean FF
> because bad page is in it.
> 
> Regards
> Sophie
> 
Did you try to start it from the commandline with the option safe-mode?

> firefox --safe-mode

Re: gpg says no user ID

[DdB]

Am 15.11.2022 um 19:20 schrieb Jeffrey Walton:
> It's not you. The whole key server architecture is a mess nowadays.
> 
> Jeff

Thank you, Jeff, for your explanation and hints.

Just one additional viewpoint from me (an afficionado for data
protection) ;-)

As i see it, the european law suffers from using wrong understanding and
wording, a problem, that is very difficult to fix, as long as teachers,
universities, and many professionals are not even aware of the cause of
their troubles.

I was hoping, that the corona epidemic would demonstrate clearly, how
that new law hinders science and other communities to collect useful and
sometimes necessary information, much to the detriment of social sanity.

But life goes on and people do not seem to notice the errors of their
ways. And the resolution is not a technical problem, it is an awareness
and an agreement moderation problem. But AFAICT there are very few
people even noticing the misdirection of our policies, and certainly not
enough people to push a change ... uptil now.

So society goes on onto the misguided path, creating further problems
along the way. i myself am rather desperate when i see people involved
in computer security regularly failing at ringing the bell. Not really
wondering, that keyservers now display noticeable problems, that would
make a global change in perspective desirable.

just my 2 cents (probably not on the best list to air this)
stopping now (AFK) :-))

DdB

Re: gpg says no user ID

[DdB]

Am 14.11.2022 um 23:17 schrieb Thomas George:
> I am still trying to do a fully verified installation of debian-11.5.0.
> 
> gpg --verify SHA512SUMS.sign SHA512SUMS responded with DF98...BE9B
> 
> gpg --recv-keys DF98...BE9B responded key DF98...BE9B: new key but
> contains no user ID - skipped.
> 
> Another source suggested gpg --key-server keyring.debian.org --recv-keys
> long numeric key
> 
> but this responded invalid option --key-server
> 
> The gpg man page is beyond me, I need help
> 
> 

Hi,
i just experienced the same problem, same difficulty understanding the
man pages. Googling suggested to try a different keyserver.
I had to try several ... until i found one, that succeeded.
Apparently, the cause was found to be in the rules in place at the
servers. Some do no longer allow connections from sources without
self-signed pubkeys. But just adding the corresponding switch did not
help either. i had to find a server with an older config in place (like
ubuntu's).

Re: Explaining snapshots (for backup)

[DdB]

Am 15.11.2022 um 15:27 schrieb rhkra...@gmail.com:
> I'm not really clear on the concept of a snapshot (for backup) -- I've done a 
> little googling but haven't found an explanation that "satisfies" me.

I am familiar with snapshots on zfs. There might be different meanings
in other contexts, idk ...

zfs is s COW filesystem, meaning "copy-on-write". Whenever you write to
an already existing block (piece of a file), a new block gets created
(copied and modified) and on closing the write (or on sync), the
(metadata-) pointer/directory entry changes to point to the new block.
This algorithm primarily implements transactions on the filesystem to
garantee, it has a consistent state at all times. but one side effect
is, that the old filesystem state still hangs around. And a snapshot
basically points to that outdated state of a filesystem and keeping a
shapshot means to prevent the reuse of those blocks until the snapshot
gets freed up again.

Does that help your understanding?

Re: RFC: What would be the "correct debian way" to clean up unwanted languages from an installation?

[DdB]

Am 15.11.2022 um 05:21 schrieb David Christensen:
> I installed the localepurge package.  Storage usage did not change. Then
> I realized that I had chosen the "C" locale during installation, so
> perhaps there is nothing to be removed (?).  So, I removed the
> localepurge package.
> 
> 
> David

I just read (from /usr/share/doc/localepurge/README.dpkg-path):
> localepurge dpkg support
> 
> 
> Starting with version 1.15.8, dpkg supports --path-include and
(...)
> 
>  * It cannot be used to purge locale files from already installed
>packages.
>- Though it will be fixed on next upgrade (or reinstall) of the
>  packages.
(...)

which indicates, that your observations had to be expected, if you opted
for dpkg-path :-)

Re: RFC: What would be the "correct debian way" to clean up unwanted languages from an installation?

[DdB]

Am 14.11.2022 um 07:16 schrieb Anssi Saari:
> Charles Curley  writes:
> 
>> On Sun, 13 Nov 2022 16:32:51 +0100
>> DdB  wrote:
>>
>>> every backup contains loads of unnecessary language files, and i saw
>>> them scroll by during rsync. So one day, i wanted to get rid of those.
>>
>> You might take a look at the localepurge package.
> 
> As I'm a little short on space on /, it's interesting but the
> description says:
> 
> "This tool is a hack which is *not* integrated with the system's
>  package management system and therefore is not for the faint of heart.
>  Its interference can provoke strange, but usually harmless, behavior in
>  programs related to apt/dpkg, such as dpkg-repack, reportbug, etc.
>  Responsibility for its usage and possible breakage of the system
>  therefore lies in the system administrator's hands.
> "
> 
> So while I'm not faint of heart, I think I'll rather resize my /
> partition. Even if it means moving partitions around a little.
> 
> 
> 
Thank you for pointing at that. I looked at that package in a VM and
found: although its language discovery is better than mine, it is
failing to find some language files, that i my script would. And
besides: my vm's do not have an actual problem with space. It is more
the backups (and their snapshots) containing multiple versions thereof
that are troubling me. I think, i am going to look at their hack itself
in an intent to learn from it, instead of using it.

RFC: What would be the "correct debian way" to clean up unwanted languages from an installation?

[DdB]

Hello,

every backup contains loads of unnecessary language files, and i saw
them scroll by during rsync. So one day, i wanted to get rid of those.

What a surprise: doing it in my (rather simplistic) fashion, i got rid
of almost 1,5 GB from every buster machine, i had ... only to find, that
they were eventually coming back through system updates. :-(

But - since my know-how on debian is very restrained - i did create a
script, that allowed me to examine the situation more easily, without
damaging the install, i was looking at. And only after verification did
i run the removal. (BTW: in bullseye, i found there was much less
unnecessary stuff) But still, the deleted files came back through
updates, because i dont know, how to tell apt about my removals the
proper way.

Would anyone be willing to take a look at what i have been doing and
guide me to a resolution of the missing parts?

My current (bash) script can be found here:
https://paste.debian.net/1260563/

Thank you for your attention
DdB

Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?))

[DdB]

Am 11.11.2022 um 07:36 schrieb hw:
> That's on https://docs.freebsd.org/en/books/handbook/zfs/
> 
> I don't remember where I read about 8, could have been some documentation 
> about
> FreeNAS.

Well, OTOH there do exist some considerations, which may have lead to
that number sticking somewhere, but i have seen people with MUCH larger
pools.

In order to avoid wasting too much space, there was a "formula" to
calculate optimum pool size:
First take an amount of disks that is a result of 2^^n (like
2/4/8/16/...) and then add the number of disks you need for redundancy
(raidz = 1, raidz2 = 2, raidz3 = 3). That would give nice spots like 4+2
= 6 (identical) disks for raidz2, or 11 for raidz3. Those numbers are
sweet spots for the size of vdevs, otherwise, more space gets waisted on
the drives. But that is only ONE consideration. My motherboard has 8
connectors for SATA, + 2 for NVME, which limited my options more than
anything.
And after long considerations, i opted for 4 mirrored vdevs, giving even
more space to redundancy, but gaining read speed.

Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?))

[DdB]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Am 10.11.2022 um 22:37 schrieb Linux-Fan:
> Ext4 still does not offer snapshots. The traditional way to do
> snapshots outside of fancy BTRFS and ZFS file systems is to add LVM
> to the equation although I do not have any useful experience with
> that. Specifically, I am not using snapshots at all so far, besides
> them being readily available on ZFS

Yes, although i am heavily dependant on zfs, my OS resides on an
nvme-ssd with ext4, i rsync as need be to an imagefile, that i keep on
compressed zfs, and which i snapshot, backup and so on, rather than
the ext4-partition itself. - Seems like a good compromise to me.
-BEGIN PGP SIGNATURE-

iQIzBAEBCAAdFiEEumgd33HMGU/Wk4ZRe3aiXLdoWD0FAmNterMACgkQe3aiXLdo
WD3coA//ZXf0/WfGVEfEi1Fxe/vYpnqqx9UZLfAL5+XrE19Gh1oVd25zGDhkaaFl
SbvwcnVII/v7Lzj6by86nJ44LvPqu/NRjzWGwM7ltK3t4t8C7C+h2lfxPuhVKxfW
zqt/kp053ZCPUj6nD8nD60MLI88sxoyTVRG6nVzqW0FiC7be3VE3l4l4O2E0Qr4U
OM2lqmLDXPJcJ6pNGZp5p4470st/ODBNuG/7nM+mM04ylZJYLJV7ykYpCx6R8uiW
ZKoY+VZuI0dPsTEe24O0CRRm2hmW99ET6p+LAZHEnhKGPB/cxOIECiLTmqy8NICi
90AMxTp+D7bLglgnF4a0ZAukYwxnj+gCMj7B/CKCT62qLVVEavLOdVQydQx/7kue
+DsJ8PrXryVhO7xL01NeZbq4Ur5vwbY1Unk5iWHq8snoh+13Dru+a3DGcRQNr2Ph
QkzHny8LwO7x4Ob+a4/YRhjGjYWPxA9Y9huUDFLEDp0v0QDixtI+oytqED/hE30b
SPWZUbQR2pfF8Mbst02zDqknv5rvt9NxNhh0tvcspVsNv4y81/wUO0O8HVc4yBL2
lcJ6Wf11MRmbJ9J4NoZom3GnFUcysqnfEsxk+XLwJUTpfHU3VZS5Ovheu/OZNrDT
ErRY9g+yES8YTtsb4Vk2hqIvU5LvGu/BzCh0kyavGWkz3+dE9go=
=Ed39
-END PGP SIGNATURE-

ZFS pool size (Re: else or Debian)

[DdB]

Am 10.11.2022 um 14:28 schrieb DdB:
> Take some time to
> play with an installation (in a vm or just with a file based pool should
> be considered).

an example to show, that is is possible to allocate hugefiles (bigger
than a single disk size) from a pool:

> datakanja@PBuster-NFox:~$ mkdir disks
> datakanja@PBuster-NFox:~$ cd disks/
> datakanja@PBuster-NFox:~/disks$ seq  -w 0 15 | xargs -i truncate -s 4T 
> disk{}.bin # this creates sparse files to act as virtual disks
> datakanja@PBuster-NFox:~/disks$ zpool create TEST raidz3 ~/disks/d*
> datakanja@PBuster-NFox:~/disks$ zpool list
> NAME   SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAGCAP  DEDUPHEALTH  
> ALTROOT
> TEST  64.0T   314K  64.0T- - 0% 0%  1.00xONLINE  -
16*4 TB = 64 TB size
> datakanja@PBuster-NFox:~/disks$ zfs list TEST
> NAME   USED  AVAIL REFER  MOUNTPOINT
> TEST   254K  50.1T 64.7K  /TEST
# due to redundacy in the pool, the maximum size of a file is slightly
over 50TB

#do not forget to clean up (destroying pool and files)

Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?))

[DdB]

Am 10.11.2022 um 13:03 schrieb Greg Wooledge:
> If it turns out that '?' really is the filename, then it becomes a ZFS
> issue with which I can't help.

just tested: i could create, rename, delete a file with that name on a
zfs filesystem just as with any other fileystem.

But: i recall having seen an issue with corrupted filenames in a
snapshot once (several years ago though). At the time, i did resort to
send/recv to get the issue straightened out.

But it is very much more likely, that the filename '?' is entirely
unrelated to zfs. Although zfs is perceived as being easy to handle
(only 2 commands need to be learned: zpool and zfs), it takes a while to
get acquainted with all the concepts and behaviors. Take some time to
play with an installation (in a vm or just with a file based pool should
be considered).

Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?))

[DdB]

Am 10.11.2022 um 04:46 schrieb hw:
> On Wed, 2022-11-09 at 18:26 +0100, Christoph Brinkhaus wrote:
>> Am Wed, Nov 09, 2022 at 06:11:34PM +0100 schrieb hw:
>> [...]
>>> FreeBSD has ZFS but can't even configure the disk controllers, so that won't
>>> work.  
>>
>> If I understand you right you mean RAID controllers?
> 
> yes
> 
>> According to my knowledge ZFS should be used without any RAID
>> controllers. Disks or better partions are fine.
> 
> I know, but it's what I have.  JBOD controllers are difficult to find.  And it
> doesn't really matter because I can configure each disk as a single disk ---
> still RAID though.  It may even be an advantage because the controllers have 
> 1GB
> cache each and the computers CPU doesn't need to do command queuing.
> 
> And I've been reading that when using ZFS, you shouldn't make volumes with 
> more
> than 8 disks.  That's very inconvenient.
> 
> Why would partitions be better than the block device itself?  They're like an
> additional layer and what could be faster and easier than directly using the
> block devices?
> 
> 
hurts my eyes to see such desinformation circulating. But i myself am
only one happy zfs user for a decade by now. I suggest to get in contact
with the zfs gurus on ZoL (or read the archive from
https://zfsonlinux.topicbox.com/groups/zfs-discuss)

Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?)

[DdB]

Am 10.11.2022 um 06:38 schrieb David Christensen:
> What is your technique for defragmenting ZFS?
well, that was meant more or less a joke: there is none apart from
offloading all the data, destroying and rebuilding the pool, and filling
it again from the backup. But i do it from time to time if fragmentation
got high, the speed improvements are obvious. OTOH the process takes
days on my SOHO servers

Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?)

[DdB]

Am 09.11.2022 um 12:41 schrieb hw:
> In any case, I'm currently tending to think that putting FreeBSD with ZFS on 
> my
> server might be the best option.  But then, apparently I won't be able to
> configure the controller cards, so that won't really work.  And ZFS with Linux
> isn't so great because it keeps fuse in between.

NO fuse, neither FreeBSD nor debian would need the outdated zfs-fuse,
use the in.kernel modules from zfsonlinux.org (packages for debian are
in contrib IIRC).