Re: [gentoo-user] Graphics configuration for a Ryzen 7 7700X chip and water cooling.

[Frank Steinmetzger]

Am Wed, May 15, 2024 at 07:08:11PM +0100 schrieb Michael:
> Hi Alan,
> 
> On Wednesday, 15 May 2024 15:23:47 BST Alan Mackenzie wrote:
> > Hello, Gentoo.
> > […]
> > So I'm looking at getting an AMD Ryzen 7 7700X processor, and using its
> > inbuilt graphics rather than buying a distinct graphics card.
> > 
> […]
> > As a somewhat tangential question, would it be worthwhile getting water
> > cooling in my new machine?  In particular, to reduce the noise it gives
> > off while building large packages such as clang and rust?  Or is water
> > cooling only sensible for really heavy users such as gamers?
> > 
> > Thanks for the upcoming answers!
> 
> WC will be quieter and more expensive than an after market air cooler.

Are you sure about the noise? First there is the water pump and second, 
the heat from the air cycle needs to get somewhere, which is donw with fans.
So unless you get a big radiator with several fans, you just relocate the 
fan noise inside the case.

I have a 10 years old i5 with a TDP of I think 84 W. On that sits a normal 
(not even high-performance) tower cooler with a single 120 mm fan. At full 
load the CPU draws around 50 W, maybe even less unless you do prime95. So my 
cooler is basically overkill. But this allows the fan to never leave the 
minimum RPM range of ~500…600 1/min and is unaudible even at full load.
However …

> You could invest the money toward more RAM, (more/bigger) case fans, a 
> better PSU, monitor, speakers, a new car, etc.  :-)
> 
> https://www.techreviewer.com/tech-specs/amd-7700x-tdp/
> 
> Cranking up 16 threads to 5.4 GHz will produce some heat, but compiles will 
> complete sooner too.

… the 7000X are hotheads, because they operate way above the efficiency 
sweetspot just to get the longest bar in benchmark diagrams. If you reduce 
the power target¹ in the BIOS, you lose a few percent in performance, but 
get a disproportionately bigger reduction in energy consumption.

¹ The TDP of a 7700X is 105 W. The maximum permanent power draw is TDP * 1.4 
(ish, can’t remember the exact details right now). So if you reduce the 
target to 84 W, you draw a little over 100 W. That’s easy-peasy for a mormal 
120 mm tower cooler. One additional advantage of an air cooler is that it 
also blows air over your mainboard and its power stages. That’s something 
you don’t get with a water loop and need an extra case fan for—IF you keep 
the CPU on high load all the time which causes more heat buildup in the VRMs.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

The perfect diet: no breakfast in the morning,
in return forego pudding at lunch and then go to bed without dinner.


signature.asc
Description: PGP signature

Re: [gentoo-user] Encrypted drives, password generation and management howto, guide.

[Frank Steinmetzger]

Am Tue, May 14, 2024 at 06:28:17AM -0500 schrieb Dale:
> Howdy,
> […]
> remember either, or write notes to remember them.  I also wanted to
> avoid the desktop copy and paste, or clipboard, mechanism.  I'm not sure
> how that data is stored in the clipboard and how good it is at erasing
> it when I clear it.

The mark-and-middleclick you describe further down is the very same as the 
“normal” clipboard. It is just accessed differently.

> First, I needed to generate a password.  I googled, a lot.  I had
> trouble finding a way to generate the type of passwords I wanted but I
> finally found one.

Care to elaborate regarding the “password you wanted”? There is the obvious 
pwgen, which can generate passwords with given character sets and length. 
Keepass can do this, too, so I assume, Bitwarden (which you use) has a 
similar function.

And if you don’t like parts of the generated PW, keep the part you like, 
generate new and pick the part you like again. Or just let pwgen generate a 
big bunch and pick what you like best from the output.

> […]
> Now that I have a password, how do I keep track of them?  I did some
> more searching.  I wanted something that was command line not GUI. 
> After all, I have BitWarden for websites and such already.  Thing is,
> it's GUI since it is a Firefox add-on.  I'd need to use the clipboard to
> copy and paste.  I want to avoid that remember?  I also wanted something
> that is on its own, separate from my main password tool BitWarden.  I
> found kpcli in the tree.

I didn’t know about kpcli and it is not available in Arch. So I looked it 
up. Turns out it is a non-graphical Keepass client (that’s what the kp 
stands for, after all).

Interestingly, there is also a bitwarden CLI client.

Did you know Keepass (the graphical one) has an autotype feature? This means 
that it simulates the pressing of keys, so it bypasses the clipboard 
entirely. Another advantage of that is that you can set up custom key 
sequences in the autotype field, so you can for example say “first enter the 
username, then press enter, then wait for a second, then enter the password 
and press enter again.” Useful for sites that use a dynamic login screen 
with animations or non-standard input fields.

> Then I needed some way to handle if the password file kpcli uses got
> lost or damaged.  If I were to lose that file, all drives and the data
> on them is lost.  I'd lose everything because there is no way to
> remember the password.

The obvious answer is: backup – encrypted or not. ;-)
My Keepass database is a simple file in my home that is backed up together 
with all the other home files by Borg. Meaning I even have a versioned 
backup of my passwords. Needless to say my backup drives are LUKSed with a 
long passphrase that I have never ever once written down anywhere on paper. 
I’ve been using it for so long now and on several drives, that it is 
ingrained in my brain.

> The kpcli file itself appears to be encrypted. 
> So, it protects itself.  That's good.  I don't need to put the file on
> something that is also encrypted, just copy it to a plain file system as
> it is.  I have a USB stick that I store things on.  Things like drive
> info, what drives go to what volume group, what drive has the OS on it
> etc and the portage world file on it.  I also have some scripts in /root
> that I don't want to lose either so I copy them to the stick as well. 

Be mindful that USB sticks aren’t very reliable. The flash chips in them are 
what is left after quality control deemed them unfit for duty in SSDs (first 
tier) and memory cards (second tier). So always keep several copies, 
possibly on different types of storage media (HDDs, SSDs, optical, whatever).

> Then one important file, my file that contains frequently used
> commands.  It is rather lengthy and is 15 years or more of additions.  I
> copied all that info to a USB stick.  It lives in the fire safe.

TBH, I wouldn’t put all my horses on one USB stick in a fire safe. (Or 
however the saying goes) After a flimsy USB stick with questionable flash 
chips has been subjected to high temperatures for a longer time, chances are 
you may not be able to access its data ever again.

> How I use all this.  I do this in a Konsole, within KDE, which has
> tabs.  Might work on a plain console to tho.  If I need to open a
> encrypted drive, or set of drives, I open kpcli and get it to show the
> password for that drive in one tab.  I then run the little script to
> open and mount that drive in another tab.  When it asks for the
> password, I highlight the password from kpcli tab and then switch tabs
> and middle click to paste the password in.

Since you’ve already scripted most of it, you could possible go the full 
way. Use the HDD’s UUID as key and either store the password in a file that 
is named with the UUID, or in keepass with the UUID as entry title. Then you 
can let the script retrieve the password all by itself without any need for 
copy-pasting – 

Re: [gentoo-user] cross-compiling environment for Rapberry

[Frank Steinmetzger]

Am Mon, Apr 29, 2024 at 11:26:49PM +0200 schrieb ralfconn:
> Hello,
> 
> I recently got me a Raspberry Pi4b to use as a PiHole [1]. As a first step I
> put user-space Gentoo (i.e. aarch64 stage3) on it and now I am trying to set
> up my desktop to cross-compile binary packages for the PI, to keep the Pi
> up-to-date in reasonable computing time.
> [...]
> /usr/aarch64-unknown-linux-gnu/var/db/pkg on the desktop. I ran emerge
> --sync on the Pi and on the desktop approximatively at the same time.

You could simply rsync the portage tree from one host to the other to get an 
identical state on both.

> [different emerge -u on both hosts]
>
>  What am I missing to make the desktop use the same list of packages to
> be upgraded as the Pi? 

Do you have the same use flags? I’ve never really dealt with crosscompiling. 
Maybe somewhere deep in the system the use flags are changed. Try comparing 
the output of emerge --info from both hosts.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

Development aid is to give money from the poor of rich countries
to the rich of poor countries.


signature.asc
Description: PGP signature

Re: [gentoo-user] NAS box and switching from Phenom II X6 1090T to FX-6300

[Frank Steinmetzger]

Am Thu, Apr 18, 2024 at 07:26:30AM -0500 schrieb Dale:

>  If you don't play games, then definitely get integrated graphics.
> > I'd add to this, you could still play many games, especially older games 
> > using 
> > a modern APU.  The integrated graphics capability is broadly comparable 
> > with 
> > the entry level discrete GPUs.  For driving a couple of monitors and 
> > watching 
> > videos an APU is more than adequate, saves money on a graphics card and 
> > consumes less power.
> >
> 
> The biggest reason I like a separate video card, I can upgrade if
> needed.

If you don’t play (big) games, then there is no reason to upgrade (except 
if you plan on working with AI stuff).

> Built in video means a new mobo.

No, a new CPU. The mobo only provides the lanes from the iGPU to the 
connectors on the back. The only constraint imposed by the motherboard may 
be an older version of the display link, like DisplayPort 1.2 instead of 
1.4. Only the latter supports 4K @ 120 Hz, the former tops out at 60 Hz.

> I'd suspect even the wimpiest video card would do what I need.

In that case, every iGPU would do what you need. 嵐 The only exception may 
be some hot new video hardware encoder. RDNA2, as can be found in Ryzen 
7000s, now supports AV1 decoding, which was still lacking in the 5000s.

-- 
Grüße | Greetings | Salut | Qapla’
“Meow”   “Woof”Jeez, it’s really raining today!


signature.asc
Description: PGP signature

Re: [gentoo-user] NAS box and switching from Phenom II X6 1090T to FX-6300

[Frank Steinmetzger]

Am Wed, Apr 17, 2024 at 01:18:39PM -0400 schrieb Rich Freeman:
> On Wed, Apr 17, 2024 at 9:33 AM Dale  wrote:
> >
> > Rich Freeman wrote:
> >
> > > All AM5 CPUs have GPUs, but in general motherboards with video outputs
> > > do not require the CPU to have a GPU built in.  The ports just don't
> > > do anything if this is lacking, and you would need a dedicated GPU.
> > >
> >
> > OK.  I read that a few times.  If I want to use the onboard video I have
> > to have a certain CPU that supports it?  Do those have something so I
> > know which is which?  Or do I read that as all the CPUs support onboard
> > video but if one plugs in a video card, that part of the CPU isn't
> > used?  The last one makes more sense but asking to be sure.
> 
> To use onboard graphics, you need a motherboard that supports it, and
> a CPU that supports it.  I believe that internal graphics and an
> external GPU card can both be used at the same time.  Note that
> internal graphics solutions typically steal some RAM from other system
> use, while an external GPU will have its own dedicated RAM (and those
> can also make use of internal RAM too).

You can usually set the amount of graphics memory in the BIOS, depending on 
your need and RAM budget.

> The 7600X has a built-in RDNA2 GPU.   All the original Ryzen zen4 CPUs
> had GPU support, but it looks like they JUST announced a new line of
> consumer zen4 CPUs that don't have it - they all end in an F right
> now.

Yup.
G-series: big graphics for games n stuff, over 3 GFlops
F-Series: no graphics at all
rest: small graphics (around 0.8 GFlops max), ample for desktops and media

X-Series: high performance
non-X: same as X, but with lower frequencies

The X series are boosted to higher frequencies which give you a bit more 
performance, but at the cost of disproportionally increased power 
consumption and thus heat. They are simply run above the sweet spot in order 
to get the longest bargraph in benchmarks. You can “simulate” a non-X by 
running an X at a lower power target which can be set in the BIOS. In fact 
once I have a Ryzen, I thing I might limit its frequency to a bit below 
maximum just to avoid this inefficient region.

But I’ll be buying a G anyways. Its architecture is different, as it is 
basically a mobile chip in a desktop package.

As to the qestion about 5/7/9 in the other mail: it’s just a tier number. 
The more interesting is the 4-digit number. 600s and below are 6-core chips, 
700 and 800 have 8 cores, 900s have 12 cores or more.

The thousands give away the generation. AM5 is denoted by 7xxx. (Though 
there is another numbering scheme that does it quite differently, like 
7845H.)

> In any case, if you google the CPU you're looking at it will tell you
> if it supports integrated graphics.

I also recommend Wikipedia. It has tables of all kinds of stuff. Including 
all processors and their core features.
https://en.wikipedia.org/wiki/Zen_4
https://en.wikipedia.org/wiki/List_of_AMD_Ryzen_processors

> If you don't play games, then definitely get integrated graphics.
> Even if the CPU costs a tiny bit more, it will give you a free empty
> 16x PCIe slot at whatever speed the CPU supports (v5 in this case -
> which is as good as you can get right now).

Not to mention a cut in power draw.

> > I might add, simply right clicking on the desktop can take sometimes 20
> > or 30 seconds for the menu to pop up.  Switching from one desktop to
> > another can take several seconds, sometimes 8 or 10.  This rig is
> > getting slower.

Wut. I am running plasma 6 on a Surface Go 1 whose Pentium Gold was slow 
even when it came out. It is half as fast as your 8350 and does not have 
such problems.
Benchmark FX 8350: https://www.cpubenchmark.net/cpu.php?id=1780
Benchmark Pentium Gold: https://www.cpubenchmark.net/cpu.php?id=3300

You have NVidia, right? Did you try the other graphics driver (i.e. 
proprietary ←→ foss)? Do those delays disappear if you disable 3D effects 
with Shift+Alt+F12?


> That sounds like RAM but I couldn't say for sure.  In any case a
> modern system will definitely help.

Well, is the RAM full? My 10 years old PC has 32 Gigs and still runs very 
smooth (with Intel integrated graphics).

> > Given the new rig can have 128GBs, I assume it comes in 32GB sticks.
> 
> Consumer DDR5 seems to come as large as 48GB, though that seems like
> an odd size.

Actually, my product search page finds sticks with up to 96 GB. I believe 
the 48 size was introduced because for those to whom 32 was too small, 64 
was too expensive. DDR5 still is relatively pricey due to its higher 
electrical requirements.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

It’s quiet in the shadow, because you can’t hear the light.


signature.asc
Description: PGP signature

Re: [gentoo-user] NAS box and switching from Phenom II X6 1090T to FX-6300

[Frank Steinmetzger]

Am Mon, Apr 15, 2024 at 08:04:15AM -0500 schrieb Dale:

> > The physical connector is called M.2. The dimensions of the “sticks” are 
> > given in a number such as 2280, meaning 22 mm wide and 80 mm long. There 
> > are 
> > different lengths available from 30 to 110 mm. M.2 has different “keys”, 
> > meaning there are several variants of electrical hookup. Depending on that, 
> > it can support SATA, PCIe, or both. NVMe is a protocol that usually runs 
> > via 
> > PCIe. So for a modern setup, one usually buys NVMe drives, meaning they are 
> > connected via PCIe either directly to the CPU or over the chipset.
> >
> 
> 
> Ahh, that's why some of them look a little different.  I was wondering
> about that.  Keep in mind, I've never seen one in real life.  Just
> pictures or videos, or people talking about them on this list. 

I use one in my 10-year-old PC. The board only provides PCIe 2.0×2 to the 
slot, so I only get around 1 GB/s instead of 3 which the SSD can reach. But 
I bought the SSD with the intention of keeping it in the next build and I 
don’t notice the difference anyways.

> > There is also the other way around that: an adapter card for the M.2 slot 
> > that gives you SATA ports.
> >
> 
> I didn't know that.

I actually thought we mentioned it already in an earlier “NAS thingy” 
thread. :)

https://www.reddit.com/r/selfhosted/comments/s0bf1d/m2_sata_expansion_anyone_use_something_like_this/
https://www.amazon.de/dp/B09FZDQ6ZB
Maybe you find something if you search for the controller chip (PCIe to 
SATA): JMB585. From what I’ve just read though, the cheap chines adapters 
don’t seem to be very sturdy. One person advised to put an adapter M.2 → 
normal PCIe into the M.2 and then use a normal-formfactor controller card. 
After all, an M.2 slot is just a PCIe×4 slot with a different connector.

BTW: there are also NVMe SSDs in the old 2.5″ format. This formfactor is 
called U.2, but beware the enterprise-level prices.

> I've seen some server type mobos that have SAS connectors which gives
> several options.  Some of them tend to have more PCIe slots which some
> regular mobos don't anymore.  Then there is that ECC memory as well.  If
> the memory doesn't cost to much more, I could go that route.  I'm not
> sure how much I would benefit from it but data corruption is a thing to
> be concerned about. 
> […]
> The problem with those cards, some of the newer mobos don't have as many
> PCIe slots to put those cards into anymore.  I think I currently have
> two such cards in my current rig.  The new rig would hold almost twice
> the number of drives.  Obviously, I'd need cards with more SATA ports. 

Indeed consumer boards tend to get fewer normal PCIe slots. Filtering for 
AM4 boards, the filter allowed me to filter up to 6 slots, whereas for AM5 
boards, the filter stopped at 4 slots.
AM4: https://skinflint.co.uk/?cat=mbam4=18869_5%7E20502_UECCDIMM%7E4400_ATX
AM5: https://skinflint.co.uk/?cat=mbam5=18869_4%7E20502_UECCDIMM%7E4400_ATX

> One reason I'm trying not to move to fast right now, besides trying to
> save up money, I'm trying to find the right CPU, mobo and memory combo. 
> None of them are cheap anymore.  Just the CPU is going to be around
> $400.  The mobo isn't to far behind if I go with a non server one. 

One popular choice for home servers is AM4’s Ryzen Pro 4650G. That’s an APU 
(so with powerful internal graphics), but also with ECC support (hence the 
Pro moniker). The APU is popular because 1) on AM4 only APUs have graphics 
at all, 2) it allows for use as a compact media server, as no bulky GPU is 
needed.

Speaking of GPU: We’ve had the topic before, but keep in mind that if you go 
with AM5, you don’t need a dGPU. Unless you go with one of those F 
processors. So there is one more slot available.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

Order is one half of your life, but the other half is nicer.


signature.asc
Description: PGP signature

Re: [gentoo-user] Re: [gentoo-dev] Current unavoidable use of xz utils in Gentoo

[Frank Steinmetzger]

Am Sun, Mar 31, 2024 at 08:33:20AM -0400 schrieb Rich Freeman:
> (moving this to gentoo-user as this is really getting off-topic for -dev)
> […]
> We're going on almost 20 years since the Snowden revelations, and back
> then the NSA was basically doing intrusion on an industrial scale.

Weeaalll, it’s been 11 years in fact. Considering that is more than 10 
years, one could argue it is approaching 20. ;-)

I can remember the year well because Snowden is the same vintage as I am and 
he became 30 about when this all came out.

-- 
Grüße | Greetings | Salut | Qapla’
Others make mistakes, too -- but we have the most experience in it.


signature.asc
Description: PGP signature

Re: [gentoo-user] NAS box and switching from Phenom II X6 1090T to FX-6300

[Frank Steinmetzger]

Am Sat, Apr 13, 2024 at 08:23:27AM -0500 schrieb Dale:
> Rich Freeman wrote:
> > On Sat, Apr 13, 2024 at 8:11 AM Dale  wrote:
> >> My biggest thing right now, finding a mobo with plenty of PCIe slots.
> >> They put all this new stuff, wifi and such, but remove things I do need,
> >> PCIe slots.
> > PCIe and memory capacity seem to have become the way the
> > server/workstation and consumer markets are segmented.
> >
> > AM5 gets you 28x v5 lanes.  SP5 gets you 128x v5 lanes.  The server
> > socket also has way more memory capacity, though I couldn't quickly
> > identify exactly how much more due to the ambiguous way in which DDR5
> > memory channels are referenced all over the place.  Suffice it to say
> > you can put several times as many DIMMs into a typical server
> > motherboard, especially if you have two CPUs on it (two CPUs likewise
> > increases the PCIe capacity).
> 
> I see lots of mobos with those little hard drives on a stick.  I think
> they called NVME or something, may have spelling wrong.

The physical connector is called M.2. The dimensions of the “sticks” are 
given in a number such as 2280, meaning 22 mm wide and 80 mm long. There are 
different lengths available from 30 to 110 mm. M.2 has different “keys”, 
meaning there are several variants of electrical hookup. Depending on that, 
it can support SATA, PCIe, or both. NVMe is a protocol that usually runs via 
PCIe. So for a modern setup, one usually buys NVMe drives, meaning they are 
connected via PCIe either directly to the CPU or over the chipset.

> For most
> people, that is likely awesome.  For me, I think I'd be happy with a
> regular SSD.  Given that, I'd like them to make a mobo where one can say
> cut off/disable that NVME thing and make use of that "lane" as a PCIe
> slot(s).  Even if that means having a cable that hooks to the mobo and
> runs elsewhere to connect PCIe cards. In other words, have one slot
> that is expandable to say three or four slots with what I think is
> called a back-plane.

There is also the other way around that: an adapter card for the M.2 slot 
that gives you SATA ports.

> I have considered getting a server type mobo and CPU for my new build. 

The only reason I got a server board for my little 4-slot NAS is to get ECC 
support. (Plus you don’t get non-server Mini-ITX with more than four SATAs). 
But it runs the smallest i3 I could get. It’s a NAS, not a workstation. It 
serves files, nothing more. I don’t mind if updates take longer than on a 
Desktop, which is why I don’t see a point in speccing it out to the top 
CPU-wise. This only adds cost to acquisition and upkeep.

I just did the profile switch to 23, and it rebuilt 685 packages in a little 
over six hours, plus 1½ hours for gcc beforehand.

> As you point out, they are packed with features I could likely use. 

“Could likely”? Which features exactly? As you say yourself:

> Thing is, the price tag makes me faint and fall out of my chair.  Even
> used ones that are a couple years old, in the floor I go.  -_-  I looked
> up a SP5 AMD CPU, pushing $800 just for the CPU on Ebay, used.  The mobo
> isn't cheap either.  I don't know if that would even serve my purpose. 

Exactly. Those boards and CPUs are made to run servers that serve entire 
SMBs so that the employees can work on stuff at the same time. As a one-man 
entity, I don’t expect you’ll ever really need that raw power. If it’s just 
for SATA ports, you can get controller cards for those.

> The biggest thing I need PCIe slots for, drive controllers.  I thought
> about buying a SAS card and having it branch out into a LOT of drives. 
> Still, I might need two cards even then. 

But it would be the most logical choice.

> It's like looking at the cereal isle in a store.  All those choices and
> most of them . . . . are corn.  ROFL 

Nice one.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

I’ve been using vi for 15 years, because I don’t know with which command
to close it.


signature.asc
Description: PGP signature

Re: [gentoo-user] How to synchronise between 2 locations

[Frank Steinmetzger]

Am Thu, Mar 28, 2024 at 09:26:33PM -0500 schrieb Grant Taylor:
> On 3/27/24 13:58, J. Roeleveld wrote:
> > Hi all,
> 
> Hi,
> 
> > I am looking for a way to synchronise a filesystem between 2 servers. 
> > Changes
> > can occur on both sides which means I need to have it synchronise in both
> > directions.
> 
> What sort of turn around time are you looking for?  seconds, minus, hours,
> longer?
> 
> > Does anyone have any thoughts on this?
> 
> I would wonder about using rsync.

Rsync can’t handle file moves. Given:

> > Also, both servers are connected using a slow VPN link, which is why I can't
> > simply access files on the remote server.

it would be beneficial to conserve traffic as much as possible.

While you can use the -u flag to only overwrite if the source is newer than 
the destination, AFAIK rsync can’t detect if the destination has also been 
altered since the last sync, so it might clobber important changes. That’s 
why sync tools use a metadata cache to remember last edit timestamps.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

In the 60's people took acid to make the world weird.
Now the world is weird and people take Prozac to make it normal.


signature.asc
Description: PGP signature

Re: [gentoo-user] How to synchronise between 2 locations

[Frank Steinmetzger]

Am Thu, Mar 28, 2024 at 05:33:43PM +0100 schrieb ralfconn:
> Il 28/03/24 07:30, J. Roeleveld ha scritto:
> > > Unison creates a local index of all files it syncronised. So when you 
> > > move a
> > > file around on one end, Unison will notice that because the file at the 
> > > new
> > > location has the same hash as the file at the old location. As a result, 
> > > it
> > > does not transmit the file anew to the remote host, but instead copies it
> > > locally on the remote host.
> > > 
> > > Since Unison uses ssh underneath, you can use ssh’s transparent 
> > > compression
> > > to speed up the transfer.
> > Unison sounds interesting. How does it handle conflicts (eg, file is 
> > changed on
> > both sides?)
> > 
> I use Unison GUI on one of the two machines (on the other peer it's just a
> program invoked from the ssh). When the analysis is complete, the GUI shows
> what it would do to sync the machines, indicating the conflicts and giving
> you the chance to choose what to do.
> 
> I believe it can be used from the command line or maybe even in batch mode
> instead of GUI but I never did it that way.

You can set up a merge command to solve conflicts on the cmdline, such as 
vimdiff. But when I set that, it blocks the GUI. Maybe I did something wrong 
with the setup. Anyways, when I get a conflict, I make a backup of the file 
locally, overwrite it with the remote and then do a conflict resolution with 
vim.

In my every-day workflow, I usually only get conflicts in text files (logs, 
notes, and so on). Binary conflicts are rare and usually due to recent 
actions, such as editing an image or music file. In that case I can decide 
on a per-case-basis which version to keep.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

How does the Heisenberg compensator work? – Thank you, fine.


signature.asc
Description: PGP signature

Re: [gentoo-user] How to synchronise between 2 locations

[Frank Steinmetzger]

Am Thu, Mar 28, 2024 at 01:08:03AM +0100 schrieb Alarig Le Lay:
> On Wed 27 Mar 2024 20:37:27 GMT, Frank Steinmetzger wrote:
> > +1 for Unison. I’ve been using it for many years now to synchronise between 
> > the four PC systems in my household.
> > 
> > Unison creates a local index of all files it syncronised. So when you move 
> > a 
> > file around on one end, Unison will notice that because the file at the new 
> > location has the same hash as the file at the old location. As a result, it 
> > does not transmit the file anew to the remote host, but instead copies it 
> > locally on the remote host.
> > 
> > Since Unison uses ssh underneath, you can use ssh’s transparent compression 
> > to speed up the transfer.
> 
> I’ve been thinking about using it to synchronise dovecot maildir folders,
> since dsync is now deprecated. But I’m not sure about it as I never used
> it under “heavy” loads. Do you have any thoughts about it?

Among the many files I sync is my Mail folder. It currently contains around
106k files. My music collection is another 50k files. If there are no 
changes and I did a recent run, then another run to look for changes takes 
only very few seconds.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

A computer only crashes if your haven’t saved your text for a long time.


signature.asc
Description: PGP signature

Re: [gentoo-user] How to synchronise between 2 locations

[Frank Steinmetzger]

Am Wed, Mar 27, 2024 at 03:59:18PM -0400 schrieb Matt Connell:

> > Syncthing is also a good idea. The major difference: syncthing is a 
> > permanently running daemon, so changes are synced very fast (the
> > interval is configurable, IIRC). OTOH, Unison is run individually by
> > you. That’s why I prefer the latter: in case I broke some file on my
> > machine, I can get it back from another machine without having to
> > break out the backup disk (which may not even have what I need
> > because my backup interval is too big).
> 
> Good point.  I mainly use syncthing as a "stuff I need on multiple
> machines" bucket, rather than a big directory of active working files.

I sync most of my files on the home and media partition. So all my 
documents, photos and music library. I do use syncthing -- between my 
android phone and PC. Because I don’t like to fiddle with the filesystem on 
the very constrained UI of touch devices.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

Can you give me a cigarette?  Mine are still in the vending machine.


signature.asc
Description: PGP signature

Re: [gentoo-user] How to synchronise between 2 locations

[Frank Steinmetzger]

Am Wed, Mar 27, 2024 at 03:42:07PM -0400 schrieb Matt Connell:
> On Wed, 2024-03-27 at 19:58 +0100, J. Roeleveld wrote:
> > Hi all,
> > 
> > I am looking for a way to synchronise a filesystem between 2 servers.
> > Changes can occur on both sides which means I need to have it
> > synchronise in both directions.
> > 
> > Does anyone have any thoughts on this?
> > 
> > Also, both servers are connected using a slow VPN link, which is why
> > I can't simply access files on the remote server.
> 
> I've been using syncthing for years and am extremely pleased with it. 
> It works so well that I sometimes forget that its there, truly in the
> It Just Works category of software.

Syncthing is also a good idea. The major difference: syncthing is a 
permanently running daemon, so changes are synced very fast (the interval is 
configurable, IIRC). OTOH, Unison is run individually by you. That’s why I 
prefer the latter: in case I broke some file on my machine, I can get it 
back from another machine without having to break out the backup disk (which 
may not even have what I need because my backup interval is too big).

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

I have seen things you lusers would not believe.
I've seen Sun monitors on fire off the side of the multimedia lab.
I've seen NTU lights glitter in the dark near the Mail Gate.
All these things will be lost in time, like the root partition last week.
Time to die.


signature.asc
Description: PGP signature

Re: [gentoo-user] How to synchronise between 2 locations

[Frank Steinmetzger]

Am Wed, Mar 27, 2024 at 08:18:14PM +0100 schrieb ralfconn:
> Il 27/03/24 19:58, J. Roeleveld ha scritto:
> > Hi all,
> > 
> > I am looking for a way to synchronise a filesystem between 2 servers. 
> > Changes
> > can occur on both sides which means I need to have it synchronise in both
> > directions.
> > 
> > Does anyone have any thoughts on this?
> > 
> > Also, both servers are connected using a slow VPN link, which is why I can't
> > simply access files on the remote server.


+1 for Unison. I’ve been using it for many years now to synchronise between 
the four PC systems in my household.

> I use it just for that but can't say anything about the VPN bit, my
> servers are on local network.

Unison creates a local index of all files it syncronised. So when you move a 
file around on one end, Unison will notice that because the file at the new 
location has the same hash as the file at the old location. As a result, it 
does not transmit the file anew to the remote host, but instead copies it 
locally on the remote host.

Since Unison uses ssh underneath, you can use ssh’s transparent compression 
to speed up the transfer.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

If all people were the same, one would basically suffice.


signature.asc
Description: PGP signature

Re: [gentoo-user] How do I zap a specific area of a gnumeric spreadsheet page?

[Frank Steinmetzger]

Am Thu, Mar 21, 2024 at 05:46:31PM -0400 schrieb Walter Dnes:

>   The province of Ontario does weekly Covid data updates which I
> summarize and post on the DSLReports Canchat subforum, e.g.
> https://www.dslreports.com/forum/r33854514-#google_vignette  Note the
> data gap in the pink and brown lines on the 3rd and 4th graphs.  That's
> actual missing data.  In the underlying spreadsheet page those gaps
> initially show up as zeros.  I manually blank out region B1258:C1299
> (i.e. 2023/09/09 to 2023/10/20) every week when I update so that it
> doesn't show up as zero hospitalizations.  How do I set up and execute a
> macro to to zap the contents of region B1258:C1299 on a page?

Why not make the alteration one step before -- in the CSV?
There are CSV abstraction tools like `q`, which gives you a SQL-like 
interface to a csv file. Or you could write a quick transformer in python, 
if you know the language a bit.

Pseudo code, as I haven’t worked with csv in Python in a looong time:

import csv
with csv.open("input file", 'r') as A:
with csv.open("output file", 'w') as B:
for rownum, row in enumerate(A):
if rownum >= 1258 or rownum <= 1299:
# write a modified row which has columns B and C blanked
B.write( [row[0] + ['', ''] + row[3:] )
else:
B.write(row)

>   Note that I have to first remove the previous week's file, because wget
> won't overwrite it, and skips the download altogether.

Maybe remove the -r from rm, just to peace of mind. Also, such minimalist 
scripts that don’t use bash features can be sh scripts instead. This 
increases performance, as sh loads faster than bash. ;-)

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

It is not enough to have no opinion, you must also be unable to express it.


signature.asc
Description: PGP signature

Re: [gentoo-user] Suggestions for backup scheme?

[Frank Steinmetzger]

Am Tue, Jan 30, 2024 at 06:15:09PM - schrieb Grant Edwards:
> I need to set up some sort of automated backup on a couple Gentoo
> machines (typical desktop software development and home use). One of
> them used rsnapshot in the past but the crontab entries that drove
> that have vanished :/ (presumably during a reinstall or upgrade --
> IIRC, it took a fair bit of trial and error to get the crontab entries
> figured out).
> 
> I believe rsnapshot ran nightly and kept daily snapshots for a week,
> weekly snapshots for a month, and monthly snapshots for a couple
> years.
> 
> Are there other backup solutions that people would like to suggest I
> look at to replace rsnapshot?  I was happy enough with rsnapshot (when
> it was running), but perhaps there's something else I should consider?

In my early backup times I, too, used rsnapshot to back up my ~ and rsync 
for my big media files. But that only included my PC. My laptop was wholly 
un-backed-up. I only syncronised much of my home and my audio collection 
between the two with unison. At some point my external 3 TB drive became 
free and then I started using borg to finally do proper backups.

Borg is very similar to restic, I actually used the two in parallel for a 
while to compare them, but stayed with borg. One pain point was that I 
couln’t switch off restic’s own password protection. Since all my backup 
disks are LUKSed anyway, I don’t need that.

Since borg works block-based, it does deduplication without extra cost and 
it is suitable for big image files which don’t change much. I do full 
filesystem backups of /, ~ and my media partition of my main PC and my 
laptop. I have one repository for each of those three filesystems, and each 
repo receives the data from both machines, so they are deduped. Since both 
machines run Arch, their roots are binary identical. The same goes for my 
unison-synced homes.

Borg has retention logic built-in. You can say I want to keep the latest 
archive of each of the last 6 days/weeks/months/years, and it even goes down 
to seconds. And of course you can combine those rules. The only thing is 
they don’t overlap, meaning if you want to keep the last 14 days and the 
last four weeks, those weekly retentions start after the last daily 
snapshots.

In summary, advantages:
+ fast dedup, built-in compression (different algos and levels configurable)
+ big data files allow for quick mirroring of repositories.
  I simply rsync my primary backup disk to two other external HDDs.
+ Incremental backups are quite fast because borg uses a cache to detect
  changed files quickly.
Disadvantages:
- you need borg to mount the backups it
- it is not as fast as native disk access, especially during restore and 
  when getting a total file listing due to lots of random I/O on the HDD.


As example, I currently have 63 snapshots in my data partition repository:

# borg list data/
tp_2021-06-07   Mon, 2021-06-07 16:27:44 
[5f9ebd9f24353c340691b2a71f5228985a41699d2e23473ae4e9e795669c8440]
kern_2021-06-07 Mon, 2021-06-07 23:58:56 
[19c76211a9c35432e6a66ac1892ee19a08368af28d2d621f509af3d45f203d43]
[... 55 more lines ...]
kern_2024-01-14 Sun, 2024-01-14 20:53:23 
[499ce7629e64cffb7ec6ec9ffbf0c595e4ede3d93f131a9a4b424b165647f645]
tp_2024-01-14   Sun, 2024-01-14 20:57:42 
[ea2baef3e4bb49c5aec7cf8536f7b00b55fb27ecae3a80ef9f5a5686a1da30d5]
kern_2024-01-21 Sun, 2024-01-21 23:42:46 
[71aa2ce6cf4021712f949af068498bfda7797b5d1c5ddc0f0ce8862b89e48961]
tp_2024-01-21   Sun, 2024-01-21 23:48:24 
[45e35ed9206078667fa62d0e4a1ac213e77f52415f196101d14ee21e79fc393d]
kern_2024-02-04 Sun, 2024-02-04 23:16:43 
[e1b015117143fad6b89cea66329faa888cffc990644e157b1d25846220c62448]
tp_2024-02-04   Sun, 2024-02-04 23:23:15 
[e9b167ceec1ab9a80cbdb1acf4ff31cd3935fc23e81674cad1b8694d98547aeb]

The last “tp” (Thinkpad) snapshot contains 1 TB, “kern” (my PC) 809 GB.
And here you see how much space this actually takes on disk:

# borg info data/
[ ... ]
 Original size   Compressed sizeDeduplicated size
All archives: 56.16 TB  54.69 TB  1.35 TB

Obviously, compression doesn’t do much for media files. But it is very 
effective in the repository for the root partitions:

# borg info arch-root/
[ ... ]
 Original size   Compressed sizeDeduplicated size
All archives:  1.38 TB 577.58 GB 79.41 GB

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

“She understands.  She doesn’t comprehend.” – River Tam, Firefly


signature.asc
Description: PGP signature

Re: [gentoo-user] rsync options after backup restore. Transfer speed again.

[Frank Steinmetzger]

Am Mon, Oct 23, 2023 at 02:29:26AM -0500 schrieb Dale:
> Dale wrote:
> >
> > Second problem.  The transfer speed is back to the old slower speed. 
> > I'm pretty sure I am using the same old options on both ends.  Still,
> > it's back to being slow again.  Some info:
> >
> >
> > <<< SNIP >>>
> > Did I miss something?  Typo maybe?  I'm pretty sure I used copy and
> > paste but still. 
> >
> > Thanks.
> >
> > Dale
> >
> > :-)  :-) 
> 
> 
> I been working on the speed problem again.  I rebuilt the kernel on
> fireball and I think some changes made a huge change.  This is the
> results from fireball now:
> 
> 
> root@fireball / # cryptsetup benchmark
> # Tests are approximate using memory only (no storage IO).
> PBKDF2-sha1   931239 iterations per second for 256-bit key
> PBKDF2-sha256    1356501 iterations per second for 256-bit key
> PBKDF2-sha512 972705 iterations per second for 256-bit key
> PBKDF2-ripemd160  648871 iterations per second for 256-bit key
> PBKDF2-whirlpool  362077 iterations per second for 256-bit key
> argon2i   5 iterations, 1048576 memory, 4 parallel threads (CPUs)
> for 256-bit key (requested 2000 ms time)
> argon2id  4 iterations, 1048576 memory, 4 parallel threads (CPUs)
> for 256-bit key (requested 2000 ms time)
> # Algorithm |   Key |  Encryption |  Decryption
>     aes-cbc    128b   570.8 MiB/s  2045.6 MiB/s
>     serpent-cbc    128b    91.1 MiB/s   310.0 MiB/s
>     twofish-cbc    128b   198.7 MiB/s   218.9 MiB/s
>     aes-cbc    256b   428.8 MiB/s  1670.4 MiB/s
>     serpent-cbc    256b    91.6 MiB/s   309.5 MiB/s
>     twofish-cbc    256b   199.8 MiB/s   219.2 MiB/s
>     aes-xts    256b  1821.2 MiB/s  1767.1 MiB/s
>     serpent-xts    256b   265.9 MiB/s   270.2 MiB/s
>     twofish-xts    256b   201.0 MiB/s   204.2 MiB/s
>     aes-xts    512b  1440.0 MiB/s  1445.9 MiB/s
>     serpent-xts    512b   265.0 MiB/s   257.2 MiB/s
>     twofish-xts    512b   198.2 MiB/s   201.6 MiB/s
> root@fireball / #

There you go. Told ya. :)

> As you can see, aes-cbc is fast now and I think that is what cryptsetup
> uses.  It used to be really slow I think. 

Cryptsetup uses aes-xts these days, I think it’s been mentioned in this 
thread somewhere.

> Now on to the nas box.  I've recompiled the kernel with some added
> options.  Still, it refuses to speed up.  I kinda think it is the CPU
> lacking support for encryption.  I'm asking others just in case I'm
> missing something.  Also, fireball uses a older kernel, 5.14 or so.  The
> nas box uses 6.1 or so.  The menus are different and that is why it is
> hard to get them to match up.  I may have missed something.

Everything you need for that is in the crypto menu at the bottom.

> This is the bench mark from nas box. 
> 
> nas ~ # cryptsetup benchmark
> # Tests are approximate using memory only (no storage IO).
> PBKDF2-sha1   700919 iterations per second for 256-bit key
> PBKDF2-sha256 924670 iterations per second for 256-bit key
> PBKDF2-sha512 729190 iterations per second for 256-bit key
> PBKDF2-ripemd160  517559 iterations per second for 256-bit key
> PBKDF2-whirlpool  359593 iterations per second for 256-bit key
> argon2i   4 iterations, 1048576 memory, 4 parallel threads (CPUs)
> for 256-bit key (requested 2000 ms time)
> argon2id  4 iterations, 1048576 memory, 4 parallel threads (CPUs)
> for 256-bit key (requested 2000 ms time)
> # Algorithm |   Key |  Encryption |  Decryption
>     aes-cbc    128b    63.6 MiB/s    41.6 MiB/s
>     serpent-cbc    128b    81.0 MiB/s   212.4 MiB/s
>     twofish-cbc    128b   192.5 MiB/s   222.1 MiB/s
>     aes-cbc    256b    47.5 MiB/s    30.0 MiB/s
>     serpent-cbc    256b    81.2 MiB/s   212.7 MiB/s
>     twofish-cbc    256b   192.3 MiB/s   221.9 MiB/s
>     aes-xts    256b    65.9 MiB/s    41.6 MiB/s
>     serpent-xts    256b   201.7 MiB/s   205.7 MiB/s
>     twofish-xts    256b   216.2 MiB/s   214.5 MiB/s
>     aes-xts    512b    48.8 MiB/s    30.0 MiB/s
>     serpent-xts    512b   202.7 MiB/s   205.6 MiB/s
>     twofish-xts    512b   216.4 MiB/s   214.0 MiB/s
> nas ~ #
> […]
> The aes shows up on fireball.  It does not on the nas box.  Is the speed
> above as good as I can expect with this older CPU?

If not done yet, you can check whether you enabled the 64 bit versions of 
the crypto modules. They could push performance by a few more percent.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

The last chance is often the second-last, if you look close enough.


signature.asc
Description: PGP signature

Re: [gentoo-user] rsync options after backup restore. Transfer speed again.

[Frank Steinmetzger]

Am Fri, Oct 20, 2023 at 09:20:45PM -0500 schrieb Dale:
> Howdy,
> 
> As most know, I had to restore from backups recently.  I also reworked
> my NAS box.  I'm doing my first backup given that I have more files that
> need to be added to the backups.  When I started the rsync, it's
> starting from the first file and updating each file as it goes as if all
> of them changed.  Given that likely 95% of the files hasn't changed, I
> figure this is being done because of a time stamp or something.  Is
> there a way to tell rsync to ignore the time stamp or something or if
> the files are the same size, just update the time stamp?  Is there a way
> to just update the time stamps on the NAS box?  Is there a option I
> haven't thought of to work around this? 
> 
> This is the old command I was using to create the backups.
> 
> time rsync -uivr --progress --delete /home/dale/Desktop/Crypt/TV_Series
> /mnt/TV_Backup/

This didn’t preserve timestamps. Hence there is one type of information lost 
from which rsync knows whether two files may be identical. So now your 
restore has more recent timestamps the the backup. If you use -u, Rsync 
should skip all files.

My perfectionist self doesn’t like discarding timestamp information, because 
then my system can’t tell me how old some file is, and how old (or young) I 
was when I created it and so on. I once didn’t pay enough attention when 
restoring a backup back when I was still on Windows, which is why I don’t 
have many files left that are dated before April 2007, even though they are 
from 2000 ± x.

BTW: -i and -v are redundant. -v will only print the file path, whereas -i 
does the same and adds the reasons colum at the beginning.

> I tried these to try to get around it.
> 
> time rsync -ar --progress --delete /home/dale/Desktop/Crypt/TV_Series 
> /mnt/TV_Backup/

-a and -r are also redundant, as -a includes -r.

> I looked at the man page and the options there.  I don't see anything
> that I think will help.  Is there a way around this? 

My muscle memory uses `rsync -ai` for almost everything. And when I do full 
root file systems or stuff where I know I will need them, I use -axAHX 
instead. Since this preserves all the usual data, I’ve never really had 
rsync wanting to do everything all over.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

Arrogance is the art of being proud of one’s own stupidity.


signature.asc
Description: PGP signature

Re: [gentoo-user] Getting output of a program running in background after a crash

[Frank Steinmetzger]

Am Thu, Oct 12, 2023 at 08:35:21PM -0500 schrieb Dale:

> Frank Steinmetzger wrote:
> > Am Thu, Oct 12, 2023 at 10:44:39PM +0100 schrieb Michael:
> >
> >> Why don't you test throughput without encryption to confirm your 
> >> assumption?
> > What does `cryptsetup benchmark` say? I used to use a Celeron G1840 in my 
> > NAS, which is Intel Haswell without AES_NI. It was able to do ~ 150 MB/s 
> > raw 
> > encryption throughput when transferring to or from a LUKS’ed image in a 
> > ramdisk, so almost 150 % of gigabit ethernet speed.
> […]
> I've never used that benchmark.  Didn't know it exists.  This is the
> results.  Keep in mind, fireball is my main rig.  The FX-8350 thingy. 
> The NAS is currently the old 770T system.  Sometimes it is a old Dell
> Inspiron but not this time.  ;-)
> 
> root@fireball / # cryptsetup benchmark
> […]
> # Algorithm |   Key |  Encryption |  Decryption
>     aes-cbc    128b    63.8 MiB/s    51.4 MiB/s
>     serpent-cbc    128b    90.9 MiB/s   307.6 MiB/s
>     twofish-cbc    128b   200.4 MiB/s   218.4 MiB/s
>     aes-cbc    256b    54.6 MiB/s    37.5 MiB/s
>     serpent-cbc    256b    90.4 MiB/s   302.6 MiB/s
>     twofish-cbc    256b   198.2 MiB/s   216.7 MiB/s
>     aes-xts    256b    68.0 MiB/s    45.0 MiB/s
>     serpent-xts    256b   231.9 MiB/s   227.6 MiB/s
>     twofish-xts    256b   191.8 MiB/s   163.1 MiB/s
>     aes-xts    512b    42.4 MiB/s    18.9 MiB/s
>     serpent-xts    512b   100.9 MiB/s   124.6 MiB/s
>     twofish-xts    512b   154.8 MiB/s   173.3 MiB/s
> root@fireball / #

Phew, this looks vry slow. As you can clearly see, this is not enough to 
even saturate Gbit ethernet. Unfortunately, I don’t have any benchmark data 
left over from the mentioned celeron.
(Perhaps that’s why the industry chose to implement AES in hardware, because 
it was the slowest of the bunch.)

It looks like there is no hardware acceleration involved. But according to 
https://en.wikipedia.org/wiki/List_of_AMD_FX_processors#Piledriver-based and 
https://www.cpu-world.com/CPUs/Bulldozer/AMD-FX-Series%20FX-8350.html it has 
the extension. I’d say something is amiss in your kernel.

Heck, even my ultra-low-end eeepc with its no-AES Atom processor N450 from 
2009 is less than 50 % slower, and for aes-xts 512b it is actually faster! 
And that was a snail even in its day. It is so low-end that its in-order 
architecture is not vulnerable to spectre and meltdown. :D It just scrunched 
several minutes on updating the GPG keyring of its arch linux installation.

eeePC # LC_ALL=C cryptsetup benchmark
# Tests are approximate using memory only (no storage IO).
PBKDF2-sha1   228348 iterations per second for 256-bit key
PBKDF2-sha256 335222 iterations per second for 256-bit key
PBKDF2-sha512 253034 iterations per second for 256-bit key
PBKDF2-ripemd160  172690 iterations per second for 256-bit key
PBKDF2-whirlpool   94705 iterations per second for 256-bit key
argon2i   4 iterations, 71003 memory, 4 parallel threads (CPUs) for 256-bit 
key (requested 2000 ms time)
argon2id  4 iterations, 71506 memory, 4 parallel threads (CPUs) for 256-bit 
key (requested 2000 ms time)
# Algorithm |   Key |  Encryption |  Decryption
aes-cbc128b31.0 MiB/s33.6 MiB/s
serpent-cbc128b28.1 MiB/s62.9 MiB/s
twofish-cbc128b28.6 MiB/s31.0 MiB/s
aes-cbc256b24.0 MiB/s25.6 MiB/s
serpent-cbc256b28.3 MiB/s62.7 MiB/s
twofish-cbc256b28.6 MiB/s31.0 MiB/s
aes-xts256b32.5 MiB/s33.4 MiB/s
serpent-xts256b50.5 MiB/s60.5 MiB/s
twofish-xts256b25.6 MiB/s30.7 MiB/s
aes-xts512b25.0 MiB/s25.6 MiB/s
serpent-xts512b60.2 MiB/s60.4 MiB/s
twofish-xts512b30.2 MiB/s30.7 MiB/s

> root@nas:~# cryptsetup benchmark
> […]
> # Algorithm |   Key |  Encryption |  Decryption
>     aes-cbc    128b   130.6 MiB/s   128.0 MiB/s
>     serpent-cbc    128b    64.7 MiB/s   161.8 MiB/s
>     twofish-cbc    128b   175.4 MiB/s   218.8 MiB/s
>     aes-cbc    256b   120.1 MiB/s   122.2 MiB/s
>     serpent-cbc    256b    84.5 MiB/s   210.8 MiB/s
>     twofish-cbc    256b   189.5 MiB/s   218.6 MiB/s
>     aes-xts    256b   167.0 MiB/s   162.1 MiB/s
>     serpent-xts    256b   173.9 MiB/s   204.5 MiB/s
>     twofish-xts    256b   

Re: [gentoo-user] Getting output of a program running in background after a crash

[Frank Steinmetzger]

Am Thu, Oct 12, 2023 at 10:44:39PM +0100 schrieb Michael:

> >  It only does this when I'm copying files over.  Right now I'm copying
> >  about 26TBs of data over ethernet and it is taking a while.  Once I
> >  stop it or it finishes the copy, the CPU goes to about nothing,
> >  unless I'm doing something else.  So it has something to do with the
> >  copy process.
> > >>> 
> > >>> Or the network. What are you using to copy? If you use rsync, you can
> > >>> make use the the --bwlimit option to reduce the speed and network
> > >>> load.
> > >> 
> > >> Reduce?  I wouldn't complain if it went faster.  I think it is about as
> > >> fast as it is going to get tho.
> > > 
> > > And that may be contributing to the CPU usage. Slowing down the flow may
> > > make the comouter more usable, and you're never going to copy 26TB
> > > quickly, especially over ethernet.
> > > 
> > >> While I'm not sure what is keeping me from copying as fast as the drives
> > >> themselves can go, I suspect it is the encryption.
> 
> Why don't you test throughput without encryption to confirm your assumption?

What does `cryptsetup benchmark` say? I used to use a Celeron G1840 in my 
NAS, which is Intel Haswell without AES_NI. It was able to do ~ 150 MB/s raw 
encryption throughput when transferring to or from a LUKS’ed image in a 
ramdisk, so almost 150 % of gigabit ethernet speed.

> > > If you're copying over the network, that will be the limiting factor.
> > 
> > Someone posted some extra options to mount with and add to exports
> > file.

Ah right, you use NFS. If not, I’d have suggested not to use rsync over ssh, 
because that would indeed introduce a lot of encryption overhead.

> > I still think encryption is slowing it down some.  As you say tho,
> > ethernet isn't helping which is why I may look into other options later,
> > faster ethernet or fiber if I can find something cheap enough. 
> 
> There are a lot of hypotheses in your statements, but not much testing to 
> prove or disprove any of them.
> 
> Why don't you try to isolate the cause by testing one system element at a 
> time 
> and see what results you get.
> […]
> Unless you're running Pentium 4 or some other old CPU, it is almost certain 
> your CPU is capable of using AES-NI to offload to hardware some/all of the 
> encryption/decryption load - as long as you have the crypto module built in 
> your kernel.

The FX-8350 may be old, but it actually does have AES instructions.

Here is my Haswell i5 (only two years younger than the FX) with AES_NI:

~ LC_ALL=C cryptsetup benchmark
# Tests are approximate using memory only (no storage IO).
PBKDF2-sha1  1323959 iterations per second for 256-bit key
PBKDF2-sha2561724631 iterations per second for 256-bit key
PBKDF2-sha5121137284 iterations per second for 256-bit key
PBKDF2-ripemd160  706587 iterations per second for 256-bit key
PBKDF2-whirlpool  510007 iterations per second for 256-bit key
argon2i   7 iterations, 1048576 memory, 4 parallel threads (CPUs) for 
256-bit key (requested 2000 ms time)
argon2id  7 iterations, 1048576 memory, 4 parallel threads (CPUs) for 
256-bit key (requested 2000 ms time)
# Algorithm |   Key |  Encryption |  Decryption
aes-cbc128b   679.8 MiB/s  2787.0 MiB/s
serpent-cbc128b91.4 MiB/s   582.1 MiB/s
twofish-cbc128b   194.9 MiB/s   368.3 MiB/s
aes-cbc256b   502.3 MiB/s  2155.4 MiB/s
serpent-cbc256b90.3 MiB/s   582.5 MiB/s
twofish-cbc256b   194.0 MiB/s   368.6 MiB/s
aes-xts256b  2470.8 MiB/s  2478.7 MiB/s
serpent-xts256b   537.4 MiB/s   526.1 MiB/s
twofish-xts256b   347.3 MiB/s   347.3 MiB/s
aes-xts512b  1932.6 MiB/s  1958.0 MiB/s
serpent-xts512b   532.9 MiB/s   522.9 MiB/s
twofish-xts512b   348.4 MiB/s   348.9 MiB/s

The 6 Watts processor in my Surface Go yields:
aes-xts512b  1122,2 MiB/s  1123,7 MiB/s

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

The severity of the itch is inversely proportional to the reach.


signature.asc
Description: PGP signature

Re: [gentoo-user] Getting output of a program running in background after a crash

[Frank Steinmetzger]

Am Sun, Oct 08, 2023 at 07:44:06PM -0500 schrieb Dale:

> Just as a update.  The file system I was trying to do a file system
> check on was my large one, about 40TBs worth.  While running the file
> system check, it started using HUGE amounts of memory.  It used almost
> all my 32GBs and most of swap as well.  It couldn't finish due to not
> enough memory, it literally crashed itself.  So, I don't know if this is
> because of some huge problem or what but if this is expected behavior,
> don't try to do a file system check on devices that large unless you
> have a LOT of memory. 

Or use a different filesystem. O:-)

> I ended up recreating the LVM devices from scratch and redoing the
> encryption as well.  I have backups tho.  This all started when using
> pvmove to replace a hard drive with a larger drive.  I guess pvmove
> isn't always safe.

I think that may be a far-fetched conclusion. If it weren’t safe, it 
wouldn’t be in the software – or at least not advertised as safe.

> P. S.  I currently have my backup system on my old Gigabyte 770T mobo
> and friends.  It is still a bit slower than copying when no encryption
> is used so I guess encryption does slow things down a bit.  That said,
> the CPU does hang around 50% most of the time.  htop doesn't show what
> is using that so it must be IO or encryption.

You can add more widgets (“meters”) to htop, one of them shows disk 
throughput. But there is none for I/O wait. One tool that does show that is 
glances. And also dstat which I mentioned a few days ago. Not only can dstat 
tell you the total percentage, but also which process is the most expensive 
one.

I set up bash aliases for different use cases of dstat:
alias ,d='dstat --time --cpu --disk -D $(ls /dev/sd? /dev/nvme?n? /dev/mmcblk? 
2>/dev/null | tr "\n" ,) --net --mem --swap'
alias ,dd='dstat --time --cpu --disk --disk-util -D $(ls /dev/sd? /dev/nvme?n? 
/dev/mmcblk? 2>/dev/null | tr "\n" ,) --mem-adv'
alias ,dm='dstat --time --cpu --disk -D $(ls /dev/sd? /dev/nvme?n? /dev/mmcblk? 
2>/dev/null | tr "\n" ,) --net --mem-adv --swap'
alias ,dt='dstat --time --cpu --disk -D $(ls /dev/sd? /dev/nvme?n? /dev/mmcblk? 
2>/dev/null | tr "\n" ,) --net --mem --swap --top-cpu --top-bio --top-io 
--top-mem'

Because I attach external storage once in a while, I use a dynamic list of 
devices to watch that is passed to the -D argument. If I don’t use -D, dstat 
will only show a total for all drives.

The first is a simple overview (d = dstat).

The second is the same but only for disk statistics (dd = dstat disks). I 
use it mostly on my NAS (five SATA drives in total, which creates a very 
wide table).

The third shows more memory details like dirty cache (dm = dstat memory), 
which is interesting when copying large files.

And the last one shows the top “pigs”, i.e. expensive processes in terms of 
CPU, IO and memory (dt = dstat top).

> Or something kernel
> related that htop doesn't show.  No idea. 

Perhaps my tool tips give you ideas. :)

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

What is the difference between two flutes? – A semitone.


signature.asc
Description: PGP signature

Re: [gentoo-user] Switching from desktop to desktop without function keys.

[Frank Steinmetzger]

Am Sun, Oct 01, 2023 at 06:44:09PM -0500 schrieb Dale:
> Frank Steinmetzger wrote:
> > Am Sat, Sep 30, 2023 at 10:47:31PM +0100 schrieb Peter Humphrey:
> >> On Saturday, 30 September 2023 20:59:04 BST Dale wrote:
> >>
> >>> Is there a way with the keyboard to switch to a desktop above 10?  Even
> >>> if it just switches up one at a time, that would work.  Say switch to 10
> >>> and then keep hitting a set of keys to go to 11, then 12, then 13 etc
> >>> etc.  Eventually, I get to the one I want. 
> >> You can set up a key combination to switch one desktop to the right and 
> >> another to switch one to the left. It's under Shortcuts, where you select 
> >> KWin, then scroll the right-hand panel down to Walk through Desktop List, 
> >> where I've put what used to be the default value: CTRL-ALT-right. Then 
> >> down 
> >> one to the next entry and enter CTRL-ALT-left. The defaults are left blank 
> >> nowadays.
> > Also note that a good while ago Plasma switched from using the Alt key to 
> > the Super key for everything Window-managerial. So these days, if you 
> > create 
> > a new user from scratch, it’s Ctrl+Super instead of Ctrl+Alt.
> >
> 
> 
> Looking at my keyboard, I have a key that looks like a Microsoft thing. 
> It looks like a window that is moving and I think is sometimes called a
> Microsoft key and might be called the meta key in Linux.

Well, you see a window on it. It is the Windows key. ;-)
The Linux world uses the more general term Super key. I think some also use 
Meta, but AFAIR there was also a Meta key on Solaris keyboards (labeled with 
a diamond shape).

> I have another key only on
> the right side between the Ctrl and the flying window key that looks
> like a document with a mouse pointer on it.

That’s the menu key. It calls up the context menu as if you clicked the 
right mouse button. It seems to have gone out of fashion with some 
manufacturers, especially on laptops and keyboards with a Fn key. Which is a 
bummer, because it forced me to assign a key combo in X11 to emulate it 
(Shift+Ctrl in my case). And this doesn’t work in Wayland.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

I used to avoid work, today I could just watch for hours.


signature.asc
Description: PGP signature

Re: [gentoo-user] 6.1.53-gentoo-r1 kernel not booting

[Frank Steinmetzger]

Am Sat, Sep 30, 2023 at 11:25:46PM +0200 schrieb Håkon Alstadheim:
> 
> Den 30.09.2023 22:57, skrev Valmor F. de Almeida:
> > 
> > Hello,
> > 
> > For a while now (3 weeks or so) I have been upgrading the linux kernel
> > on a Dell XPS laptop starting from 6.1.41-gentoo (which is my current
> > working kernel) to 6.1.53-gentoo-r1. No kernel I have built since is
> > able to boot. I have been following the same method for many years: make
> > oldconfig, etc...
> > 
> > The booting error starts at:
> > 
> > [snip]
> > 
> > * INIT: Entering runlevel: 3
> > [snip]
> > * Starting cronie ...
> > * Starting DHCP Client Daemon ...
> > * Starting laptop_mode ...
> > * Mounting network filesystems ...
> > /etc/init.d/netmount: line 45 /lib/rc/bin/ewend: Input/output error
> > /lib/rc/sh/rc-cgroup.sh: line 184: rmdir: command not found
> > INIT:
> > INIT: cannot execute "/sbin/agetty"
> > INIT: cannot execute "/sbin/agetty"
> > INIT: cannot execute "/sbin/agetty"
> > 
> > 
> Can you show /etc/fstab and the console-log for the entire boot? Seems /sbin
> is not readable. You sure you have the kernel modules loaded? Are you using
> an initramfs? If so, does that build without errors ?

The input/output error – to me – indicates a hardware problem. When you 
mounted the FS by hand, can you read ewend? For instance with md5sum.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

“If wishes were horses we’d all be eating steak.” – Jayne, Firefly


signature.asc
Description: PGP signature

Re: [gentoo-user] Switching from desktop to desktop without function keys.

[Frank Steinmetzger]

Am Sat, Sep 30, 2023 at 10:47:31PM +0100 schrieb Peter Humphrey:
> On Saturday, 30 September 2023 20:59:04 BST Dale wrote:
> 
> > Is there a way with the keyboard to switch to a desktop above 10?  Even
> > if it just switches up one at a time, that would work.  Say switch to 10
> > and then keep hitting a set of keys to go to 11, then 12, then 13 etc
> > etc.  Eventually, I get to the one I want. 
> 
> You can set up a key combination to switch one desktop to the right and 
> another to switch one to the left. It's under Shortcuts, where you select 
> KWin, then scroll the right-hand panel down to Walk through Desktop List, 
> where I've put what used to be the default value: CTRL-ALT-right. Then down 
> one to the next entry and enter CTRL-ALT-left. The defaults are left blank 
> nowadays.

Also note that a good while ago Plasma switched from using the Alt key to 
the Super key for everything Window-managerial. So these days, if you create 
a new user from scratch, it’s Ctrl+Super instead of Ctrl+Alt.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

Detours widen your local knowledge.


signature.asc
Description: PGP signature

Re: [gentoo-user] Network throughput from main Gentoo rig to NAS box.

[Frank Steinmetzger]

Am Sat, Sep 23, 2023 at 05:54:21PM +0200 schrieb ralfconn:

> On 9/23/23 14:04, Dale wrote:
> > Howdy,
> > 
> > As most everyone knows, I redone my NAS box.  Before I had Truenas on it
> > but switched to Ubuntu server thingy called Jimmy.  Kinda like the
> > name.  lol  Anyway, Ubuntu has the same odd transfer pattern as the
> > Truenas box had.  I'm not sure if the problem is on the Gentoo end or
> > the Ubuntu end or something else.  I'm attaching a picture of Gkrellm so
> > you can see what I'm talking about.  It transfers a bit, then seems to
> > stop for some reason, then start up again and this repeats over and
> > over.  I'm expecting more of a consistent throughput instead of all the
> > idle time.  The final throughput is only around 29.32MB/s according to
> > info from rsync.  If it was not stopping all the time and passing data
> > through all the time, I think that would improve.  Might even double.
> > 
> > ...
> > Has anyone ever seen something like this and know why it is idle for so
> > much of the time?  Anyone know if this can be fixed so that it is more
> > consistent, and hopefully faster?
> > 
> I found a similar pattern when I checked some time ago, while transferring
> big (several Gb) files from one desktop to the other. I concluded the cause
> of the gaps was the destination PC's SATA spinning disk that needed to empty
> its cache before accepting more data. In theory the network is 1Gb/s
> (measured with iperf, it is really close to that) and the SATA is 6Gb/s so
> it should not be the limit, but I have strong doubts as how this speed is
> measured by the manufacturer.

Please be aware there is a difference between Gb and GB: one is gigabit, the 
other gigabyte. 1 Gb/s is theoretically 125 MB/s, and after deducting 
network overhead you get around 117 MB/s net bandwidth. Modern 3.5″ HDDs 
read more than 200 MB/s in their fastest areas, 2.5″ not so much. In their 
slowest region, that can go down to 50..70 MB/s.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

A peach is like an apple covered by a carpet.


signature.asc
Description: PGP signature

Re: [gentoo-user] Network throughput from main Gentoo rig to NAS box.

[Frank Steinmetzger]

Am Sat, Sep 23, 2023 at 02:30:32PM -0500 schrieb Dale:

> I read the other replies and I think it is caching the data, the drives
> writes and catches up and then it asks for more data again.

Tool tip: dstat

It puts out one line of values every x seconds (x == 1 by default).
With arguments you can tell it what to show. To see disks in action, I like 
to run the following during upgrades that involve volumous packages:

dstat --time --cpu --disk -D  --net 
--mem-adv --swap'

The cpu column includes IO wait.

The disk columns show read and write volume. If you omit the -D option, you 
will only see a total over all disks, which might still be enough for your 
use case.

The mem-adv shows how much data is in the file system write cache (the --mem 
option does not).

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

Unburden your teacher; skip classes once in a while!


signature.asc
Description: PGP signature

Re: [gentoo-user] Re: How to move ext4 partition

[Frank Steinmetzger]

Am Wed, Sep 20, 2023 at 10:57:00PM +0100 schrieb Victor Ivanov:

> On Wed, 20 Sept 2023 at 22:29, Grant Edwards  
> wrote:
> >
> > That depends on how long it takes me to decide on tar vs. rsync and
> > what the appropriate options are.
> 
> I've done this a number of times for various reasons over the last 1-2
> years, most recently a few months ago due to hard drive swap, and I
> find tar works just fine:
> 
> $ tar -cpf /path/to/backup.tar --xattrs --xattrs-include='*.*' -C / .

Does that stop at file system boundaries (because you tar up '/')? I think 
it must be, otherwise you wouldn’t use it that way.
But when copying a root file system, out of habit I first bind-mount it in a 
subdirectory and tar/rsync from there instead. This will also make files 
visible which might be hidden under an active mount.

This is not necessary if you do it from a live system, but then you wouldn’t 
tar up / in the first place.

> Likewise to extract, but make sure "--xattrs" is present
> 
> Provided backup space isn't an issue, I wouldn't bother with
> compression. It could be a lot quicker too depending on the size of
> your root partition.

Or not, depending on the speed of the backup device. ;-)
LZO compression (or zstd with a low setting) has negligible CPU cost, but 
can lower the file size quite nicely, specially with large binaries or debug 
files.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

Do you steel taglines, too?


signature.asc
Description: PGP signature

Re: [gentoo-user] Re: Password questions, looking for opinions. cryptsetup question too.

[Frank Steinmetzger]

Am Wed, Sep 20, 2023 at 04:51:36PM -0400 schrieb Rich Freeman:

> > > Anyway, when I do that and use the new passwords successfully, I make a
> > > backup copy and on my rig, I can encrypt it with a right click.  I then
> > > shred the original.
> >
> > Just on a sidenote, once you’re on an SSD, shredding has no use and is
> > actually detrimental.
> >
> 
> I'm not sure I'd go quite that far, but it certainly isn't as effective.
> 
> No way to be certain how well it works, but it is certainly worth
> doing an ATA Secure Erase command on the drive.  A good SSD should
> implement that in a way that ensures all the data is actually
> unretrievable (probably by implementing full disk encryption and
> erasing the key).  Of course, there is no way to tell if the drive was
> implemented well.

Uhm, Dale was talking of a single file, not an entire disk. ;-)

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

do something for your image -- let yourself be seen with me.


signature.asc
Description: PGP signature

Re: [gentoo-user] Re: Password questions, looking for opinions. cryptsetup question too.

[Frank Steinmetzger]

Am Tue, Sep 19, 2023 at 11:49:24PM -0500 schrieb Dale:

> Anyway, when I do that and use the new passwords successfully, I make a
> backup copy and on my rig, I can encrypt it with a right click.  I then
> shred the original.

Just on a sidenote, once you’re on an SSD, shredding has no use and is 
actually detrimental.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

On Mondays I feel like Robinson -- I’m waiting for Friday.


signature.asc
Description: PGP signature

Re: [gentoo-user] Password questions, looking for opinions. cryptsetup question too.

[Frank Steinmetzger]

Am Wed, Sep 20, 2023 at 01:28:09PM +0100 schrieb Michael:

> > I have a question tho.  Can a person use a password/pass phrase that is
> > like this:  'This is a stupid pass phrase.'   Does it accept that even
> > with spaces? I know file names can have spaces for a long while now but
> > way back, you couldn't do that easily.  One had to use dashes or
> > underscores.

Sure, why not? It’s a string like any other. No spaces in filenames where a 
restriction of (now outdated) file systems. And I guess developers didn’t 
account for them back in those days (and later out of habit). When I used 
DOS, of course I adhered to the 8.3 rule. But ever since I started using 
Windows, XP at the latest (2001), I wholly started using spaces everywhere 
and never looked back. The programs that had problems with spaces were few 
and script authors should just adhere to best practices and put filename 
variables in quotes, so they can work with spaces. The only nuissance they 
pose for me is it may make tab completion cumbersome sometimes.

PS.: I find underscores ugly. :D

> Generally speaking space characters are a poor choice for randomness.  I 
> recall seeing some documentary about the Enigma machine used by the German 
> military during the 2nd WW.  To minimise attempts to brute force the 
> ciphertext, they started by identifying which letter(s) were most frequently 
> used in the German language - e.g. the letter "e", then the second most 
> frequent letter and so on.  This statistical analysis approach in combination 
> with likely message content reduced the number of guesses.

Here you speak of the payload, not the passphrase, which is the encyption 
key. The key was rotated after each character and the initial key setting (the 
tumbler position) was distributed in secret code books.

> In principle, a repeated space character in your passphrase could help 
> reduce the computational burden of an offline brute force attack, by e.g. 
> helping an attacker to identify the number of individual words in a 
> passphrase.

Due to the rotation, the Enigma encoded each subsequent letter differently, 
even if the same one repeated, which was (one of) the big strengths of the 
Enigma cipher. The flaws were elsewhere, for example that a character could 
never be encrypted onto itself due to the internal wiring and certain 
message parts were always the same, like message headers and greetings.

For LUKS, having spaces in your passphrase (or their frequency) has no 
influence on the ciphertext, since the passphrase itself is not used for 
encryption. The passphrase only unlocks the actual key, which is then used 
for encryption. It comes down to whether the passphrase can easily be 
guessed by dictionary attacks. So if you write normal sentences with 
correctly written words, they might be easy to crack. I don’t expect it 
makes a big difference to the brute force software whether you use spaces or 
not.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

Suicide is the most honest form of self-criticism.


signature.asc
Description: PGP signature

Re: [gentoo-user] Computer case for new build

[Frank Steinmetzger]

Am Tue, Sep 19, 2023 at 11:01:48AM -0400 schrieb Rich Freeman:

> > > The higher-end motherboards have switches, and not all
> > > the lanes may be the highest supported generation, but I don't think
> > > any modern AMD motherboards have any kind of PCIe controller on them.
> >
> > Here are the I/O capabilities of the socket:
> > https://www.reddit.com/r/Amd/comments/bus60i/amd_x570_detailed_block_diagram_pcie_lanes_and_io/
> 
> So, that is AM4, not AM5

Yup. I kept on rambling about AM4, because that’s what I laid my eyes on 
(and so did Dale a few posts up).

> > A slight problem is that it is connected to the CPU by only 4.0×4. So tough
> > luck if you want to do parallel high-speed stuff with two PCIe×4 M.2 drives.
> 
> So, that block diagram is a bit weak.  If you look on the left side it
> clearly shows 20 PCIe lanes, and the GPU only needs 16.  So there are
> 8 lanes for the MB chipset to use.

No, the chipset downlink is always four lanes wide. PCIe 4.0 for most 
AM4 CPUs, but PCIe 3.0 for the monolithic APUs (because they don’t have 
4.0 at all, as their I/O die is different). The remaining four lanes are 
reserved for an NVMe slot.

> The 4 on the left aren't the same as the 4 on the right I think.

The diagram is indeed a bit confused in that part.

> Again, that is AM4 which I haven't looked into as much.  AM5 increases
> the v5 lanes and still has some v4 lanes.

AFAIR, PCIe 5 is only guaranteed for the NVMe slot. The rest is optional or 
subject to the chipset. As in the A series doesn’t have it, stuff like that. 
But it’s been a while since I read about that, so my memory is hazy.

> All the same desktop CPUs are a bit starved for lanes.

Hey we did get four more now with AM5 vs. AM4.

> I'm sure PCIe v5 switching is hard/expensive, but they definitely
> could mix things up however they want.  The reality is that most IO
> devices aren't going to be busy all the time, so you definitely could
> split 8 lanes up 64 ways, especially if you drop a generation or two
> along the way.

Unfortunately you can’t put low-speed connectors on a marketing sheet, when 
competitors have teh shizz.

> Server hardware definitely avoids many of the limitations, but it just
> tends to be super-expensive.

Which is funny because with the global cloud trend, you would think that its 
supply increases and prices go down.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

If you were born feet-first, then, for a short moment,
you wore your mother as a hat.


signature.asc
Description: PGP signature

Re: [gentoo-user] Computer case for new build

[Frank Steinmetzger]

Am Tue, Sep 19, 2023 at 09:17:45AM -0400 schrieb Rich Freeman:

> > Well they allow you to put larger cards in, but they don’t have the lanes
> > for it. Somewhere else in the thread was mentioned that the number of lanes
> > is very limited. Only the main slot (the big one for the GPU) is directly
> > connected to the CPU. The rest is hooked up to the chipset which itself is
> > connected to the CPU either via PCIe×4 (AMD) or whatchacallit (DMI?) for
> > Intel.
> 
> So, on most AMD boards these days all the PCIe lanes are wired to the
> CPU I believe.

Not all. Only the main slot. The rest is routed through the chipset. I’m 
only speaking of expansion slots here. But for NVMe it is similar: the 
primary one is attached to the CPU, any other is connected via the chipset. 
This is for AM4. AM5 provides two NVMes.

> The higher-end motherboards have switches, and not all
> the lanes may be the highest supported generation, but I don't think
> any modern AMD motherboards have any kind of PCIe controller on them.

Here are the I/O capabilities of the socket:
https://www.reddit.com/r/Amd/comments/bus60i/amd_x570_detailed_block_diagram_pcie_lanes_and_io/
A slight problem is that it is connected to the CPU by only 4.0×4. So tough 
luck if you want to do parallel high-speed stuff with two PCIe×4 M.2 drives.

> Basically memory, USB, and PCIe are all getting so fast that trying to
> implement a whole bunch of separate controller chips just doesn't make
> sense.

However, the CPU has a limited number of them, hence there are more in the 
chipset. Most notably SATA.

> > Look for youself and filter what you need, like 1 or 2 HDMI, DP and PCIe:
> > AM4: https://skinflint.co.uk/?cat=mbam4=18869_4%7E4400_ATX
> > AM5: https://skinflint.co.uk/?cat=mbam5=18869_4%7E4400_ATX
> > Interestingly: the filter goes up to 6 PCIe slots for the former, but only 
> > to
> > 4 for the latter.
> 
> You can definitely get more PCIe slots on AM5, but the trend is to
> have less in general.

Those look really weird. “Huge” ATX boards, but all covered up with fancy 
gamer-style plastics lids and only two slots poking out.

> Look at the X670 chipset boards as those tend to have PCIe switches which 
> give them more lanes.  The switched interfaces will generally not support 
> PCIe v5.

The X series are two “B-chipset chips” daisychained together to double the 
downstream connections. Meaning one sits behind the other from the POV of 
the CPU and they share their uplink.

Here are some nice block diagrams of the different AM5 chipset families:
https://www.hwcooling.net/en/amd-am5-platform-b650-x670-x670e-chipsets-and-how-they-differ/

-- 
Grüße | Greetings | Qapla’
Please do not share anything from, with or about me on any social network.

Greet every douche, for he may be your superior tomorrow.


signature.asc
Description: PGP signature

Re: [gentoo-user] Computer case for new build

[Frank Steinmetzger]

Am Tue, Sep 19, 2023 at 04:43:02AM -0500 schrieb Dale:
> Wols Lists wrote:

> > Oh, and to the best of my knowledge, you can combine a video card and
> > an AGPU.

BTW: it’s APU, without the G. Because it is an Accellerated Processing Unit 
(i.e. a processor), not a GPU.

> I been on Newegg using their rig builder feature.  Just to get rough
> ideas, I picked a AMD Ryzen 9 5900X 12-Core 3.7 GHz Socket AM4.  Yea, I
> did a copy and paste.  lol  It's a bit pricey but compared to my current
> rig, I think it will run circles around it.  My current rig has a AMD FX
> -8350 Eight-Core Processor running at 4GHz or so.  You think I'll see
> some speed improvement or am I on the wrong track?

Twice the single-thread performance and 7 times multi-core:
https://www.cpubenchmark.net/cpu.php?cpu=AMD+FX-8350+Eight-Core=1780
https://www.cpubenchmark.net/cpu.php?cpu=AMD+Ryzen+9+5900X=3870
Naturally at lower power consumption as well.

> My problem is the mobo.  I need a few PCIe slots.  Most just don't have
> enough.  Most have a slot for a video card.  Then maybe 2 other slightly
> slower ones and maybe one slow one.  I can't recall what the names are
> at the moment. I know the length of the connector tends to tell what
> speed it is, tho some cheat and put long connectors but most of the
> faster pins aren't used.  That confuses things.

Well they allow you to put larger cards in, but they don’t have the lanes 
for it. Somewhere else in the thread was mentioned that the number of lanes 
is very limited. Only the main slot (the big one for the GPU) is directly 
connected to the CPU. The rest is hooked up to the chipset which itself is 
connected to the CPU either via PCIe×4 (AMD) or whatchacallit (DMI?) for 
Intel.

> Anyway, mobo, which I
> will likely change, CPU and memory is already adding up to about $600. 
> I don't need much of a video card tho.  The built in thing may be
> enough, as long as I can connect my monitor and TV.

The 5900X has no built-in. For the Ryzen 5000 series, only those with -G 
have graphics. The 7000 ones all have a basic GPU (may except for some with 
another suffix).

> If someone knows of a good mobo, Gigabyte, ASUS preferred, that has
> several PCIe slots, I'd like to know the model so I can check into it. 
> It's doesn't have to be the latest thing either.  I tend to drop down
> several notches from the top to save money.  I still end up with a
> pretty nice rig and save some money.

Look for youself and filter what you need, like 1 or 2 HDMI, DP and PCIe:
AM4: https://skinflint.co.uk/?cat=mbam4=18869_4%7E4400_ATX
AM5: https://skinflint.co.uk/?cat=mbam5=18869_4%7E4400_ATX
Interestingly: the filter goes up to 6 PCIe slots for the former, but only to 
4 for the latter.

-- 
Grüße | Greetings | Qapla’
Please do not share anything from, with or about me on any social network.

Wires are either too short, not available or don’t work.


signature.asc
Description: PGP signature

Re: [gentoo-user] Computer case for new build

[Frank Steinmetzger]

Am Tue, Sep 19, 2023 at 01:01:42AM -0500 schrieb Dale:

> They have added a lot of stuff to mobos since I bought one about a
> decade ago.  Maybe things have improved.  I just like PCIe slots and
> cards.  Gives me more options.

I definitely know the feeling. That is why I went with µATX instead of ITX 
nine years ago. I thought “now that I have a beefy machine, I could get a 
sound card and start music production” and stuff like that. It never 
happened. Aside from an entry-level GPU for some gaming (which broke two 
years ago, so I am back on Intel since then) I never used any of my slots. 
But in the end, they are — as you say yourself — options, not necessities.

> Given how things have changed tho, I may
> have to give in on some things.  I just like my mobos to be like Linux. 
> Have something do one thing and do it well.  When needed, change that
> thing.  ;-) 

Over the past years, boards tend to do less and less by themselves. It’s all 
been migrated into the CPU; voltage regulation, basic graphics, memory 
controller, lots of I/O. The chipset (at least in AMD land, I’ve been out of 
touch with Intel for a while now) basically determines the amount of 
*additional* I/O. The Deskmini X300 mini-PC that I mentioned earlier 
actually has no chipset on its board, everything is done by the CPU.

What irks me is again market segmentation. Even though Ryzen CPUs have the 
capability of 10 Gbps USB 3.1 Gen 2 built-in, the low-end boards do not 
route that out, not even at least one.

-- 
Grüße | Greetings | Qapla’
Please do not share anything from, with or about me on any social network.

The only thing that makes some people bearable is their absence.


signature.asc
Description: PGP signature

Re: [gentoo-user] Computer case for new build

[Frank Steinmetzger]

Am Mon, Sep 18, 2023 at 06:40:52PM -0500 schrieb Dale:

> >> I tend to need quite a few PCIe slots.  I like to have my own video
> >> card.  I never liked the built in ones.
> > You’re just asking to be asked. ;-) Why don’t you like them? (I fear I may 
> > have asked that before).
> >
> > I get it when you wanna do it your way because it always worked™ (which is 
> > not wrong — don’t misunderstand me) and perhaps you had some bad experience 
> > in the past. OTOH it’s a pricey component usually only needed by gamers and 
> > number crunchers. On-board graphics are just fine for Desktop and even 
> > (very) light gaming and they lower power draw considerably. Give it a 
> > swirl, 
> > maybe you like it. :) Both Intel and AMD work just fine with the kernel 
> > drivers.
> 
> Well, for one, I usually upgrade the video card several times before I
> upgrade the mobo.  When it is built in, not a option.  I think I'm on my
> third in this rig.
>
> I also need multiple outputs, two at least.

That is not a problem with iGPUs. The only thing to consider is the type of 
video connectors on the board. Most have two classical ones, some three, 
divided among HDMI and DP. And the fancy ones use USB-C with DisplayPort 
alternative mode. Also, dGPUs draw a lot more when using two displays.

> One for
> monitor and one for TV.  My little NAS box I'm currently using is a Dell
> something.  The video works but it has no GUI.  At times during the boot
> up process, things don't scroll up the screen.  I may be missing a
> setting somewhere but when it blanks out, it comes back with a different
> resolution and font size.

In case you use Grub, it has an option to keep the UEFI video mode.
So there would be no switching if UEFI already starts with the proper 
resolution.

> My Gentoo box doesn't do that.  I can see the screen from BIOS all the
> way to when it finishes booting and the GUI comes up.  I'm one of those
> who watches.  ;-)

Yeah, and it’s neat if there is no flickering or blanking. So modern and 
clean.

> >> Figure the case is a
> >> good place to start.  Mobo, CPU and such next.  Figure mobo will pick
> >> memory for me since usually only one or two will work anyway. 
> > One or two what?
> 
> One or two types of memory.  Usually, plain or ECC.  Mobos usually are
> usually pretty picky on their memory. 

Hm… while I haven’t used that many different components in my life, so far 
I have not had a system not accept any RAM. Just stick to the big names, I 
guess.

> >> Since no one mentioned a better case, that Define thing may end up being
> >> it.  That Gamemax is cheaper but a lot less drive capacity.  Heck, when
> >> I bought my current case, which has space for five 3.5" and six 5 1/4"
> >> drives, I thought I'd never fill up just the 3.5" ones.  Now, the 3.5"
> >> ones have been full for a while and the 5 1/4" are about full too.
> > Full with ODDs? Or drive cages? You can get 3×3.5″ cages which install into 
^

That should have been 5×3.5″. Too many threes and fives floatin’ around in 
my head and it’s getting late.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

The majority of people have an above-average number of legs.


signature.asc
Description: PGP signature

Re: [gentoo-user] Computer case for new build

[Frank Steinmetzger]

Am Mon, Sep 18, 2023 at 02:20:56PM -0500 schrieb Dale:

> >> […]
> >> The downside, only micro ATX and
> >> mini ITX mobo.  This is a serious down vote here.
> > Why is that bad? µATX comes with up to four PCIe slots. Even for ten 
> > drives, 
> > you only need one SATA expander (with four or six on-board). Perhaps a fast 
> > network card if one is needed, that makes two slots. You don’t get more RAM 
> > slots with ATX, either. And, if not anything else, a smaller board means 
> > (or can mean) lower power consumption and thus less heat.
> >
> > Speaking of RAM; might I interest you in server-grade hardware? The reason 
> > being that you can then use ECC memory, which is a nice perk for storage.¹ 
> > Also, the chance is higher to get sufficient SATA connectors on-board 
> > (maybe 
> > in the form of an SFF connector, which is actually good, since it means 
> > reduced “cable salad”).
> > AFAIK if you have a Ryzen PRO, then you can also use a consumer-grade 
> > board, 
> > because they too support ECC. And DDR5 has basic (meaning 1 bit and 
> > transparent to the OS) ECC built-in from the start.
> 
> I tend to need quite a few PCIe slots.  I like to have my own video
> card.  I never liked the built in ones.

You’re just asking to be asked. ;-) Why don’t you like them? (I fear I may 
have asked that before).

I get it when you wanna do it your way because it always worked™ (which is 
not wrong — don’t misunderstand me) and perhaps you had some bad experience 
in the past. OTOH it’s a pricey component usually only needed by gamers and 
number crunchers. On-board graphics are just fine for Desktop and even 
(very) light gaming and they lower power draw considerably. Give it a swirl, 
maybe you like it. :) Both Intel and AMD work just fine with the kernel 
drivers.

> I also have never had a good built in network port to work right either.  
> Every one of them always had problems if they worked at all.

I faintly remember a thread about that from long ago. But the same thought 
applies: in case you buy a new board, give it a try. Keep away from Intel 
I225-V though, that 2.5 GbE chip has a design flaw but manufacturers still 
use int.

> I also need PCIe slots for SATA expander cards.

That’s the use case I mostly thought of. Irritatingly, I just looked at my 
price comparison site for SATA expansion cards and all 8×SATA cards are PCIe 
2.0 with either two or even just one lane. -_- So not even PCIe 3.0×1, which 
is the same speed as 2.0×2 but would fit in a ×1 slot which many boards 
have in abundance.

2.0×2 is about 1 GB/s. Divided by 8 drives gives you 125 MB/s/drive.

> If I use
> the Define case, I'd like to spread that across at least two cards,
> maybe three.  So, network, video and at least a couple SATA cards,
> adding up fast.  Sometimes, I wouldn't mind having the larger ATX with
> extra PCIe slots.  Thought about having SAS cards and cables that
> convert to SATA.  I think they do that.  That may make it just one
> card.  I dunno.  I haven't dug deep into that yet.

After the disappointment with the SATA expanders I looked at SAS cards.
They are well connected on the PCIe side (2.0×8 or 3.0×8) and they are 
compatible with SATA drives. I found an Intel SAS card with four SFF 
connectors (meaning 16 drives!) for a little over 100 €. It’s called 
RMSP3JD160J. I don’t know why it is so cheap, though. Because the 
second-cheapest competitor is already at 190 €.

> Figure the case is a
> good place to start.  Mobo, CPU and such next.  Figure mobo will pick
> memory for me since usually only one or two will work anyway. 

One or two what?

> > I was going to upgrade my 9 years old Haswell system at some point to a new 
> > Ryzen build. Have been looking around for parts and configs for perhaps two 
> > years now but I can’t decide (perhaps some remember previous ramblings 
> > about 
> > that). Now I actually consider buing a tiny Deskmini X300 after I found out 
> > that it does support ACPI S3, but only with a specific UEFI version. No 
> > 10-gig USB and only 1-gig ethernet though. But it’s cute and small. :)
> 
> I thought about using a Raspberry Pi for a NAS box.  Just build more
> than one of them.  Thing is, finding the parts for it is almost
> impossible right now.  They kinda went away a couple years ago when
> things got crazy. 

I was talking main PC use case, not NAS. :)
The minimalist form factor doesn’t really impede me. I don’t have any HDDs 
in my PC anymore (too noisy), so why keep space for it. And while I do like 
to game a little bit, I find a full GPU too expensive and hungry, because it 
will be bored most of the time.

The rest can be done with USB, which is the only thing a compact case often 
lacks in numbers.

> Since no one mentioned a better case, that Define thing may end up being
> it.  That Gamemax is cheaper but a lot less drive capacity.  Heck, when
> I bought my current case, which has space for five 3.5" and six 5 1/4"
> drives, I thought I'd never fill up 

Re: [gentoo-user] Computer case for new build

[Frank Steinmetzger]

Am Mon, Sep 18, 2023 at 02:59:22PM -0400 schrieb Rich Freeman:

> > I have a four-bay NAS with server board (ASRock Rack E3C224D2I), actually my
> > last surviving Gentoo system. ;-) With IPMI-Chip (which alone takes several
> > watts), 16 GiB DDR3-ECC, an i3-4170 and 4×6 TB, it draws around 33..35 W
> > from the plug at idle — that is after I enabled all powersaving items in
> > powertop. Without them, it is around 10 W more. It has two gigabit ports
> > (plus IPMI port) and a 300 W 80+ gold PSU.
> 
> That's an ITX system though, and a very old one at that.

Well, you asked for entry-point server hardware with low idle consumption. 
;-)

I built it in November 2016. Even then it was old componentry, but I wanted 
to save €€€ and it was enough for my needs. I installed a Celeron G1840 for 
33 € because I thought it would be enough. I tested its AES performance 
beforehand (because it didn’t have AES-NI) and with 155 MB/s it was enough 
to saturate GbE. But since I ran ZFS on LUKS at the time (still do, until I 
change the setup for more capacity), I ran into a bottleneck during scrubs. 
So after a year, I paid over 100 € for the i3 which I should have bought 
from the get-go. :-/

> Not sure how
> useful more PCIe lanes are in a form factor like that.

Modern boards might come with NVMe slots that can be re-purposed for 
external cards.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

The more cheese, the more holes.  The more holes, the less cheese.
Ergo: the more cheese, the less cheese!


signature.asc
Description: PGP signature

Re: [gentoo-user] Computer case for new build

[Frank Steinmetzger]

Am Mon, Sep 18, 2023 at 07:16:17AM -0400 schrieb Rich Freeman:

> On Mon, Sep 18, 2023 at 6:13 AM Frank Steinmetzger  wrote:
> >
> > Am Mon, Sep 18, 2023 at 12:17:20AM -0500 schrieb Dale:
> > > […]
> > > The downside, only micro ATX and
> > > mini ITX mobo.  This is a serious down vote here.
> >
> > Why is that bad? µATX comes with up to four PCIe slots. Even for ten drives,
> > you only need one SATA expander (with four or six on-board). Perhaps a fast
> > network card if one is needed, that makes two slots.
> 
> Tend to agree.  The other factor here is that desktop-oriented CPUs
> tend to not have a large number of PCIe lanes free for expansion
> slots, especially if you want 1-2 NVMe slots.  (You also have to watch
> out as the lanes for those can be shared with some of the expansion
> slots so you can't use both.)
> 
> If you want to consider a 10GbE+ card I'd definitely get something
> with integrated graphics,

That is a recommendation in any case. If you are a gamer, you have a 
fallback in case the GPU kicks the bucket. And if not, your power bill goes 
way down.

> because a NIC is going to need a 4-8x port
> most likely 

Really? PCIe 3.0 has 1 GB/s/lane, that is 8 Gbps/lane, so almost as much as 
10 GbE. OTOH, 10 GbE is a major power sink. Granted, 1 GbE is not much when 
you’re dealing with numerous TB. And then there is network over thunderbolt, 
of which I only recently learned. But this is probably very restricted in 
length. Which will also be the case for 10 GbE, so probably no options for 
the outhouse. :D

> > Speaking of RAM; might I interest you in server-grade hardware? The reason
> > being that you can then use ECC memory, which is a nice perk for storage.
> 
> That and way more PCIe lanes.  That said, it seems super-expensive,
> both in terms of dollars, and power use.  Is there any entry point
> into server-grade hardware that is reasonably priced, and which can
> idle at something reasonable (certainly under 50W)?

I have a four-bay NAS with server board (ASRock Rack E3C224D2I), actually my 
last surviving Gentoo system. ;-) With IPMI-Chip (which alone takes several 
watts), 16 GiB DDR3-ECC, an i3-4170 and 4×6 TB, it draws around 33..35 W 
from the plug at idle — that is after I enabled all powersaving items in 
powertop. Without them, it is around 10 W more. It has two gigabit ports 
(plus IPMI port) and a 300 W 80+ gold PSU.

> > I was going to upgrade my 9 years old Haswell system at some point to a new
> > Ryzen build. Have been looking around for parts and configs for perhaps two
> > years now but I can’t decide (perhaps some remember previous ramblings about
> > that).
> 
> The latest zen generation is VERY nice, but also pretty darn
> expensive.  Going back to zen3 might get you more for the money,
> depending on how big you're scaling up.

I’ve been looking at Zen 3 the whole time, namely the 5700G APU. 5 times the 
performance of my i5, for less power, and good graphics performance for the 
occasional game. I’m a bit paranoid re. Zen 4’s inclusion of Microsoft 
Pluton (“Chip-to-Cloud security”) and Zen 4 in gereral has higher idle 
consumption. But now that Phoenix, the Zen 4 successor to the 5700G, is 
about to become available, I am again hesitant to pull the trigger, waiting 
for the pricetag.

> A big part of the cost of
> zen4 is the motherboard, so if you're building something very high end
> where the CPU+RAM dominates, then zen4 may be a better buy.

I’m fine with middle-class. In fact I always thought i7s to be overpriced 
compared to i5s. The plus in performance of top-tier parts is usually bought 
with disproportionately high power consumption (meaning heat and noise).

> If you just want a low-core system then you're paying a lot just to get
> started.

I want to get the best bang within my constraints, meaning the 5700G (
8 cores). The 5600G (6 cores) is much cheaper, but I want to get the best 
graphics I can get in an APU. And I am always irked by having 6 cores (12 
threads), because it’s not a power of 2, so percentages in load graphs will 
look skewed. :D

> The advantage of
> distributed filesystems is that you can build them out of a bunch of
> cheap boxes […]
> When you start getting up to a dozen drives the cost of getting them
> to all work on a single host starts going up.  You need big cases,
> expansion cards, etc.  Then when something breaks you need to find a
> replacement quickly from a limited pool of options.  If I lose a node
> on my Rook cluster I can just go to newegg and look at $150 used SFF
> PCs, then install the OS and join the cluster and edit a few lines of
> YAML and the disks are getting formatted...

For a simple media storage, I personally would find this too cumbersome to 
manage. Especially if you stick to Gentoo and don

Re: [gentoo-user] Computer case for new build

[Frank Steinmetzger]

Am Mon, Sep 18, 2023 at 12:17:20AM -0500 schrieb Dale:
> Howdy,
> […]
> I've found a few cases that peak my interest depending on which way I go
> with this.  One I found that has a lot of hard drive space and would
> make a descent NAS box, the Fractal Design Node 804.  It's a cube shaped
> thing but can hold a LOT of spinning rust. 10 drives plus I think space
> for a SSD for the OS as well.

These days you can always put your OS on an NVMe; faster access and two 
fewer cables in the case (or one more slot for a data drive).

> […]
> The downside, only micro ATX and
> mini ITX mobo.  This is a serious down vote here.

Why is that bad? µATX comes with up to four PCIe slots. Even for ten drives, 
you only need one SATA expander (with four or six on-board). Perhaps a fast 
network card if one is needed, that makes two slots. You don’t get more RAM 
slots with ATX, either. And, if not anything else, a smaller board means 
(or can mean) lower power consumption and thus less heat.

Speaking of RAM; might I interest you in server-grade hardware? The reason 
being that you can then use ECC memory, which is a nice perk for storage.¹ 
Also, the chance is higher to get sufficient SATA connectors on-board (maybe 
in the form of an SFF connector, which is actually good, since it means 
reduced “cable salad”).
AFAIK if you have a Ryzen PRO, then you can also use a consumer-grade board, 
because they too support ECC. And DDR5 has basic (meaning 1 bit and 
transparent to the OS) ECC built-in from the start.

> I was hoping to turn
> my current rig into a NAS.  The mobo and such parts.  This won't be a
> option with this case.  Otherwise, it gives ideas on what I'm looking
> for.  And not.  ;-)

I was going to upgrade my 9 years old Haswell system at some point to a new 
Ryzen build. Have been looking around for parts and configs for perhaps two 
years now but I can’t decide (perhaps some remember previous ramblings about 
that). Now I actually consider buing a tiny Deskmini X300 after I found out 
that it does support ACPI S3, but only with a specific UEFI version. No 
10-gig USB and only 1-gig ethernet though. But it’s cute and small. :)

> Another find.  The Fractal Design Define 7 XL.  This is more of a tower
> type shape like my current rig.  I think I read with extra trays, it can
> hold up to 18 drives.  One could have a fancy RAID setup and still have
> huge storage space with that.  I think it also has SSD spots for drives
> that could hold the OS itself.  This one is quite pricey tho.

With so many drives, you should also include a pricey power supply. And/or a 
server board which supports staggered spin-up. Also, drives of the home NAS 
category (and consumer drives anyways) are only certified for operation in 
groups of up to 8-ish. Anything above and you sail in grey warranty waters. 
Higher-tier drives are specced for the vibrations of so many drives (at 
least I hope, because that’s what they™ tell us).

> To be honest, I kinda like the Fractal Design Define 7
> XL right now despite the higher cost.  I could make a NAS/backup box
> with it and I doubt I'd run out of drive space even if I started using
> RAID and mirrored everything, at a minimum.

With 12 drives, I would go for parity RAID with two parity drives per six 
drives, not for a mirror. That way you get 2/3 storage efficiency vs. 1/2 
and more robustness; in parity, any two drives may fail, but in a cluster of 
mirrors, only specific drives may fail (not two of the same mirror). If the 
drives are huge, nine drives with three parity drives may be even better 
(because rebuilds get scarier the bigger the drives get).

> 9 pairs of say 18TB drives
> would give around 145TBs of storage with a file system on it.

If you mirrored them all, you’d get 147 TiB. But as I said, use nine drives 
with a 3-drive parity and you get 98 TiB per group. With two groups 
(totalling 18 drives), you get 196 TiB. Wh!


¹ There was once a time when ECC was supported by all boards and CPUs. But 
then someone invented market segmentation to increase profits through 
upselling.

-- 
Grüße | Greetings | Qapla’
Please do not share anything from, with or about me on any social network.

Skype jokes are oftentimes not understood, even when they’re repeated.


signature.asc
Description: PGP signature

Re: [gentoo-user] Certain packages refuse to use binary from build save, the -k thing.

[Frank Steinmetzger]

Am Fri, Sep 15, 2023 at 11:44:03PM -0500 schrieb Dale:
> Howdy,

Hi

instead of going berserk mode and wasting kWh on rebuilding “just in case it 
might help”, why not try and dig a little deeper.

> A couple of my video players are not playing videos correctly.

Whicch players?

> I've
> rebuilt a few things but no change.  Some work fine, some give a error
> about a bad index or something, some just don't even try at all.

What is the exact error message?
Can you remux the file with ffmpeg?
`ffmpeg -i Inputfile.ext -c copy -map 0 Outputfile.ext`
This will take all streams of the input file and put them into a new file 
without re-encoding. Does ffmpeg give any errors about bad data? Can you 
play the produced file?

> The videos come from various sources and are of different file extensions. 

Can we have a look at some of those videos? The least I can try is to see 
whether they work here or show any sign of corruption.

-- 
Grüße | Greetings | Qapla’
Please do not share anything from, with or about me on any social network.

Polymorphism is a multiform topic.  (SelfHTML forum)


signature.asc
Description: PGP signature

Re: [gentoo-user] long compiles

[Frank Steinmetzger]

Am Wed, Sep 13, 2023 at 12:50:20PM +0100 schrieb Wols Lists:

> Bear in mind a lot of systems are thermally limited and can't run at full
> pelt anyway ...

Usually those are space-constrained systems like mini PCs or laptops. 
Typical Desktops shouldn’t be limited; even the stock CPU coolers should be 
capable of dissipating all heat, as long as the case has enough air flow.

> You might find it's actually better (and more efficient) to run at lower
> loading. Certainly following the kernel lists you get the impression that
> the CPU regularly goes into thermal throttling under heavy load, and also
> that using a couple of cores lightly is more efficient than using one core
> heavily.

At least very current CPUs tend to go into so high clock speeds that they 
become quite inefficient. If you set a 105 W Ryzen 7700X to 65 W eco mode in 
the BIOS (which means that the actual maximum intake goes down from 144 W to 
84 W), you reduce consuption by a third, but only lose ~15 % in performance.

At very low figures (15 W), Ryzen 5000 and 7000 CPUs are almost as efficient 
as Apple M1.

-- 
Grüße | Greetings | Qapla’
Please do not share anything from, with or about me on any social network.

Einstein is dead.  Newton is dead.  I’m feeling sick, too.


signature.asc
Description: PGP signature

Re: [gentoo-user] TrueNAS not helping me now.

[Frank Steinmetzger]

Am Wed, Sep 06, 2023 at 02:45:11PM -0500 schrieb Dale:

> Oh, creating a
> vdev was the trick.  Once that is done, expand the pool.  It's one of
> those, once it is done, it seems easy.  ROFL

Note that people used to shoot themselves in the foot when lazily (or by 
accident) adding a single disk to an existing pool. If that pool was 
composed of RAID vdevs, then now they had a non-redundant single disk in 
that pool and it was not possible to remove a vdev from a pool! That 
single-disk vdev could only be converted to a mirror to at least get 
redundancy back.

The only proper solution was to destroy the pool and start from scratch. By now 
there is a partial remedy, in that it is possible to remove mirror vdevs from a 
pool. But no RAIDs:
https://forum.level1techs.com/t/solved-how-to-remove-vdev-from-zpool/192044/5
https://arstechnica.com/civis/threads/performance-when-removing-zfs-vdevs-with-zpool-remove.1481148/post-40491873
And you get some left-over metadata about the removed vdev.

> I guess vdev is like LVMs pv, physical volume I think it is.

Haven’t we had this topic before? At least twice? Including the comparison 
between 
the three layers of LVM with their equivalent in ZFS land. ;-)

ZFS is more meant for static setups, not constantly changing disk loadouts 
of varying disk sizes.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

The boss is a human just like everyone else, he just doesn’t know.


signature.asc
Description: PGP signature

Re: [gentoo-user] Email clients

[Frank Steinmetzger]

Am Sat, Jul 29, 2023 at 01:53:21AM -0400 schrieb Philip Webb:
> 230729 Peter Humphrey wrote:
> > I've been a loyal user of KMail for many years.
> > Claws mail is often mentioned hereabouts and I'd like to try it,
> > but first I'd need to export KMail's 20-odd-year maildir history
> > to mbox format.
> 
> I recommend a look at Mutt, which I've used very happily since  c 1998 ,
> well before Gentoo existed.  I've also always used Mbox, not Maildir.
> Powerful, configurable, but also simple : the UNIX approach.

When I had kmail issues back in the day of early akonadi times (remember 
Alan’s thread about data loss from then?), I tried out mutt and I’ve been 
using it ever since. I configured it to my liking re. list layout, sidebar, 
shortcuts, editing and so on.

I still use KMail these days, quite often too. But it has a few drawbacks 
and annoying little bugs that I encounter regularly, which is one reason for 
staying with mutt. Another is that mutt is much much faster when dealing 
with big directories such as lists. Still, there is no better graphical 
alternative in KDE land. Thunderbird & Co don’t fit in optically, Trojita is 
too limited.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

When things went bonkers for the captain, he had the entire ship jettisoned.


signature.asc
Description: PGP signature

Re: [gentoo-user] net-misc/x2goserver-4.1.0.3-r2

[Frank Steinmetzger]

Am Sat, Jul 15, 2023 at 08:50:47AM -0600 schrieb the...@sys-concept.com:
> x2goserver emerged with notice:
> 
> ERROR: preinst
> 
> Installation of a symlink is blocked by a directory:
>   '/etc/x2go/xinitrc.d'
> This symlink will be merged with a different name:
>   '/etc/x2go/xinitrc.d.backup.'
> 
> What does it mean?

Probably that it wants to install a symlink called /etc/x2go/xinitrc.d, but 
you already have a directory at this location. And now you have this symlink 
with the stated different name.

-- 
Grüße | Greetings | Qapla’
Please do not share anything from, with or about me on any social network.

“If you enjoy wasting time, is that time really wasted?” – Philosoraptor


signature.asc
Description: PGP signature

Re: [gentoo-user] Instrumenting emerges

[Frank Steinmetzger]

Am Sat, Jul 08, 2023 at 12:42:13PM -0300 schrieb David M. Fellows:

> while [ true ] ; do cat /proc/loadavg |logger; sleep 60; done

A spec more elegant:

while sleep 60; do ... ; done

-- 
Grüße | Greetings | Qapla’
Please do not share anything from, with or about me on any social network.

Taglines are like cars - You get a good one, then someone nicks it.


signature.asc
Description: PGP signature

Re: [gentoo-user] Update requires restarting init process

[Frank Steinmetzger]

Am Sun, Jul 02, 2023 at 11:51:52AM -0500 schrieb Dale:
> Howdy,
> 
> Just a heads up.  I almost missed this in all the other normal messages
> emerge spits out.  I think this was because of a glibc update.  Anyway,
> if you update your system, look for a message about restarting the init
> process to avoid problems.  Rebooting is one way to do this of course
> BUT if you can't or like me, don't like, to reboot, you can login as
> root and run this:  telinit u   Note, there is no dash in front of the
> u.  It doesn't work with a dash there either.  To make sure all
> processes restarted, I went to boot runlevel, then single and then
> reversed that to get back to default runlevel. 

Out of curiosity: going to boot and single runlevel requires you to log out 
and basically close all running programs anyway, right? So why not simply 
reboot then (unless you have something lying around in ramdisks)? Do you not 
like the uptime resetting? :D

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

If your laundry just wouldn’t become white,
may be that’s because it is coloured.


signature.asc
Description: PGP signature

Re: [gentoo-user] Invalid atom in /etc/portage/package.unmask: =app-text/evince-44.1 ~amd64

[Frank Steinmetzger]

Am Tue, May 02, 2023 at 01:37:50PM -0600 schrieb the...@sys-concept.com:
> Trying to emerge evince-44.1 but I get:
> 
> Invalid atom in /etc/portage/package.unmask: =app-text/evince-44.1 ~amd64
> 
> What is it looking for?

It is not looking for the ~amd64 at the end of the line.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

If two processes are running concurrently,
the less important will take processor time away from the more important one.


signature.asc
Description: PGP signature

Re: [gentoo-user] file system for new machine

[Frank Steinmetzger]

Am Sun, Apr 30, 2023 at 10:03:01AM +0800 schrieb William Kenworthy:

> > > That
> > > said btrfs has its less than stellar moments.  I still have systems that 
> > > use
> > > ext4 and they "seem" reliable for light duty but I make sure I have 
> > > backups
> > > and do not trust them with anything important - been bitten too many 
> > > times!
> > In what kind of situations did you encounter these problems?
> 
> Also a fixed number of nodes. on creation (annoying and sometimes 
> disastrous when it runs out - think lots of small files like mail 
> storage),

That would be my biggest concern, especially back in the day when I had 
rather limited hardware resources. I was “haggling” with myself as to how 
many inodes I would really need. These days I’m more generous, but still 
modify the inodes count when formatting a partition. See Dale’s recent SSD 
thread.

> power outages cause what seems like silent corruption that builds up.  I 
> will admit ext4 does seem better these days but I am not a fan.

OK, that I’ve never had. Maybe a few forced shutdowns because the machine 
hung up (e.g. memory full or a botched wake from suspend).

> How do you find f2fs? - I lose (wear out I guess) SD cards on raspberry pi
> and Odroid systems on a regular basis with any of the mainstream filesystems
> - using them as a boot drive only extends their life, but that's not always
> possible.

Well, no problems so far. But I’m not stress-testing it, it just runs™. The 
Pi is just a simple pihole/radicale/nextcloud server with not much traffic 
and the data card in my surface just holds my music collection. The only 
“issue” I currently encounter is some warning messages on Arch when I do a 
system update. I can’t remember the exact error, but it’s just a warning 
about some feature.

However:
The Arch wiki says: “F2FS has a weak fsck that can lead to data loss in case 
of a sudden power loss [3][4]. If power losses are frequent, consider an 
alternative file system.“

OTOH, Google is now using f2fs in Android data partitions. Before that, it 
was ext4. :-)

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

After humans ceased to be apes, they became Egyptians.


signature.asc
Description: PGP signature

Re: [gentoo-user] file system for new machine

[Frank Steinmetzger]

Am Sat, Apr 29, 2023 at 02:04:52PM +0100 schrieb Michael:
> On Saturday, 29 April 2023 12:45:31 BST Frank Steinmetzger wrote:
> > Am Sat, Apr 29, 2023 at 01:20:52PM +0800 schrieb William Kenworthy:
> 
> > > That
> > > said btrfs has its less than stellar moments.  I still have systems that
> > > use ext4 and they "seem" reliable for light duty but I make sure I have
> > > backups and do not trust them with anything important - been bitten too
> > > many times!
> > In what kind of situations did you encounter these problems?
> 
> Can't speak for William, but it was a case where using older/early versions 
> of 
> btrfs tools from some live-USB you found at the bottom of your bin of spares 
> could cause worse damage and data loss on btrfs.  I recall the devs 
> recommending to always use the latest version if you were attempting a 
> recovery of a damaged fs and seek advice if in doubt.

I was asking about his data loss with ext4. ;-)

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

ATARI! We make top-notch toasters affordable!


signature.asc
Description: PGP signature

Re: [gentoo-user] file system for new machine

[Frank Steinmetzger]

Am Sat, Apr 29, 2023 at 01:20:52PM +0800 schrieb William Kenworthy:

> Filesystem choice is very much to do with your particular use case.
> 
> I am not a fan of ext4 - lost too much data too many times.  I ve found
> btrfs and xfs much tougher, and the online tools much more convenient.

I’ve been using ext4 possibly (don’t know for sure) since it was available 
in standard Gentoo land. I cannot remember ever having suffered data loss.

These days I like to experiment with more flash-friendly systems like f2fs, 
which I use on the MicroSD card of my raspberry and the 400 GB data MicroSD 
in my Surface Go tablet. I also test-drive it on my mini desktop PC (all 
Arch linux) because, like all my machines, it has an SSD.

> That
> said btrfs has its less than stellar moments.  I still have systems that use
> ext4 and they "seem" reliable for light duty but I make sure I have backups
> and do not trust them with anything important - been bitten too many times!

In what kind of situations did you encounter these problems?

-- 
Grüße | Greetings | Qapla’
Please do not share anything from, with or about me on any social network.

Eating is the most important meal of the day.


signature.asc
Description: PGP signature

Re: [gentoo-user] file system for new machine

[Frank Steinmetzger]

Am Thu, Apr 27, 2023 at 04:58:02PM +0200 schrieb tastytea:

> > Does the transparent compression incur an overhead cost in processing,
> > memory use, or disk writes?  I feel like it certainly has to at least
> > use more memory.  Sorry if that's an RTFM question.
> 
> it'll use more cpu and memory, but disk writes and reads will be lower,
> because it compresses it on the fly.

The lzo algorithm which is used by default incurs a negligible performance 
penalty. Give it a try: take some big file, e.g. a video and then:
(with $FILE being the name of the file to compress)

Compression-optimised algorithms:
time gzip -k $FILE  # will take long with medium benefit
time xz -k $FILE# will take super long
time bzip2 -k $FILE # will take also long-ish

Runtime-optimised algorithms:
time lz -k $FILE# will go very very fast, but compression is relat. low
time zstd $FILE # will go fast with better compression (comp. effort 3)
time zstd -6 $FILE  # will go fast-ish with more compression

> it should detect early if a file is not compressible and stop.

AFAIK, zfs compresses the beginning of a file and only if that yields a 
certain benefit, the entire file will be compressed.

-- 
Grüße | Greetings | Qapla’
Please do not share anything from, with or about me on any social network.

The realist knows what he wants; the idealist wants what he knows.


signature.asc
Description: PGP signature

Re: [gentoo-user] Nvidia-drivers fails to patch

[Frank Steinmetzger]

Am Thu, Apr 20, 2023 at 08:33:22PM -0500 schrieb Dale:

> I cleared the tmp files to give it a fresh start.  It still failed.  The
> directory and files it complains about being missing, they are.  I went
> to the ebuild to see what patches are supposed to be installed.  This is
> the part of the ebuild. 
> 
> 
> PATCHES=(
>     "${FILESDIR}"/nvidia-drivers-470.141.03-clang15.patch
>     "${FILESDIR}"/nvidia-modprobe-390.141-uvm-perms.patch
>     "${FILESDIR}"/nvidia-settings-390.144-desktop.patch
>     "${FILESDIR}"/nvidia-settings-390.144-no-gtk2.patch
>     "${FILESDIR}"/nvidia-settings-390.144-raw-ldflags.patch
> )
> 
> 
> As you can see, it wants to apply patches from several versions so while
> odd, I guess it really does it that way.  I suspect given the age of the
> drivers that the patches no longer exist or something.  I'd think it
> would report it couldn't download the files but maybe not.  I may be
> running out of luck here.  Odd thing is, it compiled a while back. 

If I read your error output correctly, it’s not that the patch file is 
missing, but that a file that is mentioned inside the patch is.

-- 
Grüße | Greetings | Qapla’
Please do not share anything from, with or about me on any social network.

Error: this virus requires DirectX and 64 MB of RAM!


signature.asc
Description: PGP signature

Re: [gentoo-user] Re: Finally got a SSD drive to put my OS on

[Frank Steinmetzger]

Am Thu, Apr 20, 2023 at 04:29:59AM -0500 schrieb Dale:

> >> I wonder.  Is there a way to find out the smallest size file in a
> >> directory or sub directory, largest files, then maybe a average file
> >> size???
> > The 20 smallest:
> > `find -type f -print0 | xargs -0 stat -c '%s %n' | sort -n | head -n 20`
> >
> > The 20 largest: either use tail instead of head or reverse sorting with -r.
> > You can also first pipe the output of stat into a file so you can sort and 
> > analyse the list more efficiently, including calculating averages.
> 
> When I first run this while in / itself, it occurred to me that it
> doesn't specify what directory.  I thought maybe changing to the
> directory I want it to look at would work but get this: 

Yeah, either cd into the directory first, or pass it to find. But it’s like 
tar: I can never remember in which order I need to feed stuff to find. One 
relevant addition could be -xdev, to have find halt at file system 
boundaries. So:

find /path/to/dir -xdev -type f -! -type l …

> root@fireball /home/dale/Desktop/Crypt # `find -type f -print0 | xargs
> -0 stat -c '%s %n' | sort -n | head -n 20`
> -bash: 2: command not found
> root@fireball /home/dale/Desktop/Crypt #

I used the `` in the mail text as a kind of hint: “everything between is a 
command”. So when you paste that into the terminal, it is executed, and the 
result of it is substituted. Meaning: the command’s output is taken as the 
new input and executed. And since the first word of the output was “2”, you 
get that error message. Sorry about the confusion.

> >> I thought about du but given the number of files I have here,
> >> it would be a really HUGE list of files.  Could take hours or more too. 
> > I use a “cache” of text files with file listings of all my external drives. 
> > This allows me to glance over my entire data storage without having to plug 
> > in any drive. It uses tree underneath to get the list:
> >
> > `tree -afx -DFins --dirsfirst --du --timefmt "%Y-%m-%d %T"`
> >
> > This gives me a list of all directories and files, with their full path, 
> > date and size information and accumulated directory size in a concise 
> > format. Add -pug to also include permissions.
> >
> 
> Save this for later use.  ;-)

I built a wrapper script around it, to which I pass the directory I want to 
read (usually the root of a removable media). The script creates a new text 
file, with the current date and the dircetory in its name, and compresses it 
at the end. This allows me to diff those files in vim and see what changed 
over time. It also updates a symlink to the current version for quick access 
via bash alias.

-- 
Grüße | Greetings | Qapla’
Please do not share anything from, with or about me on any social network.

...llaw eht no rorrim ,rorriM


signature.asc
Description: PGP signature

Re: [gentoo-user] Re: Finally got a SSD drive to put my OS on

[Frank Steinmetzger]

Am Wed, Apr 19, 2023 at 06:09:15PM -0700 schrieb Mark Knecht:
> > I wonder.  Is there a way to find out the smallest size file in a
> directory or sub directory, largest files, then maybe a average file
> size???  I thought about du but given the number of files I have here, it
> would be a really HUGE list of files.  Could take hours or more too.  This
> is what KDE properties shows.
> 
> I'm sure there are more accurate ways but
> 
> sudo ls -R / | wc

Number of directories (not accounting for symlinks):
find -type d | wc -l

Number of files (not accounting for symlinks):
find -type f | wc -l

> give you the number of lines returned from the ls command. It's not perfect
> as there are blank lines in the ls but it's a start.
> 
> My desktop machine has about 2.2M files.
> 
> Again, there are going to be folks who can tell you how to remove blank
> lines and other cruft but it's a start.

Or not produce them in the first place. ;-)

> Only takes a minute to run on my Ryzen 9 5950X. YMMV.

It’s not a question of the processor, but of the storage device. And if your 
cache, because the second run will probably not use the device at all.

-- 
Grüße | Greetings | Qapla’
Please do not share anything from, with or about me on any social network.

Bosses are like timpani: the more hollow they are, the louder they sound.


signature.asc
Description: PGP signature

Re: [gentoo-user] Re: Finally got a SSD drive to put my OS on

[Frank Steinmetzger]

Am Wed, Apr 19, 2023 at 06:32:45PM -0500 schrieb Dale:
> Frank Steinmetzger wrote:
> > <<>>
> >
> > When formatting file systems, I usually lower the number of inodes from the 
> > default value to gain storage space. The default is one inode per 16 kB of 
> > FS size, which gives you 60 million inodes per TB. In practice, even one 
> > million per TB would be overkill in a use case like Dale’s media storage.¹ 
> > Removing 59 million inodes × 256 bytes ≈ 15 GB of net space for each TB, 
> > not 
> > counting extra control metadata and ext4 redundancies.
> 
> If I ever rearrange my
> drives again and can change the file system, I may reduce the inodes at
> least on the ones I only have large files on.  Still tho, given I use
> LVM and all, maybe that isn't a great idea.  As I add drives with LVM, I
> assume it increases the inodes as well.

I remember from yesterday that the manpage says that inodes are added 
according to the bytes-per-inode value.

> I wonder.  Is there a way to find out the smallest size file in a
> directory or sub directory, largest files, then maybe a average file
> size???

The 20 smallest:
`find -type f -print0 | xargs -0 stat -c '%s %n' | sort -n | head -n 20`

The 20 largest: either use tail instead of head or reverse sorting with -r.
You can also first pipe the output of stat into a file so you can sort and 
analyse the list more efficiently, including calculating averages.

> I thought about du but given the number of files I have here,
> it would be a really HUGE list of files.  Could take hours or more too. 

I use a “cache” of text files with file listings of all my external drives. 
This allows me to glance over my entire data storage without having to plug 
in any drive. It uses tree underneath to get the list:

`tree -afx -DFins --dirsfirst --du --timefmt "%Y-%m-%d %T"`

This gives me a list of all directories and files, with their full path, 
date and size information and accumulated directory size in a concise 
format. Add -pug to also include permissions.

-- 
Grüße | Greetings | Qapla’
Please do not share anything from, with or about me on any social network.

Computers are the most congenial product of human laziness to-date.


signature.asc
Description: PGP signature

Re: [gentoo-user] Re: Finally got a SSD drive to put my OS on

[Frank Steinmetzger]

Am Wed, Apr 19, 2023 at 01:00:33PM -0700 schrieb Mark Knecht:


> I think technically they default to the physical block size internally
> and the earlier ones, attempting to be more compatible with HDDs,
> had 4K blocks. Some of the newer chips now have 16K blocks but
> still support 512B Logical Block Addressing.
> 
> All of these devices are essentially small computers. They have internal
> controllers, DRAM caches usually in the 1-2GB sort of range but getting
> larger.

Actually, cheap(er) SSDs don’t have an own DRAM, but rely on the host for 
this. There is an ongoing debate in tech forums whether that is a bad thing 
or not. A RAM cache can help optimise writes by caching many small writes 
and aggregating them into larger blocks.

> The bus speeds they quote is because data is moving for the most
> part in and out of cache in the drive.

Are you talking about the pseudo SLC cache? Because AFAIK the DRAM cache has 
no influence on read performance.

> What I know I'm not sure about is how inodes factor into this.
> 
> For instance:
> 
> mark@science2:~$ ls -i
> 35790149  000_NOT_BACKED_UP
> 33320794  All_Files.txt
> 7840  All_Sizes_2.txt
> 7952  All_Sizes.txt
> 33329818  All_Sorted.txt
> 33306743  ardour_deps_install.sh
> 33309917  ardour_deps_remove.sh
> 33557560  Arena_Chess
> 33423859  Astro_Data
> 33560973  Astronomy
> 33423886  Astro_science
> 33307443 'Backup codes - Login.gov.pdf'
> 33329080  basic-install.sh
> 33558634  bin
> 33561132  biosim4_functions.txt
> 33316157  Boot_Config.txt
> 33560975  Builder
> 8822  CFL_88_F_Bright_Syn.xsc
> 
> If the inodes are on the disk then how are they
> stored? Does a single inode occupy a physical
> block? A 512 byte LBA? Something else?

man mkfs.ext4 says:
[…] the default inode size is 256 bytes for most file systems, except for 
small file systems where the inode size will be 128 bytes. […]

And if a file is small enough, it can actually fit inside the inode itself, 
saving the expense of another FS sector.


When formatting file systems, I usually lower the number of inodes from the 
default value to gain storage space. The default is one inode per 16 kB of 
FS size, which gives you 60 million inodes per TB. In practice, even one 
million per TB would be overkill in a use case like Dale’s media storage.¹ 
Removing 59 million inodes × 256 bytes ≈ 15 GB of net space for each TB, not 
counting extra control metadata and ext4 redundancies.

The defaults are set in /etc/mke2fs.conf. It also contains some alternative 
values of bytes-per-inode for certain usage types. The type largefile 
allocates one inode per 1 MB, giving you 1 million inodes per TB of space. 
Since ext4 is much more efficient with inodes than ext3, it is even content 
with 4 MB per inode (type largefile4), giving you 250 k inodes per TB.

For root partitions, I tend to allocate 1 million inodes, maybe some more 
for a full Gentoo-based desktop due to the portage tree’s sheer number of 
small files. My Surface Go’s root (Arch linux, KDE and some texlive) uses 
500 k right now.


¹ Assuming one inode equals one directory or unfragmented file on ext4.
I’m not sure what the allocation size limit for one inode is, but it is 
*very* large. Ext3 had a rather low limit, which is why it was so slow with 
big files. But that was one of the big improvements in ext4’s extended 
inodes, at the cost of double inode size to house the required metadata.

-- 
Grüße | Greetings | Qapla’
Please do not share anything from, with or about me on any social network.

FINE: Tax for doing wrong.  TAX: Fine for doing fine.


signature.asc
Description: PGP signature

Re: [gentoo-user] Re: Finally got a SSD drive to put my OS on

[Frank Steinmetzger]

Am Wed, Apr 19, 2023 at 12:18:14AM +0200 schrieb Frank Steinmetzger:

> If you use ext4, run `dumpe2fs -h /dev/your-root-partition | grep Lifetime` 
> to see how much data has been written to that partition since you formatted 
> it. Just to get an idea of what you are looking at on your setup.

For comparison:

I’m writing from my Surface Go 1 right now. It’s running Arch linux with KDE 
and I don’t use it very often (meaning, I don’t update it as often as my 
main rig). But updates in Arch linux can be volume-intensive, especially 
because there are frequent kernel updates (I’ve had over 50 since June 2020, 
each accounting for over 300 MB of writes), and other updates of big 
packages if a dependency like python changes. In Gentoo you do revdep-rebuild,
binary distros ship new versions of all affected packages, like libreoffice, 
or Qt, or texlive.

Anyways, the root partition measures 22 G and has a lifetime write of 571 GB 
in almost three years. The home partition (97 GB in size) is at 877 GB. That 
seems actually a lot, because I don’t really do that much high-volume stuff 
there. My media archive with all the photos and music and such sits on a 
separate data partition, which is not synced to the Surface due to its small 
SSD of only 128 GB.

-- 
Grüße | Greetings | Qapla’
Please do not share anything from, with or about me on any social network.

We shall be landing shortly.
Please return your stewardess to the upright position.


signature.asc
Description: PGP signature

Re: [gentoo-user] Re: Finally got a SSD drive to put my OS on

[Frank Steinmetzger]

Am Tue, Apr 18, 2023 at 10:05:27AM -0500 schrieb Dale:

> Given how I plan to use this drive, that should last a long time.  I'm
> just putting the OS stuff on the drive and I compile on a spinning rust
> drive and use -k to install the built packages on the live system.  That
> should help minimize the writes.

Well, 300 TB over 5 years is 60 TB per year, or 165 GB per day. Every day. 
I’d say don’t worry. Besides: endurance tests showed that SSDs were able to 
withstand multiples of their guaranteed TBW until they actually failed (of 
course there are always exceptions to the rule).

> I read about that bytes written.  With the way you explained it, it
> confirms what I was thinking it meant.  That's a lot of data.  I
> currently have around 100TBs of drives lurking about, either in my rig
> or for backups.  I'd have to write three times that amount of data on
> that little drive.  That's a LOT of data for a 500GB drive. 

If you use ext4, run `dumpe2fs -h /dev/your-root-partition | grep Lifetime` 
to see how much data has been written to that partition since you formatted 
it. Just to get an idea of what you are looking at on your setup.

-- 
Grüße | Greetings | Qapla’
Please do not share anything from, with or about me on any social network.

What woman is looking for a man who is looking for a woman looking for a man?


signature.asc
Description: PGP signature

Re: [gentoo-user] Re: Finally got a SSD drive to put my OS on

[Frank Steinmetzger]

Am Tue, Apr 18, 2023 at 09:53:18PM +0100 schrieb Wol:

> On 18/04/2023 21:01, Dale wrote:
> > > I just use tmpfs for /var/tmp/portage (16GB, I'm on 32GB RAM.)

Same.

> /var/tmp/portage on tmpfs. And on every disk I allocate a swap partition
> equal to twice the mobo's max memory. Three drives times 64GB times two is a
> helluva lot of swap.

Uhm … why? The moniker of swap = 2×RAM comes from times when RAM was scarce. 
What do you need so much swap for, especially with 32 GB RAM to begin with?
And if you really do have use cases which cause regular swapping, it’d be 
less painful if you just added some more RAM.

I never used swap, even on my 3 GB laptop 15 years ago, except for extreme 
circumstances for which I specifically activated it (though I never compiled 
huge packages like Firefox or LO myself). These days I run a few zswap 
devices, which act as swap, but technically are compressed RAM disks. So 
when RAM gets full, I get a visible spike in the taskbar’s swap meter before 
the system grinds to a halt.

-- 
Grüße | Greetings | Qapla’
Please do not share anything from, with or about me on any social network.

Night is so dark only so one can see it better.


signature.asc
Description: PGP signature

Re: [gentoo-user] Finally got a SSD drive to put my OS on

[Frank Steinmetzger]

Am Mon, Apr 17, 2023 at 10:45:46AM -0700 schrieb Mark Knecht:

> And I don't know that formatting ext4 or some other FS to 16K
> really helps the write amplification issue but it makes sense to
> me to match the file system blocks to the underlying flash
> block size.

The problem is finding out the write block size. This 7-year-old post says 
it’s reached 16 K: https://superuser.com/questions/976257/page-sizes-ssd

So I would say don’t bother. If everything is trimmed, there is no 
amplification. And if the disk becomes full and you get WA when writing 
itsy-bitsy 4 K files, you probably still won’t notice much difference, as 
random 4 K writes are slow anyways and how often do you write thousands of
4 K files outside of portage?

Erase block sizes probably go into the megabytes these days:
https://news.ycombinator.com/item?id=29165202

Some more detailed explanation:
https://spdk.io/doc/ssd_internals.html
  “For each erase block, each bit may be written to (i.e. have its bit 
  flipped from 0 to 1) with bit-granularity once. In order to write to the 
  erase block a second time, the entire block must be erased (i.e. all bits 
  in the block are flipped back to 0).”

This sounds like my initial statement was partially wrong – trimming does 
cause writing zeroes, because that’s what an erase does. But it still 
prevents write amplification (and one extra erase cycle) because 
neighbouring blocks don’t need to be read and written back.

> Real speed testing would be required to ensure reading
> 16K blocks doesn't slow him down though.

Here are some numbers and a conclusion gathered from a read test:
https://superuser.com/questions/728858/how-to-determine-ssds-nand-erase-block-size

Unless I positively need the speed for high-performance computing, I’d 
rather keep the smaller granularity for more capacity at low file sizes.

A problem is what some call “parts lottery” these days: manufacturers 
promise some performance on the data sheet (“up to xxx”), but not with which 
parts they want to achieve this (types of flash chips, TLC/QLC, controller, 
DRAM and so on). Meaning during the lifetime of a product, its internals may 
change and as a consequence those specs are not in the data sheet:

https://unix.stackexchange.com/questions/334804/is-there-a-way-to-find-out-ssd-page-size-on-linux-unix-what-is-physical-block
  “There is no standard way for a SSD to report its page size or erase block 
  size. Few if any manufacturers report them in the datasheets. (Because 
  they may change during the lifetime of a SKU, for example because of 
  changing suppliers.)
  For practical use just align all your data structures (partitions, payload 
  of LUKS containers, LVM logical volumes) to 1 or 2 MiB boundaries. It's an 
  SSD after all--it is designed to cope with usual filesystems, such as NTFS 
  (which uses 4 KiB allocation units).”

-- 
Grüße | Greetings | Qapla’
Please do not share anything from, with or about me on any social network.

The worst disease is indifference. So what?


signature.asc
Description: PGP signature

Re: [gentoo-user] Can some config files be automatically protected from etc-update?

[Frank Steinmetzger]

Am Mon, Apr 17, 2023 at 02:27:53PM -0700 schrieb Mark Knecht:

> ;-) (And shame on you for being 'a few months' behind on your updates) ;-)

It’s my NAS (basically my media library), which only runs every few months 
due to its server hardware’s high power draw.

-- 
Grüße | Greetings | Qapla’
Please do not share anything from, with or about me on any social network.

My computer waits faster!


signature.asc
Description: PGP signature

Re: [gentoo-user] Can some config files be automatically protected from etc-update?

[Frank Steinmetzger]

Am Mon, Apr 17, 2023 at 12:28:01PM -0700 schrieb Mark Knecht:
> On Mon, Apr 17, 2023 at 11:26 AM Walter Dnes  wrote:
> >
> >   Now that the (no)multilib problem in my latest update has been solved,
> > I have a somewhat minor complaint.  Can I get etc-update to skip certain
> > files?  My latest emerge world wanted to "update"...
> >
> > 1) /etc/hosts (1)
> > 2) /etc/inittab (1)
> > 3) /etc/mtab (1)
> > 4) /etc/conf.d/consolefont (1)
> > 5) /etc/conf.d/hwclock (1)
> > 6) /etc/default/grub (1)
> > 7) /etc/ssh/sshd_config (1)
> >
> > ...hosts is critical for networking.  consolefont allows me tp use the
> > true text console with a readable font, etc, etc.  I have my reasons
> > for making certain settings, and keeping them that way.
> >
> In my experience with all distros I go outside the distro for this
> sort of issue. Put a copy somewhere, white a little script that
> does a diff on the files you feel are important enough and run
> a cron job hourly that looks for any differences.

Isn’t that exactly what etc-update does? IIRC (my last Gentoo update was a 
few months ago), I select one of the files, and it lets me view a diff in 
vim (configurable) of my old version and the new one from the update. Then I 
can either merge the two files right in vim, or elect to keep the new or old 
file entirely.

-- 
Grüße | Greetings | Qapla’
Please do not share anything from, with or about me on any social network.

“I want to be free!” said the string puppet and cut its strings.


signature.asc
Description: PGP signature

Re: [gentoo-user] Finally got a SSD drive to put my OS on

[Frank Steinmetzger]

Am Sun, Apr 16, 2023 at 05:26:15PM -0500 schrieb Dale:

> > > I'm wanting to be able to boot something from the hard drive in the
> > > event the OS itself won't boot.  The other day I had to dig around and
> > > find a bootable USB stick and also found a DVD.  Ended up with the DVD
> > > working best.  I already have memtest on /boot.  Thing is, I very rarely
> > > use it.  ;-)
> >
> > So in the scenario you are suggesting, is grub working, giving you a
> > boot choice screen, and your new Gentoo install is not working so
> > you want to choose Knoppix to repair whatever is wrong with 
> > Gentoo? 
> 
> Given I have a 500GB drive, I got plenty of space.  Heck, a 10GB
> partition each is more than enough for either Knoppix or LiveGUI.  I
> could even store info on there about drive partitions and scripts that I
> use a lot.  Jeez, that's a idea. 

Back in the day, I was annoyed that whenever I needed $LIVE_SYSTEM, I had to 
reformat an entire USB stick for that. In times when you don’t even get 
sticks below 8 GB anymore, I found it a waste of material and useful storage 
space.

And then I found ventoy: https://www.ventoy.net/

It is a mini-Bootloader which you install once to a USB device, kind-of a 
live system of its own. But when booting it, it dynamically scans the 
content of its device and creates a new boot menu from it. So you can put 
many ISOs on one device as simple files, delete them, upgrade them, 
whatever, and then you can select one to boot from. Plus, the rest of the 
stick remains usable as storage, unlike sticks that were dd’ed with an ISO.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

The four elements: earth, air and firewater.


signature.asc
Description: PGP signature

Re: [gentoo-user] Finally got a SSD drive to put my OS on

[Frank Steinmetzger]

Am Sun, Apr 16, 2023 at 01:22:32PM -0700 schrieb Mark Knecht:
> Frank,
>Thank you for the in-depth explanation.
> 
>I need to do some study before commenting further other than to say
> so far I'm finding different comments depending on whether it's
> an SSD or an M.2 drive.

Uhm, I think you mix up some terms here. An M.2 drive *is* an SSD 
(literally, as the name says, a solid state drive). By “SSD”, did you mean 
the classic laptop form factor for SATA HDDs and SSDs?

Because M.2 is also only a physical form factor. It supports both NVMe and 
SATA. While NVMe is more modern and better suited for solid state media and 
their properties, in the end it is still only a data protocol to transfer 
data to and fro.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

„He who prefers security to freedom deserves to be a slave.“ – Aristotle


signature.asc
Description: PGP signature

Re: [gentoo-user] Finally got a SSD drive to put my OS on

[Frank Steinmetzger]

Am Sun, Apr 16, 2023 at 08:08:59AM -0700 schrieb Mark Knecht:

> If you have an SSD or nvme drive installed then fstrim should be
> installed and run on a regular basis. However it's not 'required'.
> 
> Your system will still work, but after all blocks on the drive have
> been used for file storage and later deleted, if they are not
> written back to zeros then the next time you go to use that
> block the write will be slower as the write must first write
> zeros and then your data.
> 
> fstrim does the write to zeros so that during normal operation
> you don't wait.

That is not quite correct. Trimming is about the oppisite of what you say, 
namely to *not* rewrite areas. Flash memory can only be written to in 
relatively large blocks. So if your file system wants to write 4 KiB, the 
drive needs to read all the many kB around it (several hundreds at least, 
perhaps eben MiBs, I’m not certain), change the small part in question and 
write the whole block back. This is called write amplification. This also 
occurs on hard drives, for example when you run a database which uses 4 kiB 
datafile chunks, but on a file system with larger sectors. Then the file 
system is the cause for write amplification.

If the SSD knew beforehand that the area is unused, it does not need to read 
it all in and then write it back. The SSD controller has no knowledge of 
file systems. And this is where trim comes in: it does know file systems, 
detects the unused areas and translates that info for the drive controller. 
Also, only trimmed areas (i.e. areas the controller knows are unused) can be 
used for wear leveling.

I even think that If you read from a trimmed area, the controller does not 
actually read the flash device, but simply returns zeroes. This is basically 
what a quick erase does; it trims the entire drive, which takes only a few 
seconds, and then all the data has become inaccessible (unless you address 
the memory chips directly). It is similar to deleting a file: you erase its 
entry in the directory, but not the actual payload bytes.

AFAIK, SMR HDDs also support trim these days, so they don’t need to do their 
SMR reshuffling. I have a WD Passport Ultra external 2.5″ HDD with 5 TB, and 
it supports trim. However, a WD Elements 2.5″ 4 TB does not. Perhaps because 
it is a cheaper series. Every laptop HDD of 2 (or even 1) TB is SMR.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

“It is hard to be a conquering hero when it is not in your nature.”
  – Captain Hans Geering, ’Allo ’Allo


signature.asc
Description: PGP signature

Re: [gentoo-user] Mouse and hibernate

[Frank Steinmetzger]

Am Wed, Apr 05, 2023 at 05:35:52PM +0800 schrieb William Kenworthy:

> > > I have suspend/hibernate set up on a desktop ... it's been working
> > > fine for years. But recently,  it's been occaisionally coming out of
> > > suspension some time after suspension without any intervention on my
> > > part.  I am suspecting the mouse - I would prefer not to disable the
> > > mouse ... Is there an alternative? BillK
> > Often there are options in the BIOS/UEFI to choose what can cause it to
> > come out of suspension.
> > 
> > 
> Unfortunately they are already off (the bios has PS2 settings) - the mouse
> is part of a keyboard/mouse set using a Logitech unifying USB dongle.  I can
> use a udev rule to turn off waking via the USB port, but I cant separate the
> mouse from the keyboard - and I need the keyboard enabled to wake the PC up.

Usually, Logitech mice have a switch on the bottom to physically turn them 
on or off. Usually I use that to circumvent wake-on-USB, rather than pulling 
out the USB wart.

-- 
Grüße | Greetings | Qapla’
Please do not share anything from, with or about me on any social network.

Someone who asks is a fool for five minutes.
Someone who never asks is a fool his entire life.


signature.asc
Description: PGP signature

Re: [gentoo-user] PCIe x1 or PCIe x4 SATA controller card

[Frank Steinmetzger]

Am Mon, Mar 27, 2023 at 07:24:47AM -0400 schrieb Rich Freeman:

> > Of course, only you can answer that in the end. Write down what you need and
> > what you care about. Weigh those factors. Then decide. Raw CPU power,
> > electricity bill, heat budget (cooling, noise, dust), the “new and shiny”
> > factor (like DDR5), and price. As I mentioned earlier, the 7xxx-X series are
> > hotheads. But when run with a lower power budget, they are very efficient
> > (which is basically what the non-X do).
> 
> Are they actually hotheads on an energy consumed per unit of work
> basis?  As you say, they're efficient.  If the CPU has 2x the power
> draw, but does 2.5x as much work in a unit of time than the "cooler"
> CPU you're comparing it to, then actually doing any job is going to
> consume less electricity and produce less heat - it is just doing it
> faster.
> […] 
> A recent trend is upping the power draw of CPUs/GPUs to increase their
> throughput, but as long as efficiency remains the same, it creates
> some thermal headaches, but doesn't actually make the systems use more
> energy for a given amount of work.  Of course if you throw more work
> at them then they use more energy.

Back in the day, CPUs were sold to run at an optimum work point, meaning a 
compromise between silicon wafer yield, power consumption and performance. 
Some of the chips were so good, they had the potential for overclocking, 
meaning they are stable enough to be clocked higher and to handle the heat. 
(But at no guarantee from the manufacturer, I presume. So if you grill it, 
it’s your loss.) And heat there was: you could increase a CPU from 4 GHz to 
4.4 GHz (10 % increase), but at a lot more power draw than just 10 %. The 
performance curve flattens at the high end; processing power does not scale 
linearly with power consumption beyond a certain point (else we would do it 
already).

These days, modern high-end CPUs seem to come over-clocked from the factory. 
Instead, if the user wants to run at a more efficient mode, the BIOS offers 
ways to tune down the power budget. You lose 10..20 % in performance, but 
gain 20 K in cooling and 30 % or more in power consumption.

10 years ago, when the very efficient Core architecture swept the market, 
the high-end “extreme” Haswell models drew 140 W. [0] Comare that to current 
generations [1] (Intel) or [2] (AMD), those go beyond 200 W. Of course they 
are much much faster, but average-Joe doesn’t need that.

Looking at concrete examples, the Ryzen 7900 has 3.7 GHz sustained max 
frequency (meaning no thermal throttling) at 65 W. The 7900X has 4.7 GHz (a 
quarter more) and 200 MHz more boost frequency, but is rated at 2½ times the 
wattage. The TDP does not tell you how much power the chip takes at most 
anymore (it can actually take much more in bursts or when it is still cool), 
but for how much thermal energy the cooling system needs to be designed in 
order to keep up the maximum (non-turbo, I think) frequency under load. This 
means that for a short time or on a low number of cores, the non-X can 
sustain almost as much boost clock as the X (it is the same silicon, after 
all), but once the cooling can’t keep up, it will throttle.

I’m not very good at explaining the math or providing hard numbers from 
memory, because all I know about this matter is from reading the occasional 
review. So please have a read yourself (see below). Another reason to take 
my word with a grain of salt: I am biased towards environmentally friendly 
choices. Power may still be cheap where you live, but every kWh produced has 
an impact on the globe.


Power efficiency (“points per Watt” metric):
https://www.techpowerup.com/review/amd-ryzen-9-7900x/24.html
Ryzen 7 5700G (i.e. laptop APU): 240.7 points
Ryzen 7 5700X: 84.5
Ryzen 7 7700X: 83.0
Ryzen 9 7900X: 47.2 at stock (meaning no down-scaling)

A comparison at https://www.xda-developers.com/amd-ryzen-9-7900-review/ 
shows only around 10 % more performance for the 7900X vs. the 7900:
  “The Ryzen 9 7900 is essentially the 7900X without PBO enabled, but it 
  would be a waste to spend more money on essentially the same chip to then 
  underclock it for better thermal performance. It's a better value choice 
  to pick up the Ryzen 9 7900 and then boost up to 7900X-level performance 
  through a simple BIOS toggle. After this has been carried out, performance 
  is pretty much identical.”

Some more reading fodder:
https://www.anandtech.com/show/18693/the-amd-ryzen-9-7900-ryzen-7-7700-and-ryzen-5-5-7600-review-ryzen-7000-at-65-w-zen-4-efficiency


[0] https://en.wikipedia.org/wiki/Haswell_(microarchitecture)
[1] https://en.wikipedia.org/wiki/Alder_Lake
[2] https://en.wikipedia.org/wiki/Zen_4

-- 
Grüße | Greetings | Qapla’
Please do not share anything from, with or about me on any social network.

“If privacy is outlawed, only outlaws will have privacy.” – Phil Zimmermann


signature.asc
Description: PGP signature

Re: [gentoo-user] PCIe x1 or PCIe x4 SATA controller card

[Frank Steinmetzger]

Am Sun, Mar 26, 2023 at 07:18:09PM -0500 schrieb Dale:

> I used to use the bogomips number as a rough guide.  Thing is, the new
> CPU has a lower bogomips number than my current CPU does.  That doesn't
> seem right.

Bogomips seems to be vry simple, because it takes the current frequency 
into account. So the number will be low when your PC idles and very high 
when you compile something. The “bogo” stands for bogus for a reason.

From Wikipedia [https://en.wikipedia.org/wiki/BogoMips]:
“It is not usable for performance comparisons among different CPUs.”

> So, I guess that number no longer means much.  So, I went
> digging on the site you linked to.  I found this but not sure what to
> make of it. 


> https://openbenchmarking.org/vs/Processor/AMD+Ryzen+9+5900X+12-Core,AMD+FX-8350+Eight-Core
> 
> Some tests, my CPU is faster.  Most, the new one is faster.

Your CPU is not faster at any of them. Look at the label at the top of each 
graph; for some tests, lower is better (as in “time taken for a task”).

For instance, the GnuPG test for encrypting a 2 GB file takes 11.6 seconds 
on the Ryzen, and 19.4 on your CPU. The test is single-threaded, so for this 
kind of task, you can expect around a ⅔ increase in performance per core (or 
rather, thread). OTOH, the m-queens 1.2 test is multi-threaded and you get 
39 s vs 238 s, meaning over 5 times more performance. Probably at lower 
electricity draw.

> I'm trying to figure if I'd be better in the
> long run to buy that expensive CPU or pick one of the cheaper ones you
> mentioned.  I started off with a 4 core on current rig and went to 8
> core and slightly higher frequency.  Money wise it was pretty painless. 
> I could do that again with new rig.

Of course, only you can answer that in the end. Write down what you need and 
what you care about. Weigh those factors. Then decide. Raw CPU power, 
electricity bill, heat budget (cooling, noise, dust), the “new and shiny” 
factor (like DDR5), and price. As I mentioned earlier, the 7xxx-X series are 
hotheads. But when run with a lower power budget, they are very efficient 
(which is basically what the non-X do).

Compiles will speed up no matter what CPU you choose. But where else do you 
need compute power? Video transcodes can be done in the background, and 
there is also a limit to what parallelisation can achieve. Encryption is 
also a non-issue for you. Even my 10 year old i3 in the NAS can encrypt over 
1 GB per second, IIRC.

-- 
Grüße | Greetings | Qapla’
Please do not share anything from, with or about me on any social network.

Dying is the most common cause of death.


signature.asc
Description: PGP signature

Re: [gentoo-user] PCIe x1 or PCIe x4 SATA controller card

[Frank Steinmetzger]

Am Sun, Mar 26, 2023 at 02:08:29PM -0500 schrieb Dale:
> Frank Steinmetzger wrote:
> > <<>>
> > With each generation, the architecture becomes more efficient, meaning more 
> > instructions per cycle, lower consumption and so on. The max frequency is 
> > not really the driving force behind performance increase anymore due to 
> > efficiency issues at higher frequencies.
> >
> > Here are some benchmark comparisons from cpubenchmark.net:
> >
> > Processor  year   power   cores   single-core score   multi-core score
> > FX-83502012   125 W   8/8   1580   6026
> > i5-4590201484 W   4/4   2086   5356
> > i5-10400   202065 W   6/12  2580  12258
> > R3 4300G   202065 W   4/8   2557  11017
> > R5 5600G   202165 W   6/12  3185  19892
> > R5 7600X   2022   145 W   6/12  4213  28753
> >
> > Sources:
> > https://www.cpubenchmark.net/singleThread.html#desktop-thread
> > https://www.cpubenchmark.net/cpu.php?cpu=AMD+FX-8350+Eight-Core=1780
> > https://www.cpubenchmark.net/cpu.php?cpu=Intel+Core+i5-4590+%40+3.30GHz=2234
> > https://www.cpubenchmark.net/cpu.php?cpu=Intel+Core+i5-10400+%40+2.90GHz=3737
> > https://www.cpubenchmark.net/cpu.php?cpu=AMD+Ryzen+3+4300G=3808
> > https://www.cpubenchmark.net/cpu.php?cpu=AMD+Ryzen+5+5600G=4325
> > https://www.cpubenchmark.net/cpu.php?cpu=AMD+Ryzen+5+7600X=5033
> > […]
> 
> It's been a while.  I been getting some things ready for garden time and
> a few spring projects as well.  I looked at a few lists of CPU
> processors.  This is a bit pricey but I may try to buy a AMD Ryzen 9
> 5900X 12-Core @ 3.7 GHz.  It has 4 more cores but clock speed is a
> little slower.  Even just comparing number of cores and the fairly close
> clock speed, that alone should make it a bit faster.  Add in that they
> make them run code more efficiently now, should be a good bit better.  I
> usually try to aim for 4 or 5 times more processing power.  I suspect
> this may help with encryption as well since newer CPUs have extra code
> just for that on there now.  Most of the mobos also handle a lot more
> memory as well.  I have 32GBs now.  Most support 64GB and I think I saw
> a 128GB version somewhere. 
> 
> Just comparing CPU to CPU, what would you expect as far as increase in
> speed?  I'm not expecting a exact number, just curious as to how much
> difference I could reasonably expect. 

Since I personally don’t have any experience with high-performance 
contemporary CPUs and can’t remember all those reviews I read in my newsfeed 
from time to time, I tend to visit benchmark sites like the aforementioned 
cpubenchmark.net. Those provide comparable numbers of synthetic and/or 
real-world benchmarks for both single- and multi-core use cases.

The Phoronix Test Suite is another notable name, and also very 
linux-centric. I haven’t used that one myself yet, but have a look and click 
around:
https://openbenchmarking.org/suites/pts

It’s open source, so you can run it on your own machine to get comparison 
numbers.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

If the cops arrest a mime, do they tell her she has the right
to remain silent?


signature.asc
Description: PGP signature

Re: [gentoo-user]Computer build, was PCIe x1 or PCIe x4 SATA controller card

[Frank Steinmetzger]

Am Wed, Mar 15, 2023 at 05:36:16PM -0500 schrieb Dale:
> Frank Steinmetzger wrote:

> > If you want to save money and aim for a low-cost AMD APU (processor with 
> > integrated graphics), you can get an older 3000-series Ryzen for a 
> > two-digit 
> > price. It’ll still be much faster than your old FX at a fraction of the 
> > power consumption. Like the 4300G, which is twice as fast for half the 
> > electricity. With today’s processors, basically none of the socktetable 
> > models are too slow unless you have specific performance requirements.
> > […]
> > Here are some benchmark comparisons from cpubenchmark.net:
> > […]
> > You can see the increase in performance. My old i5-4590, at half the cores, 
> > can keep up with your FX, even though it is only 1½ years younger. Ryzens 
> > used to be more efficient in multi workloads (look at the 2020 entries). 
> > But 
> > I’m not too sure about current generations due to Intel’s big-little 
> > concept.
> >> I was also wondering what a mobo/CPU/memory combo would cost nowadays. 
> >> Maybe someone who recently built a decent rig recalls how much they paid
> >> for those three.  I don't go cheap on power supply but I don't require a
> >> lot for a video card or anything.  Some spend half their money on a
> >> video card alone but I just don't need anything that fancy.
> > Any current Intel non-F CPU (F means no graphics) can cover your graphics 
> > need. Finally, AMD caught up and started shipping a minimal graphics chip 
> > in 
> > all of their processors with Zen 4, but as I said, that platform is still 
> > expensive.
> >
> >> I got a Nvidia GeForce GTX 650 that drives both my monitor and my TVs 
> >> through a splitter and it does just fine.
> > How cute. This should be about twice as fast as the integrated graphics in 
> > my 8-year-old i5. So you’ll be fine with *any* integrated graphics (which 
> > will also cut down on idle consuption, compared with a dGPU).
> > […]
> 
> This is all good info.  I went to Tom's Hardware and found their list by
> computing power.  I try to find a generic power rating since what I use
> my rig for is more generic.  No need looking at a chart for gaming. 
> ;-)  Anyway, I was looking at a somewhat costly Ryzen 7 5800x3d or a
> Ryzen 7 7700.

The 5800x3d got a huge boost from its cache. It may be the most powerful AM4 
socket processor around. But it is aimed mostly at gamers, because that’s 
where the cache mostly shines.

> Plus, if the video stops working, replace card instead of whole mobo.

Onboard graphics sits in the CPU. The mobo just routes the signal. And if 
any of that breaks, you can still fall back to a slot-in GPU.

> I also have to have two outputs.  One for desktop, one for TV.

Most boards have at least two outputs, nowadays usually one HDMI and DP.

> One other thing, the mobos I keep finding have few PCIe slots.  Some
> have 2 maybe 3.  That's getting to be to few for me.

Gamer boards tend to skimp on ports, because those people generally care 
mostly for their GPU (plus design and RGB). I believe you use ATX boards, 
right? Here is a list of AM4 ATX boards with at least four PCIe slots, 
totalling to 70 models (and still 31 with five slots):
https://skinflint.co.uk/?cat=mbam4=18869_4.
Feel free to play with the filters yourself.

> Then my next thing, a case.  The cases I find have a ton of lights, which 
> I hate, but as far as layout and such, they suck.  Some cost a arm and leg 
> and they are worthless to me.

Unilluminated ATX cases for at least five 3.5″ drives:
https://skinflint.co.uk/?cat=gehatx=17572_ohne%7E536_5%7E9691_ATX
Or for eight drives:
https://skinflint.co.uk/?cat=gehatx=17572_ohne%7E536_8%7E9691_ATX

There are some nice bland black boxes among them, notably the Fractal Define 
series.

> I found one the other day that is fairly plain, holds 8 or 10 hard drives 
> and has reasonably good cooling.  I'm hoping I can get it. 

Here, Linus is showcasing an 8-drives storage machine in a Fractal Define R4:
https://www.youtube.com/watch?v=DpJViwtct5g
And here a system with 18 drives 浪 in a Fractal Define 7XL:
https://www.youtube.com/watch?v=FAy9N1vX76o

> Usually I look forward to building a new rig.  Trying to find things I
> like takes the fun out of it.  I'll get there tho.  Eventually. 

I’ve been thinking for two years now. But my old PC keeps on running. I even 
play some Cities Skylines with its 8-year-old intel iGPU. I just can’t bring 
myself to discard that or to dosh out the money. And I can’t decide for a 
form factor. I want to have all at once and use them depending of the mood 
of the day. :D

-- 
Grüße | Greetings | Qapla’
Please do not share anything from, with or about me on any social network.

Win a Google Nexus! Scratch here ▒▒ with
a key or any other sharp object to reveal your winning code!


signature.asc
Description: PGP signature

Re: [gentoo-user] PCIe x1 or PCIe x4 SATA controller card

[Frank Steinmetzger]

Am Wed, Mar 15, 2023 at 11:03:45PM +0100 schrieb Frank Steinmetzger:
> Am Wed, Mar 15, 2023 at 07:45:25AM -0500 schrieb Dale:
> > Mark Knecht wrote:

> > > […]
> > > PixInsight has a benchmark program built in and all the results
> > > are open to look at:
> > >
> > > https://pixinsight.com/benchmark/index.php?sort=cpu=all
> > >
> > > Interestingly I didn't find your processor even on the list
> 
> That’s probably because the FX processors are old. Old and hungry. ^^

Correction: it is there. Search for just the number 8350 and you’ll find it.


-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

The brain of Donald Trump has two sides,
the left side and the right side.
In the left side there’s nothing right.
In the right side there’s nothing left.


signature.asc
Description: PGP signature

Re: [gentoo-user] PCIe x1 or PCIe x4 SATA controller card

[Frank Steinmetzger]

Am Wed, Mar 15, 2023 at 07:45:25AM -0500 schrieb Dale:
> Mark Knecht wrote:

> > > Another question.  My rig is getting a bit aged.  I have a AMD FX-8350 8
> > > core CPU running at 4GHz.  I also have 32GBs of memory.  I've read that
> > > Intel currently has the best bang for buck on CPUs nowadays.  I'm open
> > > to the idea of switching.  As far as speed goes, if I built a new rig
> > > that is using a reasonably cost CPU and memory, would I see any real
> > > improvements?
> >
> > I think it all depends on what you're going to use the machine for and
> > whether you really use all your CPU for extended periods of time. 

This! My mini PC with its passive 10 W Celeron N5100 is enough for desktop 
use, including encrypted storage. But maybe not for Gentoo. :)

> > […]
> > PixInsight has a benchmark program built in and all the results
> > are open to look at:
> >
> > https://pixinsight.com/benchmark/index.php?sort=cpu=all
> >
> > Interestingly I didn't find your processor even on the list

That’s probably because the FX processors are old. Old and hungry. ^^

> Sometimes a CPU that costs $500 can only be just a fraction faster than a 
> $200 CPU.

That’s still the case today for those impatient gamer enthusiasts who are 
after the “longest bars” [in benchmarks]. The same goes for power 
consumption. With Zen 4, AMD of course launched the fastest X-processors 
first with a gargantuan power demand. A few months later the non-X were 
released. They used 40 % or so less power at a performance cost of maybe 10 
% (not actual numbers, but figuratively speaking from memory).

> Given that my rig, as you point
> out, sits here and waits on me to do something most of the time, that's
> a lot of money for something I won't see much time savings on.  I might
> add tho, I do sometimes convert videos from 1080p to 720p.  That makes
> the CPU max out pretty good.  Compiling Libreoffice, Firefox etc also
> maxes out the CPU but those are what, once a month or so???

Intel and AMD are giving themselves quite a race these days about who offers 
more bang for the buck, or rather, more bang. In the past, Intel used to 
have more to offer at the lower end (below 100 € CPUs, like Pentiums and 
i3’s, while AMD was milking the market with high-end chips due to their 
limited manufacturing capacities).

If you want to save money and aim for a low-cost AMD APU (processor with 
integrated graphics), you can get an older 3000-series Ryzen for a two-digit 
price. It’ll still be much faster than your old FX at a fraction of the 
power consumption. Like the 4300G, which is twice as fast for half the 
electricity. With today’s processors, basically none of the socktetable 
models are too slow unless you have specific performance requirements.

With each generation, the architecture becomes more efficient, meaning more 
instructions per cycle, lower consumption and so on. The max frequency is 
not really the driving force behind performance increase anymore due to 
efficiency issues at higher frequencies.

Here are some benchmark comparisons from cpubenchmark.net:

Processor  year   power   cores   single-core score   multi-core score
FX-83502012   125 W   8/8   1580   6026
i5-4590201484 W   4/4   2086   5356
i5-10400   202065 W   6/12  2580  12258
R3 4300G   202065 W   4/8   2557  11017
R5 5600G   202165 W   6/12  3185  19892
R5 7600X   2022   145 W   6/12  4213  28753

Sources:
https://www.cpubenchmark.net/singleThread.html#desktop-thread
https://www.cpubenchmark.net/cpu.php?cpu=AMD+FX-8350+Eight-Core=1780
https://www.cpubenchmark.net/cpu.php?cpu=Intel+Core+i5-4590+%40+3.30GHz=2234
https://www.cpubenchmark.net/cpu.php?cpu=Intel+Core+i5-10400+%40+2.90GHz=3737
https://www.cpubenchmark.net/cpu.php?cpu=AMD+Ryzen+3+4300G=3808
https://www.cpubenchmark.net/cpu.php?cpu=AMD+Ryzen+5+5600G=4325
https://www.cpubenchmark.net/cpu.php?cpu=AMD+Ryzen+5+7600X=5033

You can see the increase in performance. My old i5-4590, at half the cores, 
can keep up with your FX, even though it is only 1½ years younger. Ryzens 
used to be more efficient in multi workloads (look at the 2020 entries). But 
I’m not too sure about current generations due to Intel’s big-little 
concept.
DDR5 and PCIe5 have higher requirements at signal quality, making the boards 
and components much more expensive (and, again, more power hungry). That’s 
why, even though DDR4 platforms are on their way out technologically, they 
are still an economically sound choice.

> I was also wondering what a mobo/CPU/memory combo would cost nowadays. 
> Maybe someone who recently built a decent rig recalls how much they paid
> for those three.  I don't go cheap on power supply but I don't require a
> lot for a video card or anything.  Some spend half their money on a
> video card alone but I just don't need anything that fancy.

Any current Intel 

Re: [gentoo-user] df command no longer working

[Frank Steinmetzger]

Am Sun, Feb 12, 2023 at 02:37:22PM -0600 schrieb Dale:
> Howdy,
> 
> As some may recall, I'm bad to fill up a hard drive.  I regularly use df
> -h to see where drives are as far as filling up and such.  Usually, it
> takes only a second or so to list them all.  Speed is one reason I use
> it.  I did my regular updates last weekend and for the past few days, it
> hasn't worked.  I did a re-emerge of coreutils, then some of its
> friends.  It still doesn't work.  When I run df -h, it just sits there. 
> It will sit there for hours, doing nothing it seems.  Eventually, I hit
> ctrl c to kill it. 
> 
> Anyone else running into this?  Any idea as to why it stopped working? 
> I can't find anything on BGO.  Searching for only two characters is a
> bit hard tho.  o_O

Sounds a little bit like a hanging NFS share; df goes through all mounts and 
tries to access them, but one isn’t responding.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

Democracy is a process which guarantees that
we are not governed any better than we deserve.


signature.asc
Description: PGP signature

Re: [gentoo-user] NAS and replacing with larger drives

[Frank Steinmetzger]

Am Wed, Dec 21, 2022 at 08:03:36PM + schrieb Wol:
> On 21/12/2022 06:19, Frank Steinmetzger wrote:
> > Am Wed, Dec 21, 2022 at 05:53:03AM + schrieb Wols Lists:
> > 
> > > On 21/12/2022 02:47, Dale wrote:
> > > > I think if I can hold out a little while, something really nice is going
> > > > to come along.  It seems there is a good bit of interest in having a
> > > > Raspberry Pi NAS that gives really good performance.  I'm talking a NAS
> > > > that is about the same speed as a internal drive.  Plus the ability to
> > > > use RAID and such.  I'd like to have a 6 bay with 6 drives setup in
> > > > pairs for redundancy.  I can't recall what number RAID that is.
> > > > Basically, if one drive fails, another copy still exists.  Of course,
> > > > two independent NASs would be better in my opinion.  Still, any of this
> > > > is progress.
> > > 
> > > That's called either Raid-10 (linux), or Raid-1+0 (elsewhere). Note that 
> > > 1+0
> > > is often called 10, but linux-10 is slightly different.
> > 

> > In layman’s term, a stripe of mirrors. Raid-1 is the mirror, Raid-0 a (JBOD)
> > pool. So mirror + pool = mirrorpool, hence the 1+0 → 10.
> 
> Except raid-10 is not a stripe of mirrors.
> It's each block is saved to two different drives. (Or 3, or more, so long
> as you have more drives than mirrors.)

Yes? In a mirror setup, all member drives of a mirror have the same content
(at least in ZFS).

Raid 10 distributes its content across several mirrors. This is the cause
for its increased performance. So when one of the mirrors (not single drive,
but a whole set of mirrored drives) fails, the pool is gone.

> Linux will happily give you a 2-copy mirror across 3 drives - 3x6TB drives
> will give you 9TB useful storage ...

I admit, I’ve never head of that. (Though it sounds like raid-5 to me.)

> > If I wanted to increase my capacity, I’d have to replace *all* drives with
> > bigger ones. With a mirror, only the drives in one of the mirrors need
> > replacing. And the rebuild process would be quicker and less painful, as
> > each drive will only be read once to rebuild its partner, and there is no
> > parity calculation involved. In a RAID, each drive is replaced one by one,
> > and each replacement requires a full read of all drives’ payload.
>
> If you've got a spare SATA connection or whatever, each replacement does not
> need a full read of all drives. "mdadm --add /dev/sdx --replace /dev/sdy".
> That'll stream sdy on to sdx, and only hammer the other drives if sdy
> complains ...

Strange that I didn’t think of that, even though it’s a perfectly clear
concept. In ZFS there is also a replace function which would do just that.
Currently I plan on keeping my old drives (who would want to buy them off of
me anyways) and just reorganise them in Z1 over Z2. I’ll just have to move
all data off to temprary external drives.

> > With older
> > drives, this is cause for some concern whether the disks may survive that.
> > That’s why, with increasing disk capacities, raid-5 is said to be obsolete.
> > Because if another drive fails during rebuild, you are officially screwed.
> > 
> > Fun, innit?
> > 
> They've always said that. Just make sure you don't have multiple drives from
> the same batch, then they're less likely statistically to fail at the same
> time. I'm running raid-5 over 3TB partitions ...

Yeah, I bought my drives from different shops back then for that reason.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

When the going gets tough, the tough get going.
... and so do I. – Alf


signature.asc
Description: PGP signature

Re: [gentoo-user] NAS and replacing with larger drives

[Frank Steinmetzger]

Am Wed, Dec 21, 2022 at 05:53:03AM + schrieb Wols Lists:

> On 21/12/2022 02:47, Dale wrote:
> > I think if I can hold out a little while, something really nice is going
> > to come along.  It seems there is a good bit of interest in having a
> > Raspberry Pi NAS that gives really good performance.  I'm talking a NAS
> > that is about the same speed as a internal drive.  Plus the ability to
> > use RAID and such.  I'd like to have a 6 bay with 6 drives setup in
> > pairs for redundancy.  I can't recall what number RAID that is.
> > Basically, if one drive fails, another copy still exists.  Of course,
> > two independent NASs would be better in my opinion.  Still, any of this
> > is progress.
> 
> That's called either Raid-10 (linux), or Raid-1+0 (elsewhere). Note that 1+0
> is often called 10, but linux-10 is slightly different.

In layman’s term, a stripe of mirrors. Raid-1 is the mirror, Raid-0 a (JBOD)
pool. So mirror + pool = mirrorpool, hence the 1+0 → 10.

> I'd personally be inclined to go for raid-6. That's 4 data drives, 2 parity
> (so you could have an "any two" drive failure and still recover).
> A two-copy 10 or 1+0 is vulnerable to a two-drive failure. A three-copy is
> vulnerable to a three-drive failure.

At first, I had only two drives in my 4-bay NAS, which were of course set up
as a mirror. After a year, when it became full, I bought the second pair of
drives and had long deliberations by then, what to choose. I went for raid-6
(or RaidZ2 in ZFS parlance). With only four disks, it has the same net
capacity as a pair of mirrors, but at the advantage that *any* two drives
may fail, not just two particular ones. A raid of mirrors has performance
benefits over a parity raid, but who cares for a simple Gbit storage device.

With increasing number of disks, a mirror setup is at a disadvantage with
storage efficiency – it’s always 50 % or less, if you mirror over more than
two disks. But with only four disks, that was irrelevant in my case. On the
plus-side, each mirror can have a different physical disk size, so you can
more easily mix’n’match what you got lying around, or do upgrades in smaller
increments.

If I wanted to increase my capacity, I’d have to replace *all* drives with
bigger ones. With a mirror, only the drives in one of the mirrors need
replacing. And the rebuild process would be quicker and less painful, as
each drive will only be read once to rebuild its partner, and there is no
parity calculation involved. In a RAID, each drive is replaced one by one,
and each replacement requires a full read of all drives’ payload. With older
drives, this is cause for some concern whether the disks may survive that.
That’s why, with increasing disk capacities, raid-5 is said to be obsolete.
Because if another drive fails during rebuild, you are officially screwed.

Fun, innit?

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

The boss is a human just like everyone else, he just doesn’t know.


signature.asc
Description: PGP signature

Re: [gentoo-user] NAS and replacing with larger drives

[Frank Steinmetzger]

Am Thu, Dec 15, 2022 at 10:08:02PM -0600 schrieb Dale:

I just read a news story about a new NAS model from Terramaster.
(Interestingly, they have their OS on an internal USB stick, so it’s easy to
swap it out for a standard Linux. And it uses a nice Celeron N5100 x86
processor.)

> Eventually, I plan to build a Raspberry Pi NAS.  When I do, I'll post
> everything major I needed, boards, case etc for everyone to look at. 
> I'll even try to upload some pics, or share as attachments if there is
> interest.  Unless I find one heck of a deal on a used NAS that is. 
> Still may build one even then.  ;-)

In the comments section of the article, there was a link to this:
https://www.youtube.com/watch?v=R2S2RMNv7OU
A Raspberry Pi NAS with 1 Petabyte of storage. Enjoy. :)

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

Windows is great, you can download Linux with it.


signature.asc
Description: PGP signature

Re: [gentoo-user] Gaming on gentoo

[Frank Steinmetzger]

Am Sat, Dec 17, 2022 at 11:54:28AM -0500 schrieb David Rosenbaum:
> Dave


Out of curiosity: what is the purpose of those mails which simply full-quote
another mail, but have no visible reply from you? I’ve been noticing them
quite often of late. Am I missing somehting?

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

Down with the date line! (32nd September Initiative)


signature.asc
Description: PGP signature

Re: Living in NGL: was: [gentoo-user] NAS and replacing with larger drives

[Frank Steinmetzger]

Am Sun, Dec 18, 2022 at 03:53:28PM -0600 schrieb Dale:

> > Did you ever tell us the exact CPU you have in there? All I can remember is
> > it has 4 cores. And some AMD processor with a II in its name, but that was
> > you main rig, right?
>
> It took some digging around but I found out it is a AMD Phenom 9750 quad
> core.

Great! That’s a model from 2008:
https://en.wikipedia.org/wiki/List_of_AMD_Phenom_processors#%22Agena%22_(B2/B3,_65_nm,_Quad-core)

> It lists some extensions but I don't see any that are related to
> encryption.  No AES or whatever for sure.  Unless I missed it.  I used

The instruction set’s original proposal is about as young as your processor:
https://en.wikipedia.org/wiki/AES_instruction_set

> My main rig has a AMD FX-8350 Eight-Core Processor and 32GBs of memory. 
> I'm thinking about a new rig eventually.  Rig is getting a little age on
> it.  ;-) 

At least it has AES ;-)
https://en.wikipedia.org/wiki/List_of_AMD_FX_processors#Piledriver_Core_(Vishera,_32_nm)

I’ve been thinking about a new build for about a year now, aiming at a
power-efficient Ryzen 5700G. But I can’t let go of my trusty old (8 years)
i5-4590.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.
So this unvaccinated man comes in no bar ...


signature.asc
Description: PGP signature

Re: Living in NGL: was: [gentoo-user] NAS and replacing with larger drives

[Frank Steinmetzger]

Am Sun, Dec 18, 2022 at 01:30:45PM -0700 schrieb Mark Knecht:

> I don't know how to get the CPU flags on FreeBSD nor
> how to determine if encryption is hardware or software
> based on TrueNAS. Given some time I might Google
> that.

Wikipedia has lists of practically everything:
https://en.wikipedia.org/wiki/List_of_AMD_processors
(We just need to know on which to click ;-) )

Another good source for such stuff is https://www.cpu-world.com/ .

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

Remember: now are the good old times
which you will be raving about in ten year’s time.


signature.asc
Description: PGP signature

Re: Living in NGL: was: [gentoo-user] NAS and replacing with larger drives

[Frank Steinmetzger]

Am Sun, Dec 18, 2022 at 01:07:43PM -0600 schrieb Dale:

> See other reply that has more info.  I'm pretty sure it is the
> encryption maxing out the CPU.

The most simple benchmark is dd: unlock the LUKS layer on your HDD. then
first read from the HDD directly and then from the LUKS device:

dd if= of=/dev/null bs=1M count=1000
dd if= of=/dev/null bs=1M count=1000

It will tell you the transfer rate in MB/s.

> Mostly, I need a better CPU.  If I encrypt anyway. 

Did you ever tell us the exact CPU you have in there? All I can remember is
it has 4 cores. And some AMD processor with a II in its name, but that was
you main rig, right?

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

It's not a bug, it's tradition!


signature.asc
Description: PGP signature

Re: [gentoo-user] NAS and replacing with larger drives

[Frank Steinmetzger]

Am Sun, Dec 18, 2022 at 12:38:45PM -0600 schrieb Dale:

> I suspect it has something to do with this being a older system.

Very likely. :)

> I wouldn't be surprised if the SATA was a older and slower version.

I hate to repeat myself, but no. Here are the speeds of SATA:

Generation   Year   Gross bandwidth   Net bandwidth
-
SATA 1   2003 1.5 Gbps1.2 Gbps (150 MB/s)
SATA 2   2004 3.0 Gbps2.4 Gbps (300 MB/s)
SATA 3   2008 6.0 Gbps4.8 Gbps (600 MB/s)

Even SATA 1 is faster than your new ethernet card.

> I guess I could google it. 

My source (as most often): https://en.wikipedia.org/wiki/SATA
If you know the name of a technical thing, the quickest way to concise
facts is wikipedia.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

The stupid keep tidy.  The genius controls chaos.


signature.asc
Description: PGP signature

Re: [gentoo-user] NAS and replacing with larger drives

[Frank Steinmetzger]

Am Sun, Dec 18, 2022 at 09:12:37AM -0600 schrieb Dale:

> > On Sat, Dec 17, 2022 at 4:42 PM Dale  > > wrote:
> > 
> > >
> > > My reasoning is simple, I'm already familiar with LVM and how to
> > manage it.  
> > 
> > […]
> > Wipe the machine. You'll be happier.
> >
> > Best wishes,
> > Mark
>
> Well, I finally got it so I could do a backup.  I didn't need a hammer
> but the thought crossed my mind.  lol  Even tho I now have a 1GB network
> card, it's still really slow.  It shows up as a 1GB connection on both
> my Gentoo machine and the NAS machine.  This is a example of the speeds
> I'm seeing.  Just snippets. 
> 
> 
> 277,193,507 100%   16.18MB/s    0:00:16
> 519,216,571 100%   18.86MB/s    0:00:26
> 738,078,565 100%   23.54MB/s    0:00:29
> 
> 
> As you can see, the files sizes are large enough it should do better. 

Gbit nets at around 116..117 MB/s.

> When I use iftop, it shows it isn't doing anywhere near the speed it
> should, maybe 1/4th or so.  I'd expect at least double or triple that
> speed.  In all honesty, I'd think the hard drive would be the limiting
> factor.  Even on my Gentoo rig I only get about 50 to 60MBs/sec for
> encrypted drives.  I think the encryption slows that down.  When copying
> from a plain drive to a plain drive, I get 100MBs/sec or so. 
>
> I can't figure out why it is so slow tho.  The NAS rig is a 4 core CPU
> and 8GBs of memory.

OK, so you already noticed that encryption slows you down. This won’t happen
with a CPU that has AES instructions (well, and if the encryption you chose
actually uses AES, and not something else like Blowfish). So I guess your
CPU is too old, given your earlier descriptions.

When I built my NAS in November 2016, I installed a Celeron G1840 at first.
A very affordable (33 €) and frugal CPU (2 cores, 53 W, which were never
actually drawn). I knew it didn’t have AES back then (Intel removed that
limit from Celerons in architectures after Haswell), but from experiments I
knew it would achieve around 150..160 MB/s with LUKS, which was enough for
Gbit ethernet. But not for scrubs, when all HDDs were worked in parallel. So
after a year I did an upgrade after all and bought the smallest and cheapest
CPU that had AES, an i3-41xx.

> It should have enough horsepower under the hood. 
> Maybe it is something I'm not aware of.  It is a older rig so maybe it
> isn't SATA's fastest version, maybe even the original or something.  I

SATA 2 is 3 Gbit/s, so still not saturated by a single HDD.

Network transfers are single-core work. If it is really such an old machine,
I guess the CPU is the bottleneck again. Do you transfer via ssh? If so, use
something else that doesn’t encrypt the transport stream. When I am bound by
CPU in such cases (like with my ancient netbook with an Atom N450), and I
don’t want to set up a file server (that is nowhere near as flexible as ssh
anyways), I use netcat:

On the receiving end, start a netcat listener and extract from it:
nc -l -p $Portnumber | tar xf -
The portnumber must be any number above 1024, if you’re not root.

And on the sender, pack all your stuff into a tar (uncompressed!, since
videos aren’t compressible further and it will bog down the CPU again) and
pipe it to the receiver:
tar cf - * | nc $Destination_IP $Portnumber

Once the client is done, press Ctrl+C on the receiver.

Or maybe use rsync with the rsync-protocol instead of ssh. That’ll be more
flexible, because the tar-and-nc method doesn’t know about existing files on
the receiving end. (But I’ve never tested that approach.)

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

You sould borrow money only from pessimists, because they don’t expect it back.


signature.asc
Description: PGP signature

Re: [gentoo-user] NAS and replacing with larger drives

[Frank Steinmetzger]

Am Sat, Dec 17, 2022 at 09:09:48AM -0700 schrieb Mark Knecht:

> On Sat, Dec 17, 2022 at 8:52 AM Dale  wrote:

> > > I
> > > wonder, could one install the LVM stuff and use that?  That would be
> > > interesting.  I wonder if there is a NAS software that uses LVM
> > > instead.  Interesting thought.  I just may go bug google on that one.
> >
> > Maybe I'm missing the point but why would you want LVM on a
> > storage pool? If I'm doing backups I just want space. I let TrueNas
> > put it on disk and give it back if asked. Why put another layer
> > of indirection?
> > […]
> It's more about me being more used to using LVM.  Also, more used to
> Linux as well.  BSD is not something I have much experience with and until
> recently, none with ZFS.  Even the little experience I have with BSD was
> well over a decade ago, maybe two decades ago.  I barely remember it really.

The truenas interface is supposed to hide all that from you. As Mark wrote
two posts up, it’s an appliance. You install it once, that’s it. I think you
can install addons, but don’t have to. You maybe do updates once in a while,
perhaps not even that if it’s a critical production host. You don’t ssh into
it to do maintenance or to add or remove disks on the commandline. That’s
what the web UI is for. All of it. The biggest and most obvious difference
for me would be how devices are named in comparison to Linux.

> Hummm...I don't know Dale, I don't know... ZFS is a file system.
> LVM is an abstraction on top (or underneath?) of a file system.
> My understanding of LVM is that it frees you from hard decisions
> on partition sizes, not that it replaces ZFS or ext3/4/5.

LVM is “just” an abstraction layer between file system and storage. At the
bottom there are your block devices (drives, image files), which then may or
may not be RAIDed. Then you have LVM on top of that to encompass all those
devices into one big “virtual hard drive” and partition it (logical
volumes). And lastly you put file systems into those volumes. So there are
two, three (or even four, if you add encryption somewhere in between) layers
stacked onto your raw devices.

ZFS OTOH is all of that in one. It takes the raw block devices as a whole,
puts them (optionally) into redundancy structures, allows for “partitions”
(i.e. datasets) and lastly also *is* the file system. The advantage is that
it can combine its knowledge about all those layers to improve performance
and reliability. So for instance it distributes writes according to vdev
occupancy (LVM has no knowledge about the FS layer above it). Or when you
rebuild a RAID, only those parts that are actually used by the FS need to be
reconstructed, not the whole disk.

I may come off as a ZFS fanboy in this thread. But I am in no way an expert,
just a small-time user with just that one NAS with one RAID setup. That’s
it. I did a lot of reading beforehand, whether I should use it or mdadm or
btrfs. All my knowledge comes from that time, I never worked with it in my
professional life.

I also use LVM on my systems these days, just in case my root partition
becomes crowded and needs some extra space that I can take from the media
partition. Had to do this once, it went quick and was a fun experience.

> WRT you I recommend that you try living in NGL for a while. Possibly
> you are just a bit too indoctrinated in the religion of building packages
> 30-50 times a year believing (without hard data) that it provides value.

+1
I admit I like the occasional update of my Gentoo NAS. But that’s also
because it’s my last living Gentoo device. I always liked the environment,
still do. Gentoo was my first Linux after all, that stuff stays in your
mind. But oh the time-consuming hassle sometimes.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

The fantasy of men is often inadequate to grasp the reality of women.


signature.asc
Description: PGP signature

Re: [gentoo-user] NAS and replacing with larger drives

[Frank Steinmetzger]

Am Sat, Dec 17, 2022 at 12:49:01AM -0600 schrieb Dale:

> > While ZFS has the same pooling feature as LVM, meaning you can bunch several
> > disks together to create a JBOD, it has one big disadvantage over LVM: you
> > can grow a pool, but not shrink it. Actually, while reading up on stuff for
> > this thread, I learned that these days it is actually possible to remove a
> > mirror vdev from a mirror-only pool (a mirror can technically also be a
> > single device). But according to
> > https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSWhyNoVdevRemoval
> > it’s not perfect either.
> >
> 
> One thing about LVM, I already know how to use it.  I've got notes on
> how to do things and it has worked.  With ZFS I'd have to learn it all
> over again plus it could confuse me with my using LVM on my main rig. 
> In a way, if I build a Raspberry NAS, I'd like to have Linux and LVM on
> the thing.  At least then I have experience moving data and such.  Right
> now, I don't even have the basics of ZFS or BSD.  I'm not saying TrueNAS
> is bad or anything, just that I don't really care for the learning
> curve.  Then the confusion part on top of that. 

The beauty of the WebUI is that it abstracts all of that away. You don’t
have to be in the know about the inner works and create pools and vdevs by
hand. Though it might be handy in case something breaks in an unexpected
way.

> I've read there are distros built for Raspberry thingys.  I'd be shocked
> if it isn't doable.  I'd be shocked if someone hasn't already done it
> even.   

It’s called RaspberryPi OS (formerly knows as Raspbian). It has the kernel,
some start scripts and settings files that control which stuff to load at
boot. Pis boot differently to x86 hardware; there is no Grub & co.

I don’t particularly like Debian’s apt, which is why I’m also experimenting
with Arch on arm, Arch being my favorite distro for my everyday these days.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

“Oh, gravity, thou art a heartless bitch.” – Dr. Sheldon Cooper


signature.asc
Description: PGP signature

Re: [gentoo-user] NAS and replacing with larger drives

[Frank Steinmetzger]

Am Fri, Dec 16, 2022 at 09:50:01PM -0600 schrieb Dale:

> > My point is: there is no need for a monitor. ;-)
> >
> 
> I noticing that now.  Once the install is done and you have the IP
> address, heck, you don't need a monitor for much of anything it
> appears.  I even found a console so one can type in things to do.  I
> wonder, could one install the LVM stuff and use that?  That would be
> interesting.
>
> I wonder if there is a NAS software that uses LVM instead.  Interesting
> thought.  I just may go bug google on that one.  o_O

Interestingly, after feeding LVM NAS to my search engine, one of the first
results was a Qnap user forum thread which reads that Qnaps use LVM. It
makes sense, since they most likely use a “normal” linux software RAID
underneath, and LVM is then the best way to dynamically manage the space on
top of that.

While ZFS has the same pooling feature as LVM, meaning you can bunch several
disks together to create a JBOD, it has one big disadvantage over LVM: you
can grow a pool, but not shrink it. Actually, while reading up on stuff for
this thread, I learned that these days it is actually possible to remove a
mirror vdev from a mirror-only pool (a mirror can technically also be a
single device). But according to
https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSWhyNoVdevRemoval
it’s not perfect either.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

5 of 4 people have problems with subsets.


signature.asc
Description: PGP signature

Re: [gentoo-user] NAS and replacing with larger drives

[Frank Steinmetzger]

Am Fri, Dec 16, 2022 at 04:43:25PM -0600 schrieb Dale:

> > 4) While I cannot tell you if the card you ordered has a Linux or FreeBSD
> > driver, I can point out that on the left hand side of your TrueNAS 
> > dashboard, accessible in your browser hopefully, there is a pulldown
> > called 'Network'. It should hopefully show you the current network
> > interface which in my case is called 're0'. On the right you might,
> > hopefully
> > possibly see a big blue button called "ADD". Consider giving that button 
> > a push after you've installed your new card.

> 4:  I have a monitor hooked up still so I can do it the text way if
> needed.  It mentions about setting up the network as one of the
> options.
>
> Since you mentioned it has a GUI option, I may just do that. 
> So long as it works. 

That’s what I referred to earlier: you get a web interface built-in that
takes care of all the chores. I always wanted to set up shares on my NAS so
that guests could easily access it anonymously, but safely (meaning: no
write access). I never had the patience to go through the whole setup of ftp
and/or samba with the proper users and directories. But a web UI could take
care of all of that. Plus it’s shiny. :D

My point is: there is no need for a monitor. ;-)

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

LCARS - Linux Can Also Run Starships


signature.asc
Description: PGP signature

Re: [gentoo-user] NAS and replacing with larger drives

[Frank Steinmetzger]

Am Thu, Dec 15, 2022 at 10:08:02PM -0600 schrieb Dale:

> I have a couple questions.  I currently have the NAS thingy on a older
> Dell machine.  It has a 4 core CPU and 8GBs of ram so it is acceptable,
> for the time being at least.  Bad thing is, only two drive bays.  :/  I
> have a few questions that I can't quite find answers to with google. 
> 
> 1:  I have the OS on a USB stick.  From what I've read, they do fail due
> to wear at some point.

OTOH, TrueNAS is designed to run from it, so I would assume it handles its
root drive with care. Perhaps you can disable verbose logging and such.

> If I reinstall TrueNAS on a new USB stick, will it automatically see the
> previous pools and such or do I have to set everything up again fresh?

Pools and their metadata are stored inside the pools. In Linux, you don’t
even need to set up fstab. The pool stores its mount point internally. So
you just start the zfs daemon and it does everything magically.

> In other words, will I lose data?

You won’t lose data, of course. But I think you meant settings(?). Probably
about users, shares and such. Perhaps it has an export feature which can be
run periodically.

> This also includes if it is encrypted.

Encryption is a built-in ZFS feature. So yes, it will remember that. Not
sure about the decryption process (keyfile).

> 2:  Hardware change.  The Dell comes with a 100MB network card.  I
> ordered a 1GB card.  I plan to put it in when it gets here.  Will it see
> the new card and work automatically or will it take some work to get the
> network going?

I assume the kernel is built like many general-purpose-distros: with
everything in it you may need for the purpose. But since it is BSD, it may
have driver issues (availability and stability for certain cards).
Sometimes, when I read news about a new product, people complain that the
NIC is not Intel and will thus cause problems with BSD, especially with
niche stuff like the Killer-brand ethernet cards.

> and recompile.  I'm not sure about BSD tho.  Since it is sort of a
> binary thing, does TrueNAS handle hardware changes such as a network
> card well? 

I don’t see a connection between being a “binary thing” and hardware change.
Your gentoo is also a binary thing once it is compiled. ;-)

> I also found out something power wise.  The Dell when booted and sitting
> idle consumes about 120 watts monitor and all.

I figured as much when you mentioned its 100 Mbps card. It must be old then,
and back then, idle power was a non-issue.

> My main rig consumes just under 200 watts.  Not to bad

That’s a very lot for my taste. With a lower mid-range GPU (110 W Radeon R7
370) and one spinning rust, my 8-year-old PC used to idle at 50 W. Without
the HDD and with Intel graphics it is now at 27 W. Still not a good number
when compared with today’s hardware.

> but a Raspberry Pi would likely consume 15, 20 watts max according to what
> I've read.

My 3B idles at 5 W tops, I think. It cannot be much more under load since it
comes without a built-in heat spreader.

> Given the number of hard drives, it could pull 25 or 30 watts max but
> doubtful it would get that high.  I'm looking at 4 bays but also found a 6
> bay.  I think 6 is overkill tho. 

My four-bay NAS has four 6 TB drives and it draws around 50 W at idle. But
that’s because it is a server board, incuding IPMI chip (and—interestingly—
an internal USB-A for an OS stick). And it’s Haswell generation, so almost a
decade old design. For this reason I switch it on only every few weeks or
even months and only keep it running for a short time.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

Death is a permanent damage.


signature.asc
Description: PGP signature

Re: [gentoo-user] NAS and replacing with larger drives

[Frank Steinmetzger]

Am Sun, Dec 11, 2022 at 08:44:42AM -0700 schrieb Mark Knecht:

> Also, I think there are ways for you to build complex pools like a RAID0
> from your 6TB and 8TB drives, and then a RAID1 using the RAID0 and your
> 14TB drive but I've never tried it because mine don't have enough drive
> slots for that.

After a longer fruitless search on the interwebs (I ddidn’t want to start up
my NAS just to check this) I finally found the right search keywords and
found a reddit thread about that. And it even throws LVM into the
discussion. ^^
https://www.reddit.com/r/zfs/comments/fitc73/raidz_with_nested_vdevs/

Also :
“Here's a definitive answer from the man page for zpool.

Virtual devices cannot be nested, so a mirror or raidz virtual device can
only contain files or disks. Mirrors of mirrors (or other combinations) are
not allowed.”


I would advise against a JBOD pool anyways. Because if one drive dies, the
whole JBOD is gone. That goes for ZFS and probably for LVM, too (though I am
not sure how writes are distributed across JBOD disks). If the goal is
redundancy, you could buy a second drive to match the size of an existing
one and build a mirror. If redundancy is not a goal, then use the drives
separately like you do now. If one fails, then only its content is gone (or
even just the files sitting on the broken sector).

> Also, turn on compression. It saves me between 15-20% so 14TB becomes 16TB
> storage. YMMV. Video files don't compress, at least not much. Data files
> generally do.

It doesn’t hurt to switch it on, especially with lzo. But with video, the
benefit will be negligible. When storing a block of data (a “record” in ZFS
speak), it is passed through the compressor and only if the compression gain
is above a given threshold (10 % methinks), the block is written to disk
with compression.

What is more relevant in filesystems for big files (i.e. videos): set the
record size to 1 MB. The default is 64 kB, IIRC. Each record requires one
block of metadata (which includes the record checksum). So bigger records →
fewer meta blocks → better storage efficiency.

If you use big records for small files, then efficiency goes down a little.
It’s a similar (but a little more complicated) principle as when you write a
100 byte text file to a file system that uses 4 kB clusters. That file will
still use up 4 kB on disk.

The record size can be set per-dataset. So in your pool you could create a
dataset with a smaller record size for office documents, images and music,
and another dataset just for videos.

> Hope this helps. I think you'll find TrueNAS fun actually but there is a
> learning curve. I've used it for about a year and barely scratched the
> surface.

The main reason for me why I would wanna use it as opposed to a standard
Gentoo install: the OOTB web interface to manage all sorts of accounts,
access and permissions under one nice hood.

-- 

Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

A fermata comes to the doctor: “I can’t hold it any longer...”


signature.asc
Description: PGP signature

Re: [gentoo-user] NAS and replacing with larger drives

[Frank Steinmetzger]

Am Sat, Dec 10, 2022 at 09:20:17AM + schrieb Wols Lists:

> > > Depending on the PVR make/model I've seen 1080p resolution recordings with
> > > .m2ts and .ts file extensions, while the codecs inside them are the same.
-^^^
> 
> > I wasn’t aware that ts could contain h264. But then again—I never really
-^^^
> > bothered with live TV recordings in recent years.

> I think this is confusing CONTAINER and CODEC.

Where do we confuse those two? We specifically talked of codecs and
“contain”.

> .ts is a container format, h264 is a codec. I don't understand it myself,
> either but think of ts as your directory structure and h264 as your file
> structure.

Now you are confusing me. You say you don’t understand it, but then explain
it. TS is like AVI and MKV: a file structure for the payload data. And
payload data can be all kinds of stuff, from ASS plaintext subtitles, over
opus audio to mpeg2 or h264 video.

> Incidentally, sticking this stuff in a .tar is probably okay - that's just
> another container, but sticking it in a .tar.gz is not, the gz is your codec
> and will make the file BIGGER in all probability.

Tar does not compress, it simply puts all inputs in a 1:1 stream. It does
add some metadata (filename and so on). Packers reduce data volume by
increasing information-per-byte. So if the total information stays the same
(for lossless coding), the number of bytes decreases. Encoded video data
ideally has even entropy. It is indistinguishable from random noise. That’s
why compressing it again does not yield anything, or even adds some volume
again.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

“Mankind must put an end to war or war will put an end to mankind.”
 – John F. Kennedy


signature.asc
Description: PGP signature

Re: [OT] Re: [gentoo-user] NAS and replacing with larger drives

[Frank Steinmetzger]

Am Fri, Dec 09, 2022 at 02:27:07PM + schrieb Peter Humphrey:
> On Friday, 9 December 2022 13:38:32 GMT Frank Steinmetzger wrote:
>
> > ...I never really bothered with live TV recordings in recent years. These
> > days, if I find something interesting, I download the show form the TV
> > channel’s website (called Mediathek in Germany, a word play on Bibliothek,
> > meaning library). Interestingly though, the picture quality is noticably
> > worse than what I receive via DVB-T.
>
> I do nearly the opposite: I record every TV programme I want to watch, then
> watch it at my leisure.

This summer I bought my first TV ever since I live on my own – after about
20 years of not having one. I haven’t missed TV and still don’t (just wanted
a bigger screen for movies played from the PC and to relax on the couch in
the evening).

> That way I can skip through all the adverts, which I loathe [1].

Who doesn’t? With my little antenna, I get all the public stations with good
quality and no ads, but I don’t care for the encrypted private, ad-financed
channels for which I need to pay a yearly fee these days (on top of the
mandatory public TV “tax”).

> Freesat in the UK allows recording of radio series as well, which Sky
> cannot do, so it's easy to capture late-night series and listen to them at
> my convenience. I have something like 600 radio programmes on my satellite
> box.

Ooof, I listen to podcasts and can barely keep up, time-wise. To have a
conneciton to the original topic: I still have lots of movies and series on
the NAS which I haven’t even started to watch.

> 1.  Mind you, UK TV adverts are nowhere near as gross as the screeching
> horrors I was subjected to in Minneapolis 30 years ago.

I can only imagine.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

Bees aren’t at all hard working, they just can’t fly any slower.


signature.asc
Description: PGP signature

Re: [gentoo-user] NAS and replacing with larger drives

[Frank Steinmetzger]

Am Fri, Dec 09, 2022 at 01:13:50PM + schrieb Michael:

> > > I get the impression Dale isn't actually PLANNING his disk storage. It's
> > > just a case of "help I'm downloading all this stuff where do I put it!!!"
> >
> > Haha, thanks for the laugh.
>
> Actually this had me thinking what is the need to back up the ... Internet?
> […]
>
> I appreciate some of these video files may be rare finds, or there may be a
> risk some of these may be taken off the interwebs sooner or later.  This
> should leave a rather small subset of all downloads, which may merit a local
> backup, just in case.  I'd thought the availability of higher fiber download
> speeds negates the need for local backups, of readily downloadable media.

Good points. I am a big fan of having stuff locally as well, because I don’t
want to be dependent on a company’s servers and a working Internet connection.
But this mostly applies to my mobile device, because I don’t have a data plan
for mobile Internet.

> > Well, ts uses mpeg2 encoding, just like old video DVDs, which is very
> > inefficient when compared with modern h264/h265. Modern digital TV broadcast
> > uses h264 by now.
>
> Depending on the PVR make/model I've seen 1080p resolution recordings with
> .m2ts and .ts file extensions, while the codecs inside them are the same.

I wasn’t aware that ts could contain h264. But then again—I never really
bothered with live TV recordings in recent years. These days, if I find
something interesting, I download the show form the TV channel’s website
(called Mediathek in Germany, a word play on Bibliothek, meaning library).
Interestingly though, the picture quality is noticably worse than what I
receive via DVB-T.

> > ¹ I do have several external USB disks, plus the big NAS. All of which don’t
> > run very often. And I don’t want to turn them on just to look for a certain
> > file. That’s why I have another little script. ;-) It uses the `tree`
> > command to save the complete content listing of a directory into a text
> > file and names the file automatically by the name of the directory it
> > crawls. So if I want to find a file, I just need to grep through my text
> > files.
>
> Backup scripts utilising rsync, tar, etc. can output a log file which contains
> (some) details of all the backed up files.  Nothing as sophisticated as
> Frank's script, but it allows for a quick search against the name of the file
> or directory, before extraction.

Naturally, I just discovered two bugs in the script while I was re-reading
my mail. One of them broke the creation of the symlink which points to the
most recent version of a script output. The other prevented normal operation
if only gzip was available amongst the used compressors.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

We promise nothing, but that we keep.
#!/usr/bin/env bash

# history
# 2018-02-10 initial version
# 2020-06-10 added -a option
# -??-?? gzip result
# 2021-03-22 put current date into output filename
# 2021-04-06 code refactoring with more functions and input sanitisation
# 2021-12-22 prefer zstd over gzip
# 2022-01-23 don't create symlink if there is no previous file
# 2022-01-30 added xz to compressors
# 2022-12-08 some cleanup, added -C and -K options
# 2022-12-09 bug fixes

die() {
echo "${@}" > /dev/stderr
exit 1
}

usage() {
cat <<-EOF
Usage: $(basename "$0") [-o NAME] [DIR]
A wrapper to tree, it writes the content of DIR into a text file.
The file is named after DIR and the current date, and symlink to the
most recent version is set. The file is automatically compressed to
zstd, xz or gzip, whichever is available in that order.

Options:
  -aaccess attributes: owner, group, permissions
  -Cdo not compress result file
  -Kdo not keep backup of existing file in case of overwriting
  -oThe destination where to write the trees.
Default: .
If it is a directory: write in there
If it is a filename: use that as base. If not,
use the name of the directory as name base.
EOF
}

test_writable() {
touch "$1" 2>/dev/null || die "Cannot create file in $(dirname "$1")."
}

# run tree and redirect output to destination file
call_to_tree() {
WHICH="$1"
OUTPATH="$2-$1"
shift 2

declare TREE_ARGS
TREE_ARGS+=("$@")
TREE_ARGS+=("-o")
TREE_ARGS+=("$DATED_PATH")

local DATED_PATH="$OUTPATH-$TODAY"
local EXT
local PACK
local CREATE_SYMLINK=no

if [ "$COMPRESS" = "no" ]; then
0
elif command -v zstd > /dev/null; then
PACK="zstd --rm -q -13"
EXT=".zst"
elif command -v xz > /dev/null; then
PACK="xz"
EXT=".xz"
else
PACK="gzip"
EXT=".gz"
fi

ls "$OUTPATH-"* > /dev/null 2>&1 && CREATE_SYMLINK=yes

# pack yet unpacked file

Re: [gentoo-user] NAS and replacing with larger drives

[Frank Steinmetzger]

Am Thu, Dec 08, 2022 at 05:30:18PM -0600 schrieb Dale:

> > I use USB3 hard drives on Pis for my bulk storage because I care about
> > capacity far more than performance, and with a distributed filesystem
> > the performance is still good enough for what I'm doing.  If I needed
> > block storage for containers/VMs/whatever then use a different
> > solution, but that gets expensive fast.
> > […]
>
> From my understanding, you are right about USB3 and GB ethernet being
> the big change.  They also have more memory and faster CPUs but if you
> bottleneck the data with slow USB and ethernet with the old ones, who
> needs a fast CPU?  I think they realized that the USB and ethernet had
> to improve.  It got better from there. 
> 
> https://shop.allnetchina.cn/collections/sata-hat/products/dual-sata-hat-open-frame-for-raspberry-pi-4
> 
> I found the above.  From my understanding, it allows a SATA drive to
> connect to either 2 or 4 bays.

Looking at the pics, it looks all very wibbly-wobbly. You will either have
the parts lying around open on a desk or you need to find a case for all
that stuff which adheres to no industry standard form factor. Pi accessories
are quite hard to come by, since they’re often sold out.

> One thing I like about the Raspberry option, I can upgrade it later.  I
> can simply take out the old, put in new, upgrade done.  If I buy a
> prebuilt NAS, they pretty much are what they are if upgrading isn't a
> option.

If you just do storage, what do you need upgrades for, anyway? All it needs
to do is receive your data and write it to disk. And then return it later
when asked for. I don’t remember you mentioning running VMs or some such.
Any current commercial NAS has enough oomph for that, unless it’s a very
cheap ARM-based one. (Only the ecryption part remains to be solved with a
ready-made NAS.)

> I just wonder, could I use that board and just hook it to my USB port
> and a external power supply and skip the Raspberry Pi part?  I'd bet not
> tho.  ;-)

From a practical standpoint, what is the difference then to an HDD dock or a
simple USB-SATA-Adapter? Except that a dock is a “proper”, clean solution
with a nice case, a secure stand on your desk and no finnicky open SATA
cables that could cause disconnects during operation if you touch them the
wrong way.

I know what it’s like to ponder all kinds of options, and it’s fun. But it
seems to me, you’re looking for a solution for a problem you’re still
looking for.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

Someone who works a lot makes a lot of mistakes.
Someone who makes no mistakes gets awarded.


signature.asc
Description: PGP signature

Re: [gentoo-user] NAS and replacing with larger drives

[Frank Steinmetzger]

Am Thu, Dec 08, 2022 at 06:36:14PM + schrieb Wols Lists:

> >  > I've pretty much reached a limit on my backups.  I'm up to a 16TB hard
> >  > drive for one and even that won't last long.  Larger drives are much
> >  > more costly.  A must have NAS is quickly approaching.  I've been
> >  > searching around and find some things confusing.  I'm hoping someone can
> >  > clear up that confusion.  I'm also debating what path to travel down.
> >  > I'd also like to keep costs down as well.  That said, I don't mind
> >  > paying a little more for one that would offer a much better option.
> >  >
> >  > Path one, buy a NAS, possibly used, that has no drives.  If possible, I
> >  > may even replace the OS that comes on it or upgrade if I can.  I'm not
> >  > looking for fancy, or even RAID.  Just looking for a two bay NAS that
> >  > will work.  First, what is a DAS?  Is that totally different than a
> >  > NAS?  From what I've found, a DAS is not what I'm looking for since I
> >  > want a ethernet connection and the ability to control things over the
> >  > network.  It seems DAS lacks that feature but not real sure.  I'm not
> >  > sure I can upgrade the software/OS on a DAS either.
> >  > […]
> >
> > DAS is direct-attached-storage. I don't think you want that.
>
> Depends. If it fits in the safe, and can be connected using one of these
> eSATA thingy connectors, it might be a very good choice.
>
> […]
>
> I get the impression Dale isn't actually PLANNING his disk storage. It's
> just a case of "help I'm downloading all this stuff where do I put it!!!"

Haha, thanks for the laugh.

> Get yourself a basic 4-way DAS/JBOD setup, PLAN where you're putting all
> this stuff, and plug in and remove drives as required. You don't need all
> these huge drives if you think about what you're going to do with it all.

That’s actually a good idea. Either use a hot swap frame for an internal 5¼″
PC bay, a desktop dock for bare drives or a multi-bay enclosure. The market
is big, you have lots of choices. USB (with or without integrated hub),
eSATA, one or two bays, etc: https://skinflint.co.uk/?cat=hddocks

Advantages:
- no separate system to maintain just for storage: save $$$, time and power
- very flexible: no chassis limitation on number of disks
- no bulky external enclosures, each using a different power brick and cable
- minimum volume to put into a safe (just get or make a bulk storage case)

Disadvantages:
- not as “fancy” as a NAS
- possibly not all disks can be used at the same time
- physical handling of naked disks takes more care
- LVM is not practical, so use each disk separately
- you gotta remember which files are where¹
- SATA connectors aren’t made for very many insertion cycles (I think the
  spec says 50?), which doesn’t mean they endure much more, but still …

> (And while it takes time and hammers the system, I regularly record off the
> TV getting a 2GB .ts file, convert it to mp4 - same resolution - and reduce
> the size by an order of magnitude - maybe more.

Well, ts uses mpeg2 encoding, just like old video DVDs, which is very
inefficient when compared with modern h264/h265. Modern digital TV broadcast
uses h264 by now.


Incidentally, I got myself a new HDD today: an external 2.5″ WD Passport
Ultra 5 TB with USB-C 3.0. Just because I like portable storage and also
because I need temporary space if I want to convert my NAS RAID-Z2 to Z1.


¹ I do have several external USB disks, plus the big NAS. All of which don’t
run very often. And I don’t want to turn them on just to look for a certain
file. That’s why I have another little script. ;-) It uses the `tree` command
to save the complete content listing of a directory into a text file and
names the file automatically by the name of the directory it crawls. So if I
want to find a file, I just need to grep through my text files.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

The whale is characterised by its bulky form factor.
#!/bin/sh

# history
# 2018-02-10 initial version
# 2020-06-10 added -a option
# -??-?? gzip result
# 2021-03-22 put current date into output filename
# 2021-04-06 code refactoring with more functions and input sanitisation
# 2021-12-22 prefer zstd over gzip
# 2022-01-23 don't create symlink if there is no previous file
# 2022-01-30 added xz to compressors
# 2022-12-08 some cleanup, added -C and -K options

die() {
echo "${@}" > /dev/stderr
exit 1
}

usage() {
cat <<-EOF
Usage: $(basename "$0") [-o NAME] [DIR]
A wrapper to tree, it writes the content of DIR into a text file.
The file is named after DIR and the current date, and symlink to the
most recent version is set. The file is automatically compressed to
zstd, xz or gzip, whichever is available in that order.

Options:
  -aaccess attributes: owner, group, permissions
  -Cdo not compress result file
  -Kdo 

Re: [gentoo-user] NAS and replacing with larger drives

[Frank Steinmetzger]

Am Thu, Dec 08, 2022 at 06:37:52AM -0600 schrieb Dale:
> Howdy,
>
> I've pretty much reached a limit on my backups.  I'm up to a 16TB hard
> drive for one and even that won't last long.  Larger drives are much
> more costly.  A must have NAS is quickly approaching.

Hear hear, ye olde story. ;-)

> Path one, buy a NAS, possibly used, that has no drives.  If possible, I
> may even replace the OS that comes on it or upgrade if I can.

Difficult in consumer-grade stuff, but there are ways, like for Synology:
NetBSD on old Synology hardware:
https://wiki.netbsd.org/ports/sandpoint/instsynology/
And another alternative OS for Synology: https://xpenology.com/forum/

However, even though Synology’s current trend of development is a little
concerning with vendor lock-in and hardware restrictions in their newest
devices, why not use the built-in software? It still is very good and easy
to use and offers all you need like HTTP/HTTPS, FTP, CIFS, SSH, Dav etc. It
uses btrfs or ext4 internally.

Disclaimer: I have no first-hand experience with any of those devices, my
knowledge comes from news about new devices and stuff that I read in a PC
tech forum. Qnap’s software quality does not compete with Synology, and they
also have a worse security track record. So don’t hook it up to the Internet
directly.

> I'm not looking for fancy, or even RAID.  Just looking for a two bay NAS
> that will work.

Why just two? Sooner or later, it will become cramped again. Go for four
bays and leave them empty for the time being.

> First, what is a DAS?  Is that totally different than a NAS?  From what
> I've found, a DAS is not what I'm looking for since I want a ethernet
> connection and the ability to control things over the network.

According to https://en.wikipedia.org/wiki/Direct-attached_storage, you are
right: no ethernet, but direct connection to the host. A beefed-up external
drive enclosure, if you will (from the little understanding I got from the
article).

> It seems DAS lacks that feature but not real sure.  I'm not sure I can
> upgrade the software/OS on a DAS either. 

There is no software, it is just a drive bay and the host that you hook it
up to does all the logic work.

> Next thing.  Let's say a NAS comes with two 4TB drives for a total of
> 8TB of capacity from the factory, using LVM or similar software I
> assume.

AFAIK, consumer NASes don’t use LVM. Probably only standard Raid-1/5/6/10,
JBOD or single disk access.

> Is that limited to that capacity or can I for example replace one or both
> drives with for example 14TB drives for a total of 28TBs of capacity?

Sure, why not? But then I’d buy one without any drives from the start and
install the drives later myself. I wouldn’t know what to do with those small
drives if I replaced them with something larger right away.

> If one does that, let's say it uses LVM, can I somehow move data as well
> or is that beyond the abilities of a NAS?

What do you mean by move? AFAIK, Synology offers SSH access, but I have no
idea what you can do with it in terms of plumbing. And why would you? It is
supposed to do everything under the hood. But as I said, I don’t expect any
of those to use LVM in the first place.

> Could it be done inside my computer for example?

With a DAS, you could. ;-) But if push comes to shove, pull out the drives
and hook them up to your “puter”.


> Path two, I've researched building a NAS using a Raspberry Pi 4 8GB as
> another option.  They come as parts, cases too, but the newer and faster
> models of Raspberry Pi 4 with more ram seem to work pretty well.

Just today, in a forum thread about a new Synology with underwhelming
hardware features, people were posting alternatives. One of them was
https://kubesail.com/homepage. Currently it’s only a small case with
2×2.5″. But they also announced a soon-to-come 5×3.5″.

> The old slower models with small amounts of ram don't fair as well.  While
> I want a descent speed, I'm not looking for or expecting it to be
> blazingly fast.

Only the very old devices with puny ARM chips were so slow they couldn’t
saturate Gbit ethernet—with and without encryption. Synologies of recent
years with a Celeron J4000 will have no problem. Current models with AMD
Ryzen R1600 won’t either, but draw much more power in idle and have no
graphics unit. OTOH, they gain ECC memory support.

> I just wonder, if from a upgrade and expansion point of view, if building
> a NAS would be better.

Regardless of whether DIY or OOTB, a NAS is much more practical than a
collection of external single enclosures. Given the rate of your growth and
need of space, I do recommend some kind of RAID for resilience against hard
disk failure. Does LVM offer this at all? TrueNAS runs from a USB stick and
uses ZFS under the hood.

> I've also noticed, it seems all Raspberry things come with a display port.

My Pi 3B has HDMI – and HDMI only.

> That means I could hook up a monitor and mouse/keyboard when needed.  That
> could be a bonus.  Heck, 

Re: [gentoo-user] Dolphin and adding a option, if it exists.

[Frank Steinmetzger]

Am Wed, Nov 23, 2022 at 06:16:12AM -0600 schrieb Dale:

> >> What does it try to do in simple terms?  Or, how would I use it may be a
> >> better question? 
> > Look at my mail from 23.10., it has a textual description (the one with the
> > file tree): it basically wraps the old script over all files.
> >
> > You have a directory 'A' with videos and a subdirectory 'A/temp' with new
> > videos. Start the script in A and it will go through all files in A/temp and
> > ask you what to do with each – skip (do nothing), keep (move to A without
> > renaming), or overwrite a certain file in A (keeping the extension), which 
> > is
> > the same as the old script.
>
> OK.  I think I get it.  It's kinda like dispatch-conf except it is for
> files not lines in a file.  No 'merge' option tho. Kinda hard to merge a
> video.  lol 
>
> I may try that sometime.  I may copy the files to a different location
> just in case I screw up something. It won't be the real files so no harm
> if I botch it.

Always have a backup. ;-)
I deny any responsibility for data loss. :-P

If you want to try it before doing real stuff: you could comment the two
lines where it says 'mv -f' and 'rm -f'. Then it won’t do anything, but
there are still progress messages being printed.

> P. S.  Now I'm trying to figure out how to change the resolution of all
> videos in a directory.  Usually going from 1080p to 720p.  If you have a
> script for that, awesome.

I use ffmpeg for all my encoding stuff, and have been using wrapper scripts
for years now to make things easier. However, none of them has resized yet.
It’s not difficult to configure, but the wrapper needs much more logic. As
in: find out the current resolution, see if it is actually larger, then
calculate the new resolution while keeping the aspect intact and so on.

> I'm currently using handbrake and it works but there may be a better way. 

I know handbrake by name, have been aware of it for many a year, but never
used it. In my Windows days I used VirtualDub for my editing stuff, and on
Linux I’ve always used commandline tools, beginning with mencoder back in
the day.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

I don’t mind dreaming in a foreign language.
What bugs me are the subtitles!


signature.asc
Description: PGP signature

Re: [gentoo-user] Dolphin and adding a option, if it exists.

[Frank Steinmetzger]

Am Wed, Nov 23, 2022 at 05:54:30AM -0600 schrieb Dale:
> Frank Steinmetzger wrote:
> > There ya goo. As I suspected, I already wrote most of it in October right
> > away, I only couldn’t find the file until now. I just had to fix the same
> > bugs as in the first script (and some more in the additional parts :D ),
> > and put some more grease into the interal logic and output strings.
> >
>
> I saved it and then tried to figure it out.  I got lost.  I see the
> commented sections which kinda help I guess but still got lost. 

I was kinda verbose on the comments for you. But in the end, you don’t need to
understand every line of it. ;-) Suffice it to say it won’t syphon off your
data to me.


> What does it try to do in simple terms?  Or, how would I use it may be a
> better question? 

Look at my mail from 23.10., it has a textual description (the one with the
file tree): it basically wraps the old script over all files.

You have a directory 'A' with videos and a subdirectory 'A/temp' with new
videos. Start the script in A and it will go through all files in A/temp and
ask you what to do with each – skip (do nothing), keep (move to A without
renaming), or overwrite a certain file in A (keeping the extension), which is
the same as the old script.

> I been using the other script and it is coming in very handy.  It saves
> me quite a bit of time. 

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

When I’m old I’ll do nothing but complain. That’ll be a blast.


signature.asc
Description: PGP signature

Re: [gentoo-user] Dolphin and adding a option, if it exists.

[Frank Steinmetzger]

Am Tue, Nov 22, 2022 at 06:17:09AM -0600 schrieb Dale:
> Frank Steinmetzger wrote:

> >> I added a little . on that last line before the extension bit.  I'm a
> >> happy camper.
> > Give me a nudge if you want the more luxurious version with interactive
> > selection of the overwrite destination. I think I already started a
> > prototype somewhere, but can’t find it right now.
> >
>
> Feel free to share.  Even if I don't use it, someone else may find it
> and make good use of it.


There ya goo. As I suspected, I already wrote most of it in October right
away, I only couldn’t find the file until now. I just had to fix the same
bugs as in the first script (and some more in the additional parts :D ),
and put some more grease into the interal logic and output strings.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

“To some degree people say you should not micro-optimise.
But if what you love is micro-optimisation, that’s what you should do.”
– Linus Torvalds
#!/usr/bin/env bash

# author: Frank Steinmetzger
# version: 2022-11-23

# Helper functions 

# oops, something occured and the script must exit with an error message
die() {
echo "$PROGNAME error: ${@}" > /dev/stderr
exit 1
}

# move a file after printing an info text
move() {
echo "  Moving '${1##*/}' -> '$2'"
mv -f "$1" "$2"
}

# print help text
usage() {
cat <<-EOF
$PROGNAME: replace files in one dir with files from another, but 
keeping the extension

Synposis: $PROGNAME [-h|-s DIR]
  -h  show this help and exit
  -s  the source directory from which to pick files. Default: 
./$SRC_DIR
EOF
}

# Preamble 

PROGNAME="$(basename "$0")"

# halt script on any uncaught error
set -e

# the default directory
SRC_DIR="temp"

# parse arguments
while getopts "hs:" OPTION; do
case $OPTION in
h) usage; exit 0 ;;
s) SRC_DIR="$OPTARG" ;;
*) die "Unknown option '$OPTION'" ;;
esac
done

# input sanitisation
[ -d "$SRC_DIR" ] || die "'$SRC_DIR' is not a directory or does not exist."
[ "$(readlink -m "$SRC_DIR")" = "$(readlink -m .)" ] && die 'Source dir equals 
destination dir.'

# generate list of possible destination files
declare -A DST_FILES
for DST_FILE in *; do
[ -f "$DST_FILE" ] || continue
DST_FILES["$DST_FILE"]=''
done

# Main part ---

# loop over all files in the source dir
for SRC_PATH in "$SRC_DIR"/*; do
[ "$SRC_PATH" = "$SRC_DIR/*" ] && die 'Source dir is empty.'

SRC_FILE="${SRC_PATH##*/}"

# selection of destination file
echo
echo "Select destination for file '$SRC_FILE':"
unset DST_FILE
select DST_FILE in '[-skip-]' '[-keep as is-]' "${!DST_FILES[@]}"; do
[ "$DST_FILE" ] && break
done

# don't do anything with the file
if [ ! "$DST_FILE" ] || [ "$DST_FILE" = "[-skip-]" ]; then
echo "  Skipping '$SRC_FILE'"
continue
fi

# move the file to destination without renaming
if [ "$DST_FILE" = "[-keep as is-]" ]; then
if [ -f "$SRC_FILE" ]; then
echo -n "The file '$SRC_FILE' already exists. Overwrite? [y/N] "
read INPUT
if [ "$INPUT" != "y" ] && [ "$INPUT" != "Y" ]; then
echo "  Skipping '$SRC_FILE'"
continue
fi
fi
move "$SRC_PATH" "./$SRC_FILE"
else
# replace destination with source file
SRC_EXT="${SRC_FILE##*.}"
DST_BASE="${DST_FILE%.*}"
NEW_DST_FILE="${DST_BASE}.${SRC_EXT}"
if [ "$DST_FILE" != "$NEW_DST_FILE" ]; then
echo "  Removing '$DST_FILE'"
rm -f "$DST_FILE"
fi
move "$SRC_PATH" "./$NEW_DST_FILE"

# remove the destination file from future selections
unset DST_FILES["$DST_FILE"]
fi
done


signature.asc
Description: PGP signature

Re: [gentoo-user] Dolphin and adding a option, if it exists.

[Frank Steinmetzger]

Am Tue, Nov 22, 2022 at 03:59:27AM -0600 schrieb Dale:

> > If you still want to stick to a terminal solution akin to mv, then there is
> > no way around a little script which wraps mv by extracting the extension and
> > filename base. You could also add some “intelligence” with regards to
> > directories, in order to reduce the amount of effort required to use the
> > command—in case your directories follow some schema or are constant.
> >
> >
> > #!/usr/bin/sh
> >
> > [ "$#" -ne "2" ] && exit 1
> > SRC="$1"
> > DST="$2"
> >
> > SRC_EXT="${SRC##*.}"
> > DST_BASE="${DST%.*}"
> >
> > # remove destination for the case that the extensions differ
> > rm "$DST"
> >
> > mv "$SRC" "${DST_BASE}${SRC_EXT}"
> >
> 
> I finally got a chance to try this.  I saved it and made it executable. 
> It runs but gave me this error. 
> 
> 
> dmv torrent/video_name-old-place.mp4 video-name-new-place.mp4
> bash: /bin/dmv: /usr/bin/sh: bad interpreter: No such file or directory
> dale@fireball ~/Desktop/Crypt/Series $
> 
> 
> My scripting skills are minimal at best.  Still, I kinda got what your
> script was doing.  Those who have known me for a while understand how
> miraculous that is.  ROFL  I did some googling.  It seems to not be able
> to find the 'shebang' part.  Sure enough, sh isn't located in /usr/bin
> here.  It's in /bin tho.  I edited that line so it can find it.  When I
> tried it, it worked but noticed another problem. […]

Well, it would have been boring to provide you with a turn-key solution. ;-)
Congrats on getting it working. In my Arch setup, sh is in /usr/bin. A
flexible solution is to use #!/usr/bin/env sh, which looks the command up
before executing it.

> I added a little . on that last line before the extension bit.  I'm a
> happy camper.

Give me a nudge if you want the more luxurious version with interactive
selection of the overwrite destination. I think I already started a
prototype somewhere, but can’t find it right now.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

The best thing about Sundays is Saturday evening.


signature.asc
Description: PGP signature

Re: [gentoo-user] Re: Upgrading from 5.14 to 6.0 version

[Frank Steinmetzger]

Am Mon, Nov 21, 2022 at 07:37:01PM - schrieb Grant Edwards:

> >> My office setup had three screens, each with four virtual desktops.
> >> 
> >> When using multiple screens, you develop the habit of using one screen
> >> for common, always-on stuff (e.g. email, web browser) and the other
> >> screen(s) for working on code (or whatever).
> >
> > I found Enlightenment to be most versatile in this respect.  Unlike
> > say Plasma, which has two monitors locked on the same virtual
> > desktop and when you switch to another virtual desktop *both*
> > monitors flip over,
> 
> That's how all of virtual-desktop window managers I've tried over the
> decades work.

As a workaround within Plasma: set the window on your static monitor to show
on all virtual desktops. It won’t even animate when you change the desktop.
I use this feature mostly with video players, and even set a window rule to
be applied automatically upon launching a player.

> > in Enlightenment each monitor can switch to a different virtual
> > desktop independently.

Awesome WM also does this independently on separate screens. Just tested it.

-- 
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

Gentoo Linux is a rainbow with no end and no pot of gold.


signature.asc
Description: PGP signature

Re: [gentoo-user] Dolphin and adding a option, if it exists.

[Frank Steinmetzger]

Am Sun, Oct 23, 2022 at 06:16:04AM -0500 schrieb Dale:
> Frank Steinmetzger wrote:
> > Am Sun, Oct 23, 2022 at 01:35:55AM -0500 schrieb Dale:
> >
> >> Well, I ran into a slight problem.  This isn't much of a problem with
> >> Linux but I'm not sure how this would work on windoze tho.  The problem,
> >> if it is one, is the file extension.  Let's say I have a mp4 file that
> >> is the older original file that I intend to replace.  If the file I
> >> intend to put in its place is a .mkv file, mv uses the .mp4 extension
> >> because all it cares about is the name of the file, not what it is or
> >> its content.  So, I end up with a .mkv file that has a .mp4 extension. 
> >> It works here on Linux but not sure about windoze and such.
> > It’s not a problem for as long as the application you open the file with
> > does its own detection. I.e. you feed mp4 to mpv, but it recognises by
> > itself that it’s mp4 and can handle it.
> 
> That is true on Linux.  Most linux software could care less what the
> extension is or if it even has one.

Mpv or Vlc on Windows will probably just work™, too.

> Heck, you could likely change a
> .mp4 to .txt and it would open with a video player just by clicking on
> it.  Thing is, if I share a file with someone who uses windoze, I'm not
> sure if it would work the same way.  A wrong extension could cause
> problems, either not opening at all or crashing something.  It's
> windoze, one can't expect much.  ROFL 

Now you’re talking about double-clicking in a file manager and open the
registered application. That’s the same—to some extent—on Linux file
managers. I was referring to an application that could work out the details.

> I thought about looking to see if there is a way to "scan" a directory
> and look at each file and if needed, change the extension to the correct
> one.  Thing is, I couldn't write a fancy script if my life depended on
> it.  I also looked into using Krename to do it but it refuses to change
> a extension.  Doing it one file at a time manually puts me back to where
> it is easier to change the file the old way.  Time consuming but works. 

Well, ther is the `file` tool, plus maybe `mediainfo` or `identify` for
images. But their output may not always be sufficient.

> > If you still want to stick to a terminal solution akin to mv, then there
> > is no way around a little script which wraps mv by extracting the
> > extension and filename base.

> H.  I get a little of that but then I get lost.

The script first checks wheter it receives exactly two arguments, and exits
otherwise. In theory it should also check whether both paths exist and are
files. First rule of programming: always sanitise your inputs!

Now it gets the extension of the source file and the base part (i.e.
everything without the extension) from the destination. Then it deletes the
original destination file and finally moves the source by concatenating the
original destination’s base part with the source’s extension part.


>   Just how does that work and how would I use it?

I have a lot of little helper scripts. I collect them in ~/usr/bin, to which
my PATH is expanded in ~/.bashrc with export PATH=~/usr/bin:$PATH. Actually,
I keep the script files in git repositories under ~/dev, and then put
symlinks into ~/usr/bin, which point to the repository file.

> I think I would save that as a file, make it executable and then run it
> with whatever name I give it.

Exactly.

> I'm not sure exactly how to tell it what files to move tho.  Same as mv
> maybe? 

Yes. You give it two arguments. That’s what $1 and $2 are for in the script.
I always write my scripts so that they can handle spaces in filenames. I
find it an anachronism to still use underscores or dots in filenames where
spaces would go in normal language. File systems have been able to deal with
spaces for decades now.

> Currently, I move to the main directory that files are in when I am in
> Konsole and running as my user, so file permissions don't switch to root. 

That’s the proper way to do it. I also have a root console open all the
time, but don’t do normal file operations in there. The risk is too big that
I may be typing too fast for my own good.

> My process on file organizing goes a little like this.  I have a set of
> videos that go together.  When I have a new version of one or more videos,
> I place them in a sub-directory until they are named properly or something
> so I can move to the main directory.  Like this:
> 
> Main Directory  #Permanent location for files - Sub-directory 
> #Temporary location for files needing names changed etc.  Once done, they
> move up to main directory.

I don’t quite understand the formatting of that line. But basically, you
have a directory for your videos