Re: [gentoo-user] Video database software

[Laurence Perkins]

On Mon, 2019-01-28 at 16:54 -0600, Dale wrote:
> Howdy,
> 
> As some know, I've accumulated a lot of videos.  I googled around and
> found some software but not sure based on what they claim if they
> will
> do something I'm looking for.  I installed a couple but not real big
> on
> how they work.  One requires me to add videos, one at a time.  I have
> over 20,000 videos now.  Some are short youtube type videos, some are
> long videos.  It could take me years to add them all doing it one at
> a
> time.  Needless to say, that one didn't last long.  The biggest thing
> I'm looking for, software that can tell me what videos have a low
> resolution.  As a example, some videos I downloaded a long time ago
> have
> been updated to have higher resolutions.  I may have one that is a
> 360P,
> or even less, but I may can locate a new version that is HD or even
> very
> HD.  I'd like software that will tell me this sort of info as well as
> other nifty features as well.  Obviously, I'd like to start with the
> lower resolution videos first.
> 
> So far, I have installed Griffith and GCStar.  I been googling for
> others but some either are not in the tree or I already know they
> won't
> do one thing I'd like to see.  I'd also like to be able to point it
> to a
> directory and let it build the database on its own.  Adding them one
> at
> a time manually just isn't feasible at all. 
> 
> Does anyone know of a software package that will sort a lot of videos
> by
> resolution as well as track other things as well?  It could be that
> what
> I'd like to have doesn't exist at all.  Then again, maybe I just
> haven't
> found it yet.  ;-)
> 
> Thanks.
> 
> Dale
> 
> :-)  :-) 
> 

Digikam might be worth a try.  It does some pretty advanced stuff with
photos and I think at this point it can do quite a bit of the sorting
and searching stuff with videos as well.  I've not tried it with videos
myself though so I can't make any promises.

LMP

Re: [gentoo-user] Video database software

[Laurence Perkins]

On Mon, 2019-01-28 at 18:01 -0600, Dale wrote:
> 
> I do currently use exiftool to get the resolution.  One, it is
> accurate
> and true every time.  I've never had it be wrong.  I do it this way: 
> exiftool  | grep size   

Assuming you're using bash as your shell then you're pretty close
already. Make it:
echo ", $(exiftool  | grep size)"

Then you just need to run that over every file and append the output to
a data file for searching.  There are a variety of ways to do that
ranging from using loops in your script to doing abominable things with
spreadsheet programs or other editors, so I leave that choice up to
you.

LMP

Re: [gentoo-user] Video database software

[Laurence Perkins]

On Tue, 2019-01-29 at 17:57 +, Mick wrote:
> On Tuesday, 29 January 2019 02:55:02 GMT Dale wrote:
> > Andrew Udvare wrote:
> > > > On 2019-01-28, at 17:54, Dale  wrote:
> > > > 
> > > > So far, I have installed Griffith and GCStar.  I been googling
> > > > for
> > > > others but some either are not in the tree or I already know
> > > > they won't
> > > > do one thing I'd like to see.  I'd also like to be able to
> > > > point it to a
> > > > directory and let it build the database on its own.  Adding
> > > > them one at
> > > > a time manually just isn't feasible at all.
> > > 
> > > Seems like you could import via command line?
> > > http://wiki.gcstar.org/en/execution
> > > 
> > > You can build the database you need locally with something like
> > > exiftool
> > > or MediaInfo, or even ffmpeg https://stackoverflow.com/a/8191228/
> > > 374110 .
> > > I highly doubt anyone with serious collections is building their
> > > database
> > > one item at a time.> 
> > > > Does anyone know of a software package that will sort a lot of
> > > > videos by
> > > > resolution as well as track other things as well?  It could be
> > > > that what
> > > > I'd like to have doesn't exist at all.  Then again, maybe I
> > > > just haven't
> > > > found it yet.  ;-)
> > > 
> > > The closest thing I can think of is Kodi since it's scanner will
> > > retrieve
> > > all this information and store it in a straightforward database
> > > format.
> > > You can choose SQLite or MySQL (of course MySQL is definitely the
> > > better
> > > choice for larger collections). The downside is the scanner is
> > > very slow,
> > > especially over a network (and not optimised). The only viewer
> > > for this
> > > data (at the time being) is Kodi itself.
> > 
> > Not ignoring.  Just pondering this one.  May take some time for me
> > to
> > test some stuff here.  ;-) 
> > 
> > Thanks much.
> > 
> > Dale
> > 
> > :-)  :-) 
> 
> Installing and having to maintain Kodi just to manage a list of
> videos is 
> probably inefficient - unless you have a regular use for other Kodi 
> functionality.  I use it mostly for audio and also the odd video.  It
> has 
> loads of useful plugins to play with.
> 
> If Kodi is of no use, or you prefer a more portable stand alone CLI
> solution, 
> you could look into some basic bash scripts. I couldn't code my way
> out of a 
> paper bag, but here's two basic ideas to get you started.  First to
> list all 
> the videos into a csv file:
> 
> find . -xtype f -iname '*.mp4' -o -iname '*.avi' -o -iname '*.mkv' > 
> video_list.csv
> 
> You may have to add other types of video file containers depending on
> your 
> video collection.  As a second step, in order to list all the video 
> resolutions you could pass the find output to xargs:
> 
> find . -xtype f -iname '*.mp4' -o -iname '*.avi' -o -iname '*.mkv' |
> tee 
> video_list.csv | xargs -d '\n' exiftool -T -ImageSize
> 
> Given my non-existent coding skills I am not sure how to append the
> output of 
> xargs as a second column to the video_list.csv, which you could
> thereafter 
> open with localc to do your searches, or manipulate further.  Of
> course, 
> localc is not necessary.  You can always use less or grep to search
> the csv 
> file very efficiently and also re-create it quickly when you
> add/delete to 
> your videos.
> 
> Other more knowledgeable contributors should be able to polish and
> complete 
> the above, or indeed propose something different than bash (python?)
> to 
> perform the same task.
> 
> HTH.


Nah, bash works fine and is less verbose when interacting with system
utilities.  

To meld it all together use a for loop:

#! /bin/bash
#Top line lets you save it as a file and run it if you want.
#Or you can run it a line at a time in your shell.  Bash isn't picky.
IFS="
"  #This bit tells it to only count new lines as new entries.  
   #It's only necessary if your file names have spaces in them.

#This assumes all your videos have proper file extensions.  If
#not then you'll want to make use of the "file" utility to determine
#the type of your files and use grep to sort out which ones are
#videos and then run that list through the for loop instead.
for VIDEO in $(find . -xtype f -iname '*.mp4' -o -iname '*.avi'\
-o -iname '*.mkv'); do
  #Things in $() get run and their stdout gets stuffed into the 
  #command line at that point. ${} is how you insert variable values.
  echo "${VIDEO},$(exiftool -T -ImageSize '${VIDEO}')" 
done

The bit with the backslashes at the end of the lines makes it not count
the newline as the end of a command so it will hopefully go through the
mail without getting too mangled.  You should be able puzzle out how to
fix it if it does.


LMP

signature.asc
Description: This is a digitally signed message part

Re: [gentoo-user] VRFs / Jails / Containers

[Laurence Perkins]

On Sat, 2019-02-02 at 19:32 -0700, Grant Taylor wrote:
> Does Gentoo have any support for VRFs or (chroot) Jails or
> Containers 
> without going down the Docker (et al) path?
> 
> I'm wanting to do some things with a Gentoo router that is trivial to
> do 
> with network namespaces via manual commands ~> scripts.  But that's
> far 
> from standard Gentoo init script based system.  And I'd like
> something 
> more Gentoo standards based.
> 
> Does Gentoo have or support anything like this natively?  Or am I 
> getting into territory where I'm rolling my own?
> 

Have you tried firejail?  It gives you convenient ways to set up the
container parameters consistently and is in the repo.  Its invocation
is also simple enough to not clutter up your startup scripts.

LMP

Re: [gentoo-user] Coming up with a password that is very strong.

[Laurence Perkins]

On Sun, 2019-02-03 at 23:47 -0600, Dale wrote:
> 
> 
> How do you, especially those who admin systems that are always being
> hacked at, generate strong passwords that meet the above?  I've
> googled
> and found some ideas but if I use the same method, well, how many
> others
> are using that same method, if you know what I mean.  ;-)  Just
> looking
> for ideas. 
> 
> Thanks much.
> 
> Dale
> 
> :-)  :-) 
> 
> P. S.  I haven't had time to deal with the video thing in previous
> thread.  It's on my todo list still.  :-( 
> 

Take 80 to 100 characters of something you already have memorized. 
Poetry, bible verses, RFCs, pages of the phone book, digits of pi out
of the middle, whatever.  Run it through a transposition, substitution,
or combination cipher that you can calculate in your head on-the-fly. 
(Do avoid the substitutions that everyone uses since those will be
tried first.)

Now you only need to remember a pointer to the memorized section, the
length, and the cipher specification.  There are enough possible
combinations that an attacker won't be able to make a meaningful
reduction in entropy by examining your social media.

As an example:  The second paragraph of Hamlet's soliloquy and invert
the case based on whether the corresponding digit of e is odd or even.

LMP

Re: [gentoo-user] Emerge --sync source

[Laurence Perkins]

On Fri, 2019-03-01 at 10:12 +, Peter Humphrey wrote:
> On Thursday, 28 February 2019 15:47:41 GMT Rich Freeman wrote:
> 
> > In general it is usually simplest to just remove /usr/portage
> > anytime
> > you change the sync settings.  At least until portage gets smarter
> > about it.
> 
> That works well on a sufficiently powerful box; it only took - oh, I
> don't 
> know - maybe a couple of minutes on this workstation. On my little
> Atom box, 
> though, it takes 75 minutes.
> 
> [OT]
> Evidence is mounting that the Atom box is in terminal decline. I get
> things 
> like batches of files in the portage tree changing owner, and then
> when I 
> correct that, long lists of supposedly locally changed ebuilds
> preventing 
> syncing. And when I boot weekly into its little rescue system to
> backup the 
> main system, the root filesystem remounts itself read-only while tar
> is 
> running. Smartd recognises the SSD and runs daily tests, but reports
> no 
> errors. No amount of wiping and reinstalling has helped so far.
> 
What filesystem are you running and how old is the SSD?  That sounds
like some of the symptoms EXT4 had on early generation flash media
where its assumptions about what order writes would physically make it
to the disk in were wrong, leading to corruption.  So unless it was
working correctly at some point in the past, try a different
filesystem.  EXT3 or BTRFS didn't have the same problems.

If it's just that the SSD is failing, then get a new one before
something important gets damaged and you have to redo the whole thing. 
The self-test capability of storage media is almost universally
horrible and you generally don't get a failure report until your data
has already been lost.  If your SMART output gives you the raw
statistics on the device instead of just pass/fail then analyzing that
usually gives a better indication of whether something is about to go
wrong.

LMP

signature.asc
Description: This is a digitally signed message part

Re: [gentoo-user] Quad UART PCIe Adapter (Oxford-Chipset) seems (not?) to work? Check?

[Laurence Perkins]

> You can get pin status with
>  statserial /dev/ttyS9
> 

I hate to butt in, but statserial seems like it would be quite
useful...  And yet I can't figure out which package it's in on Gentoo. 
It's not in setserial, doesn't have its own package, and isn't
discoverable via eix or pfl...  Is there an ebuild hidden somewhere? 
Or do I need to hunt it down by hand?

LMP

signature.asc
Description: This is a digitally signed message part

Re: [gentoo-user] Quad UART PCIe Adapter (Oxford-Chipset) seems (not?) to work? Check?

[Laurence Perkins]

On Mon, 2019-03-25 at 16:58 -0500, Dale wrote:
> Laurence Perkins wrote:
> > > You can get pin status with
> > >  statserial /dev/ttyS9
> > > 
> > I hate to butt in, but statserial seems like it would be quite
> > useful...  And yet I can't figure out which package it's in on
> > Gentoo. 
> > It's not in setserial, doesn't have its own package, and isn't
> > discoverable via eix or pfl...  Is there an ebuild hidden
> > somewhere? 
> > Or do I need to hunt it down by hand?
> > 
> > LMP
> 
> I tried to figure out what package has it but am having no luck.  I'm
> wondering if a USE flag is required to enable this to be included? 
> If
> that USE flag is disabled by default, then it wouldn't be available
> on
> most systems unless someone knew what it was and enabled it.  If it
> is a
> USE flag, I'm having no luck finding it either. 
> 
> Dale
> 
> :-)  :-) 
> 

If you installed it via portage then doing:
qfile `which statserial` 
should tell you what package it came from.  

If that finds nothing then it got installed by some other means and
probably isn't packaged for Gentoo.


LMP

Re: [gentoo-user] USB external hard drives

[Laurence Perkins]

On Sun, 2019-04-07 at 12:09 -0400, james wrote:
> Hello,
> 
> I have a windows pro 10, bargain HP laptop, with a 1 T mechanical
> internal drive. New. I'd like to make the system dual boot off of
> an external (USB3) SSD, that is exclusive for Gentoo. I have about
> 500M of person and /usr/local files on other gentoo systems that I'd
> like to DD over to the new drive.
> 
> 1. I'm not sure what file system to use. I' like to use btrfs, but
> I'd need some sort of guide to set that up.
> 
> OR
> 
> 2. If I used ext-4 what should the formatting and partition table
> look
> like?  Ideally, I'd like to put the 'gentoo on a partition to the
> native
> hard drive. and be able to put /usr/local and /projects only on the
> the new SSD external only, so I can move those mount point partitions
> to various gentoo systems.
> 
> 
> Perhaps I'll get (2) external SSDs and put gentoo on one, just for
> the
> HP laptop and the second one set up just for /usr/local and
> /projects.
> 
> 
> I'm just thinking out loud and ideas or discussions are welcome. I'm
> not
> the sharpest tack, when in comes to windows pro 10. or complex
> file
> system setups.
> 
> All my systems are AMD64, the new laptop is Rizen 5 2500-U
> 
> 
> James
> 

There are not really any differences to how you'd format an external
disk compared to an internal one on modern systems.  If you want
different sections of the system on different drives you just set that
up in fstab so the system attaches everything to the proper places on
boot.  Having your linux on a separate disk that you can disconnect
when using Windows has the advantage of avoiding Windows' propensity to
scribble all over things it really shouldn't be touching, like your
boot setup.

Btrfs/ZFS have some advantages compared to more traditional
filesystems, but they are slower.  It is handy to be able to just plug
in another drive and add it to the storage pool if you temporarily need
more space for some project though, and depending on what kind of data
you spend most of your time shuffling the built in compression option
can improve performance quite a bit.  Ext4 on the other hand has a
longer track record and supports built-in encrpytion.  So compare the
features and see which fits your use case better.  Btrfs is pretty
stable at this point, I use it almost exclusively and haven't had it
eat any data in years, plus the data checksums have saved me from
corruption a few times when drives went marginal but didn't fail
outright, but you have to have it keeping multiple copies for that to
work (Raid or dup.)

LMP


signature.asc
Description: This is a digitally signed message part

[gentoo-user] Decent single-user/embedded-device security standard

[Laurence Perkins]

When the security auditors come through and ask what standard I use for
securing my systems I'd like to have something to tell them.

I've had a few suggestions like USGCB, etc.  But looking at them they
all seem to start from the direction of "take a bloated, wide-open
Microsoft/Redhat default OS and do these things to make it 'secure' so
you can let several dozen users play around on it without fear."

A lot of the stuff on the list doesn't apply to or would slightly
reduce the overall security of the device (I think I'll keep my default
umask at 077 thanks...)

I'm hoping somebody here knows of a commonly used security
specification for bottom-up minimal systems so I can minimize the time
I have to waste explaining that it simply doesn't have a print server,
email server, cifs server, etc., (or even any way for any user to
obtain shell access without first being in possession of administrator-
level credentials) and that half to two-thirds of the checklist doesn't
even apply.

LMP


signature.asc
Description: This is a digitally signed message part

Re: [gentoo-user] Decent single-user/embedded-device security standard

[Laurence Perkins]

>You could still use USGCB (or which ever standard the auditors regard highly) 
>but then document the differences with a
>note explaining why. For USGCB I'd add another column to the spreadsheet with 
>options of compliant/non compliant with
>mitigations/non compliant/not applicable and another column for notes. eg 
>umask 077 would be compliant, and in the
>notes column "stricter than required".
>
>From their point of view they need to justify passing you, and USGCB states 
>"these recommendations do not address
>site-specific configuration issues. Care must be taken when implementing these 
>settings to address local operational
>and policy concerns" so deltas are expected. Don't worry if it seems like its 
>all deltas...

Yeah, that was the fallback option.  I was just hoping there was something in 
reasonably common usage that wouldn't end up being 60% deltas and didn't look 
like it was compiled by a practitioner of voodoo instead of someone who 
actually understands how the system works.


From: Adam Carter 
Sent: Wednesday, July 10, 2019 5:27:55 PM
To: gentoo-user@lists.gentoo.org
Subject: Re: [gentoo-user] Decent single-user/embedded-device security standard

On Thu, Jul 11, 2019 at 9:30 AM Laurence Perkins 
mailto:lperk...@openeye.net>> wrote:
When the security auditors come through and ask what standard I use for
securing my systems I'd like to have something to tell them.

I've had a few suggestions like USGCB, etc.  But looking at them they
all seem to start from the direction of "take a bloated, wide-open
Microsoft/Redhat default OS and do these things to make it 'secure' so
you can let several dozen users play around on it without fear."

A lot of the stuff on the list doesn't apply to or would slightly
reduce the overall security of the device (I think I'll keep my default
umask at 077 thanks...)


You could still use USGCB (or which ever standard the auditors regard highly) 
but then document the differences with a note explaining why. For USGCB I'd add 
another column to the spreadsheet with options of compliant/non compliant with 
mitigations/non compliant/not applicable and another column for notes. eg umask 
077 would be compliant, and in the notes column "stricter than required".

>From their point of view they need to justify passing you, and USGCB states 
>"these recommendations do not address site-specific configuration issues. Care 
>must be taken when implementing these settings to address local operational 
>and policy concerns" so deltas are expected. Don't worry if it seems like its 
>all deltas...

Re: [gentoo-user] Re: escape from i3lock

[Laurence Perkins]

You could also leave DontVTSwitch on all the time and set a keyboard
shortcut to run chvt (man 1 chvt) with appropriate permissions and
parameters instead.  Keyboard shortcuts shouldn't get processed if the
screen is locked.

LMP

On Thu, 2019-07-11 at 21:01 +, artur.tamm...@gmail.com wrote:
> I tried to google if it is possible to change xorg serverflags in
> runtime,  
> but was unable to find anything. I think that would be a cleaner
> solution  
> (changing the DontVTSwitch option before locking and then restoring
> later).
> 
> Artur
> 
> Ian Zimmerman writes:
> 
> > On 2019-07-11 09:57, Ian Zimmerman wrote:
> > 
> > > > setxkbmap -option srvrkeys:none
> > > > i3lock -c 003355 -n
> > > > setxkbmap -option ''
> > > 
> > > Thanks for the idea!  It won't work as is for me because I
> > > already use
> > > some non-default xkb options.  But it is closer than anything
> > > that has
> > > come up yet.  I'll get there.
> > 
> > Okay, I got it to work in a brute force way: I just added another
> > setxkbmap command to set my normal options, the same ones as in my
> > xorg.conf.
> > 
> > But something weird happens when I try the fancy way: saving the
> > options
> > with "setxkbmap -print >FILE" and then restoring them with "xkbcomp
> > FILE".  It seems that the change I make with "setxkbmap -option
> > FOO" is
> > never reflected in the output of "setxkbmap -print".
> > 
> > Looks like another place with multiple "levels" of configuration
> > stepping over each other.
> > 
> > --
> > Please don't Cc: me privately on mailing lists and Usenet,
> > if you also post the followup to the list or newsgroup.
> > To reply privately _only_ on Usenet and on broken lists
> > which rewrite From, fetch the TXT record for no-use.mooo.com.
> > 


signature.asc
Description: This is a digitally signed message part

Re: [gentoo-user] Re: escape from i3lock

[Laurence Perkins]

> So the solution is to just use "xscreensaver" by jwz. Which can be
> configured to just blank the screen etc. as wanted by the op. See
> also
> the FAQ: https://www.jwz.org/xscreensaver/faq.html
> 
> HTH,
> -dnh
> 

Except I use xscreensaver myself and it in no way prevents VT switch,
which is what the OP was hoping to find a way to do if and only if the
screen is locked.  Programs that grab input still don't get to block
combos that are processed by the X server before they even get to the
program's input queue any more than grabbing input will block the alt-
sysrq kernel-level interrupt keys.

Disabling VT switch by the X server and then setting up some other way
to trigger a switch that will be blocked by whatever screen locking
program the OP wishes to use is the best solution I can think of.  chvt
would be the program to call.  Whether he wants it to be a keyboard
shortcut handled by his DE or some other trigger method is a matter of
taste.

LMP


signature.asc
Description: This is a digitally signed message part

Re: [gentoo-user] Re: 2 months into an 8-month computation.

[Laurence Perkins]

On Fri, 2019-07-12 at 07:18 +0300, Nikos Chantziaras wrote:
> On 11/07/2019 20:59, Alan Grimes wrote:
> > 'ey, I have the 2.3 months into an 8-month computation blues...
> > [...]
> > So basically all gentoo updates will have to be done at the end of
> > this 
> > run, I'm not really sure when, sometime in the December-January
> > timeframe.
> 
> I guess you should have written your code in a way that can store 
> current state so that it can resume. Failing that, you could have
> used a 
> VM that can save ("suspend") the guest state so that you can resume
> later.
> 
> Food for thought for the future, I guess :-)
> 
> 
If he wants to live dangerously he could try sys-process/criu...

Probably would want to spin up another instance of the computation and
test with that and make sure it works correctly first.

LMP


signature.asc
Description: This is a digitally signed message part

Re: [gentoo-user] Re: escape from i3lock

[Laurence Perkins]

On Fri, 2019-07-12 at 09:01 -0700, Ian Zimmerman wrote:
> On 2019-07-11 21:28, Nuno Silva wrote:
> 
> > vlock -n -a
> 
> Does vlock work from an XWindow session?  Or would I have to use it
> on
> top of whatever I do to lock the XWindow session -
> xscreensaver/i3lock
> etc?
> 
> (I browsed to the vlock README page on github but it doesn't answer
> this
> question.)
> 
vlock -a is supposed to lock all virtual consoles.  The -n makes it
create a new vt, so it'll work even from within a running X session and
it seems to block the ability to switch back to the X session until the
system is unlocked.  But -n also requires root privileges.  Might also
want to throw in a -s to temporarily disable alt-sysrq.

This can basically take the place of your current screen locker and
does a better job than even xscreensaver of locking out the system.




signature.asc
Description: This is a digitally signed message part

Re: [gentoo-user] amanda-3.4.5

[Laurence Perkins]

On Mon, 2019-07-15 at 08:56 +0200, Stefan G. Weichinger wrote:
> Am 15.07.19 um 08:41 schrieb Stefan G. Weichinger:
> > Does anyone have a binary package for app-backup/amanda-3.4.5 he
> > could
> > share?
> > 
> > I have to (test the) downgrade because of issues with 3.5.1, and my
> > binary package doesn't install anymore (perl now upgraded etc)
> > 
> > and the latest amanda doesn't talk to a legacy client which I can't
> > upgrade at all
> > 
> > Maybe someone could share the package via dropbox or so.
> 
> managed a workaround:
> 
> the compile issues (related to NDMP and RPC somehow) showed me that
> the
> USE-flag "ndmp" isn't interpreted by the current ebuild at all.
> 
> Patched that and disabled NDMP, now it compiles and amcheck works ok
> for
> now.
> 
> 
You should file a bug report on that if you haven't already.  USE flags
to nowhere isn't something anybody wants.

LMP


signature.asc
Description: This is a digitally signed message part

Re: [EXTERNAL] Re: [gentoo-user] multipath.conf : learning how to use

[Laurence Perkins]

On Wed, 2019-08-14 at 20:20 +0200, J. Roeleveld wrote:
> On woensdag 14 augustus 2019 14:17:23 CEST Stefan G. Weichinger
> wrote:
> > Am 14.08.19 um 13:20 schrieb J. Roeleveld:
> > > See next item, make sure you do NOT mount both at the same time.
> > 
> > I understand and agree ;-)
> 
> good :)
> 
> > > > # /usr/bin/sg_vpd --page=di /dev/sdb
> > > > 
> > > > Device Identification VPD page:
> > > >   Addressed logical unit:
> > > > designator type: NAA,  code set: Binary
> > > > 
> > > >   0x600605b00d0ce810217ccffe19f851e8
> > > 
> > > Yes, this one is different.
> > > 
> > > I checked the above ID and it looks like it is already correctly
> > > configured. Is " multipathd " actually running?
> > 
> > no!
> 
> Then "multipath -l" will not show anything either. When you have a
> chance for 
> downtime (and that disk can be umounted) you could try the following:
> 1) stop all services requiring that "disk" to be mounted
> 2) umount that "disk"
> 3) start the "multipath" service
> 4) run "multipath -ll" to see if there is any output
> 
> If yes, you can access the "disk" via the newly added entry under
> "/dev/
> mapper/"
> If you modify "/etc/fstab" for this at the point, ensure multipath is
> started 
> BEFORE the OS tries to mount it during boot.
> 
> Other option (and only option of "multipath -ll" still doesn't show
> anything) 
> is to stop the "multipath" service and leave it all as-is.
> 
> > > If it were running correctly, you would mount " /dev/mapper/
> > > " instead
> > > of " /dev/sdc " or " /dev/sdd ".
> > > 
> > > > In the first week of september I travel there and I have the
> > > > job to
> > > > reinstall that server using Debian Linux (yes, gentoo-users, I
> > > > am
> > > > getting OT here ;-)).
> > > 
> > > For something that doesn't get updated/managed often, Gentoo
> > > might not be
> > > the best choice, I agree.
> > > I would prefer Centos for this one though, as there is far more
> > > info on
> > > multipath from Redhat.
> > 
> > I will consider this ...
> 
> The choice is yours. I just haven't found much info about multipath
> for other 
> distributions. (And I could still use a decent document/guide
> describing all 
> the different options)
> 
> > As I understand things here:
> > 
> > the former admin *tried to* setup multipath and somehow got stuck.
> 
> My guess: multipath wasn't enabled before the boot-proces would try
> to mount 
> it, the following needs to be done (and finished) in sequence for it
> to work:
> 
> 1) The OS needs to detect the disks (/dev/sdc + /dev/sdd). This
> requires 
> modules to be loaded and the fibrechannel disks to be detected
> 
> 2) multipathd needs to be running and correctly identified the
> fibrechannel disk 
> and the paths
> 
> 3) The OS needs to mount the fibrechannel disk using the
> "/dev/mapper/..." 
> entry created by multipath.
> 
> I run ZFS on top of the multipath entries, which makes it all a bit
> "simpler", 
> as the HBA module is built-in and the "zfs"  services depend on
> "multipath".
> All the mounting is done by the zfs services.
> 
> > That's why it isn't running and not used at all. He somehow
> > mentioned
> > this in an email back then when he was still working there.
> > 
> > So currently it seems to me that the storage is attached via
> > "single
> > path" (is that the term here?) only. "directly"= no redundancy
> 
> Exactly, and using non-guaranteed drive-letters. (I know for a fact
> that they 
> can chance as I've had disks move to different letters during
> subsequent boots. 
> I do have 12 disks getting 4 entries each, which means 48 entries ;)
> 
> > That means using the lpfc-kernel-module to run the FibreChannel-
> > adapters
> > ... which failed to come up / sync with a more recent gentoo
> > kernel, as
> > initially mentioned.
> 
> Are these modules not included in the main kernel?
> And maybe they require firmware which, sometimes, requires specific
> versions 
> between module/kernel versions.
> 
> > (right now: 4.1.15-gentoo-r1 ... )
> 
> Old, but if it works, don't fix it. (Just don't expose it to the
> internet)
> 
> > I consider sending a Debian-OS on a SSD there and let the (low
> > expertise) guy there boot from it. (or a stick). Which in fact is
> > risky
> > as he doesn't know anything about linux.
> 
> I wouldn't take that risk on a production server
> 
> > Or I simply wait for my on-site-appointment and start testing when
> > I am
> > there.
> 
> Safest option.
> 
> > Maybe I am lucky and the debian lpfc stuff works from the start.
> > And
> > then I could test multipath as well.
> 
> You could test quickly with the gentoo-install present as described
> above. The 
> config should be the same regardless.
> 
> > I assume that maybe the adapters need a firmware update or so.
> 
> When I added a 2nd HBA to my server, I ended up patching the firmware
> on both 
> to ensure they were identical.
> 
> > The current gentoo installation was done with "hardened" profile,
> > not
> > touched

Re: [EXTERNAL] Re: [gentoo-user] empty cdrom drive is busy or mounted

[Laurence Perkins]

On Fri, 2019-08-16 at 12:21 -0400, Jack wrote:
> On 2019.08.16 12:00, Helmut Jarausch wrote:
> > On 08/16/2019 05:25:34 PM, Jack wrote:
> > > try "lsof /cdrom"?  It says the mount point, not the device,
> > > might  
> > > be busy.
> > 
> > This didn't show anything.
> > I still don't know the cause of my problems.
> > But fortunately, they have been resolved by recompiling the
> > kernel  
> > (5.2.0),
> > systemd and all packages depending on systemd - using the new  
> > gcc-9.2.0
> > 
> > Furthermore I had to add the use flags
> > cgroup-hybrid -sysv-utils
> > for systemd. This hasn't been necessary before - very strange.
> > For me, systemd is a monster which I haven't understood.
> > I try to not use it since I am using openrc.
> > Perhaps I have to remove it from my system and use eudev instead
> > of  
> > udev as part of systemd.
> > 
> > Thanks for trying to help me - it was a really strange situation.
> 
> If you are using openrc (as I am) I would say you really don't want  
> systemd installed at all.  I can't imagine any good coming from
> that.   
> You do need eudev,  I also have elogind installed, but I'm not sure
> if  
> it's absolutely required, or if I had some other reason for
> installing  
> (possibly to get rid of consolekit?)

elogind will be needed if you have one of the desktop environments or
login managers that relies on it, and yeah, it mostly just takes the
place of console kit with regard to giving the user who is actually
sitting at the keyboard extra privileges.

LMP


signature.asc
Description: This is a digitally signed message part

Re: [EXTERNAL] Re: [gentoo-user] empty cdrom drive is busy or mounted

[Laurence Perkins]

On Thu, 2019-08-22 at 10:03 +1000, Adam Carter wrote:
> On Thu, Aug 22, 2019 at 5:48 AM james  wrote:
> > On 8/16/19 12:44 PM, Jack wrote:
> > > ps auxf | grep systemd
> > 
> > This is new turf for me. Upon issuing this command string I get::
> > 
> > # ps auxf | grep systemd
> > root 24947  0.0  0.0  13964   996 pts/6S+   15:43   0:00
> >   |   |   |   \_ grep --colour=auto systemd
> > 
> 
> This is showing that the only process with systemd in its name is the
> grep command itself; you could pass anything to grep and it will be
> found in the process list, eg;
> 
> $ ps auxf | grep blah
> adam   52359  0.0  0.0   7708   940 pts/3S+   09:55   0:00  
>\_ grep --colour=auto blah
> 
> So, there's no systemd process running on this system.

A common tactic is to use grep twice:
ps auxf | grep -v grep | grep blah

That strips out all instances of grep from the results.
Putting what you're searching for first is more efficient, but putting
it last keeps the colorized output intact.

LMP


signature.asc
Description: This is a digitally signed message part

Re: [EXTERNAL] Re: [gentoo-user] cannot open display

[Laurence Perkins]

On Sun, 2019-09-01 at 12:16 +0100, Mick wrote:
> On Saturday, 31 August 2019 23:41:13 BST Tamer Higazi wrote:
> > Hi people,
> > 
> > For a specific time I have a very strange behaviour on gentoo.
> > I cannot start any applicaion on xfce, just logout or shutdown the
> > machine.
> > 
> > When I try through an existing open shell to execute a program I
> > get
> > this error:
> > 
> > tamer@tux / $ firefox-bin
> > No protocol specified
> > Error: cannot open display: :0.0
> > 
> > Can somebody tell me what's wrong here?
> > 
> > 
> > best, Tamer
> 
> It may be related to an obscure old networkmanager bug, whereby it
> changes the 
> hostname in /etc/hosts.  I don't use networkmanager and I don't use
> xfce to 
> comment on particulars, but here's some generic things to check:
> 
> 1. Have a look at the 'Host and domain information' section and at
> the 'The 
> hosts file' section in the handbook and configure them accordingly:
> 
>  https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/System
> 
> 2. Does 'echo $HOSTNAME' show what you have configured in your
> system, after 
> you restart networkmanager, or reboot?
> 
> 3. Have you added /etc/init.d/hostname to your boot runlevel?
> 

Also check if your system is actually running the X server on :0.  If
it's running on :1 or higher for some reason then you'll need to update
your DISPLAY variable.

Just how you would have gotten it into such a state I wouldn't care to
guess at without more information.

LMP


signature.asc
Description: This is a digitally signed message part

Re: [EXTERNAL] Re: [gentoo-user] OpenSource Cellular

[Laurence Perkins]

On Tue, 2019-09-10 at 20:47 -0400, Walter Dnes wrote:
> On Tue, Sep 10, 2019 at 06:33:23PM -0400, james wrote
> > I have an expensive cellualr service and a Galaxy Note 9.
> > It SUCKS really bad. I spend way too much time, just dealing with
> > a close source system and way too many vendor traps. I'm done with
> > with,
> > and am curious if others have similar experiences and are
> > interested in
> > alternatives, that we the citizens control. In my newly forming 
> > approach, there is room for vendors and consumers and open source 
> > developers. I'm in the USA, but, there is absolutely no reason
> > this 
> > movement I am proposing, could not sweep the globe, IP centric,
> > free and 
> > open.
> 
>   I think Lineage OS is what you're looking for 
> https://lineageos.org/
> Open source, etc.  The main problem is that OEM's are locking down
> their
> phones, and making it harder to reflash firmware.  Plus, you've got
> to
> find somebody who knows what they're doing, and won't brick your
> phone.
> I'd be interested in anything you come up with.  Here's why...
> 
>   I'm in suburban Toronto Canada.  The CRTC (our equivalant of the
> FCC)
> have gone full asshole mode and decreed that *ALL* cellphone alerts
> are
> sent at the Presidential Alert level (it's re-labelled "Emergency
> Alert"
> up here).  *ALL* means *ALL*.  Yes that includes missing kid or child
> custody dispute several hundred miles away.  What doesn't help is
> that
> the OPP (Ontario Provincial Police) has a policy that *THE ENTIRE
> PROVINCE* gets a Presidential-Alert-masquarading-as-an-Amber-Alert
> when
> one is launched.  For the American audience, Ontario is intermediate
> in
> size between Alaska and Texas.  To add insult to injury, an "Amber
> Alert" often results in 3 (***THREE***) Presidential alert messages,
> even in the middle of the night...
> 
> * The original unblockable alert, sent in English
> * Oh yeah, Canada is officially bilingual, let's send out an
> unblockable
>   message in French half an hour later
> * an hour later the kid is found, so send out an unblockable "All
> Clear"
> 
>   Fortunately, the Canadian system is relatively new (spring 2018)
> and
> is only designed to work with LTE.  By forcing my phone (Alcatel Go
> Flip) down to 3G-only, I avoid these alerts.  Eventually, 3G will go
> away, so I want a solution for that day.  So far in my neck of the
> woods,
> there have been...
> 
> * 7 missing-kid alerts
> * 2 test alerts
> * 0 nuclear plant meltdowns
> * 0 chemical spills
> * 0 terrorist attacks
> * 0 tornadoes
> * 0 partidges-in-a-pear-tree-e-e-e-e-e
> 

Hardwarewise you might check out the Librem 5 and the "fairphone". 
Both are designed to give you more visibility into what's actually
going on and better ability to control what the blasted thing does in
the middle of the night.

LMP


signature.asc
Description: This is a digitally signed message part

Re: [EXTERNAL] [gentoo-user] Chromium on linux, build question

[Laurence Perkins]

On Sun, 2019-09-15 at 05:45 -0400, John Covici wrote:
> Hi.  I want to have Chromium  on linux, but I want to build the
> Chrome
> OS version, so I can have their version of the accessibility plugin
> which is called Chrom next.  I did not see any use flags, so how can
> I
> do this on gentoo?  I am using the unstable version of gentoo.  It is
> my understanding that this can be done on Debian and other
> distributions.
> 
> Thanks in advance for any suggestions.
> 
I believe what you're probably looking for is the chrome-binary-plugins 
package.  This downloads the binary version of Chrome and pulls all the
Google-proprietary bits out of it and loads them into your Chromium.

Of course, at that point you might as well just use Chrome...  But I
suppose you could try whacking out the ones you don't want and see what
happens.

LMP




signature.asc
Description: This is a digitally signed message part

Re: [EXTERNAL] [gentoo-user] CIFS VFS: cifs_mount failed w/return code = -112

[Laurence Perkins]

On Tue, 2019-09-24 at 10:13 -0600, the...@sys-concept.com wrote:
> I'm trying to mount Windows 10 directory on Gentoo
> 
> mount -t cifs -o username=xxx,password=xxx //10.0.0.146/customer
> /home/joseph/ttt/
> mount error(112): Host is down
> 
> I can ping 10.0.0.146 OK
> customer -dir  on Windows 10 has sharing enabled.
> 

Note that in Windows 10 sharing is always enabled and it just blocks or
unblocks it on the firewall.  Furthermore the status indicator is
pretty rubbish at determining if it's actually working correctly.

For testing, try disabling the Windows Firewall temporarily and see if
that makes it work.  If it does you might have to turn sharing off and
on again, or you might have to sift through the firewall rules and
correct the mistake yourself.

Possibly also the system thinks it's attached to a public network and
has the firewall set to block sharing services even though it says it's
turned on.

Regardless, the error you're getting means that something is blocking
the communication once it leaves the client machine.

LMP


signature.asc
Description: This is a digitally signed message part

Re: [EXTERNAL] [gentoo-user] Re: UEFI data corruption? [FIXED]

[Laurence Perkins]

On Wed, 2019-09-25 at 11:19 -0700, Ian Zimmerman wrote:
> On 2019-09-24 09:50, Peter Humphrey wrote:
> 
> > The Gentoo Handbook says to create a small unformatted partition at
> > the beginning of the (primary?) disk, then to create a FAT-32
> > partition for /boot, then whatever other partitions are required.
> 
> Does /boot really have to be a FAT partition, and not ext[234]?
> 
No.  But it can simplify things.

The UEFI standard requires that boards support FAT32.  They may or may
not support other filesystems at the manufacturer's discretion.

You can have /boot be one partition and /boot/efi be a different one if
you want.  But whatever you're using for an EFI partition needs to be a
filesystem that your board's firmware can read.  FAT32 is part of the
spec, some boards will also read NTFS.  I haven't personally seen any
that will read the ext family, but that doesn't mean they don't exist.

LMP


signature.asc
Description: This is a digitally signed message part

Re: [EXTERNAL] [gentoo-user] gentoo robot vacuum

[Laurence Perkins]

On Thu, 2019-10-03 at 20:13 -0400, james wrote:
> Gentoo community,
> 
> Robotic vacuum cleaners are all the rage nowadays.
> I'd like to buy/build one, that also has remote camera (so I can see 
> what troubles it is having by reviewing stored video) find it easy,
> and 
> make sure it's not just banging against the wall. I'm not so
> interested 
> in a slick, massively miniturized model, as much as I am
> something where I can get to the firmware, or is completely open 
> sourced; so I can fix/enhance the thing.
> If it's already done, then my searches have missed it, or a
> community 
> work on such linux centric solutions to automating home/small-office 
> flooring.
> 
> Anyone know of such a robotic vacuum that is basically very open, if
> not 
> completely open source?
> 
> Gentoo friendly vaccuum?
> 
> 
> Lawnmower is next on the list.
> 
> All suggestions and comments are most welcome.
> 
> James
> 

I don't know of any current models that have open firmware, but a bunch
of the older ones have a serial interface that you can attach to and
send them commands.  People used to tape a Pi or something and a camera
to the top of them and then program them to do all sorts of things.  I
suspect that's still your best bet.  Most of them don't have a lot of
brains, just a simple response pattern that will eventually cover the
whole room and maybe an IR homing system to let it find its dock again
when the battery runs low.

LMP


signature.asc
Description: This is a digitally signed message part

Re: [EXTERNAL] Re: [gentoo-user] OCR for music (OMR)

[Laurence Perkins]

On Mon, 2019-10-21 at 16:52 +0100, Mick wrote:
> Thanks Daniel,
> 
> On Monday, 21 October 2019 15:27:18 BST dan...@sonck.nl wrote:
> > I vaguely remember using some windows based program when archiving
> > old
> > music. The community in question had maps full of paper scores and
> > making
> > them digital would have made it much more compact and versatile.
> > However
> > even back then I remember manually writing lilypond files. I did
> > use Linux
> > at that time but wasn't aware of any OMR tools.
> > 
> > Eventually I stopped visiting and the project fell out of favor due
> > to the
> > immense manual labour. Literally hundreds of sheet music ranging
> > from single
> > page meant for the artists up to the combined score for the
> > conductor.
> 
> There have been a relatively large number of OMR projects falling in
> and out 
> of obscurity, forked, taken over, mixed with closed source and then
> die.
> 
> I was looking for an OMR package in portage to scan PDF sheets of
> music and 
> then split different voice parts out into separate midi files - but
> if a 
> single application will do the full workflow then I would be more
> than happy.
> 
> 
> > A quick search led me to audiveris. I have no experience but it
> > might do
> > what you want. It has the option to export to MusicXML. I tried to
> > use
> > Rosegarden to read a MusicXML which "worked". It did get the notes
> > right,
> > but alternated with trebble and bass lines, being a somewhat
> > monophonic end
> > result.
> > 
> > Hope this helps
> > 
> > Daniel
> 
> I understand there are a few apps by/for Apple, who seem to be the
> favourite 
> OS for arty users, but I no longer have OSX running here or any
> applications 
> for it.  There are also a number of other applications some of them
> for Linux 
> - but I haven't found one yet in portage (TBH I wouldn't know its
> name off-
> hand).
> 
> From what I read here Rosegarden will not perform OMR itself, but
> will import 
> various file formats:
> 
> https://www.rosegardenmusic.com/doc/en/file-other.html
> 

http://www.music-notation.info/en/compmus/omr.html

There are a few here that have potential, both free and paid.  If you
find one that works well, writing an ebuild for it shouldn't be too
terribly difficult.

LMP


signature.asc
Description: This is a digitally signed message part

Re: [gentoo-user] OT: looking for email provider

[Laurence Perkins]

On Sat, 2020-02-01 at 17:08 -0500, Jack wrote:
> CAUTION: This is an EXTERNAL email. Do not click links or open
> attachments unless you recognize the sender and know the content is
> safe.
> 
> Relying on the collective experience and advice of the group here.
> 
> As may be obvious to many of you, the address this message is sent
> from
> "...@users.sourceforge.net" isn't really a fully functional address.
> Email sent to that address will be forwarded by the sourceforge
> system
> to a personal address I specify.  When I send a message "From: " that
> address, however, I cannot send it through the sourceforge system, as
> I
> don't actually have an email account with them.  Currently, I send it
> through my gmail account.  That works because I added that address in
> my gmail Settings under "Accounts and Import" /  "Send mail as:".  To
> set it up, gmail sends a message to that address, and I click on a
> link
> in the message to prove it does come to me.  That's been working find
> for a long time, but, ...
> 
> I'm trying to move away from gmail.  Especially for mailing lists
> like
> this one, if I send a message to the list, I never see that I get the
> message from the list, because gmail refuses to show it in my inbox
> because it's a duplicate of a message already in my sentbox.
> 
> I do have an email account with privateemail.com (thorough
> namecheap.com) but they are unable or unwilling to have a similar
> setup.  I'm not even sure they actually understand what I'm asking of
> them, but I've wasted more than enough time trying.
> 
> So - I'm asking if anyone can recommend an email service provider
> that
> understands this and will let me set it up.  I have my own domain,
> but
> namecheap.com does seem willing to have the appropriate DNS record
> point to a different email provider.  At this point, I'm not
> interested
> in running my own email server.  I currently only need two mailboxes,
> maybe a small number more in the future, but this is personal, not
> commercial.  I don't need to do bulk emails, maybe up to a dozen or
> so
> recipients.  I do NOT expect it to be free, but cost is at least some
> consideration.  I don't need huge storage limits, as although I use
> IMAP access when on the road, when I'm home, I use POP3 to download
> everything.  I'd also like at least minimal control over spam
> filtering, mainly to let almost anything through for me to filter
> locally.  If privateemail.com has false positives for everything from
> some sender (such as ups.com, for example) I need to open a ticket
> with
> them to add a whitelist.  No such thing as clicking on "Not spam" and
> apparently no intent to ever do so.
> 
> Thanks for any suggestions.
> 
> Jack

You might talk to your ISP.  A number of them offer custom email
hosting to businesses and will maintain the server for you, but allow
you a rather customized configuration.  So kind of like having your own
server along with someone to manage it for you.

If you do end up running your own system, look through your options
thoroughly.  Sure you can set up just a simple email server, but there
are also projects like http://citadel.org/doku.php that offer more,
integrated features for an experience similar to gmail, but without the
spying.

LMP 

Re: [gentoo-user] Is there a way to misconfigure USB ports in the kernel?

[Laurence Perkins]

On March 30, 2021 10:11:56 AM PDT, Dr Rainer Woitok  
wrote:
>On Saturday, 2020-12-05 19:07:51 +0100, I myself wrote:
>
>("> >" refers to Michael )
>
>> Michael,
>> 
>> On Friday, 2020-11-27 19:07:17 +, you wrote:
>> 
>> > ...
>> > A 4k block size is recommended for ntfs-3g which is the default
>sector created 
>> > by fdisk and friends on Linux these days.  This will align your
>partition 
>> > optimally.  In addition, mkfs.ntfs will use 4096 bytes as the
>default cluster 
>> > size, so you should be good in that respect.
>> > 
>> > Another setting you may want to try is mounting the USB with
>'big_writes' - 
>> > check the man page.  This should help particularly with large
>files, which 
>> > will use larger blocks up to 128KB when copying data to the NTFS.
>> 
>> Both, the VeraCrypt command line (--fs-options=big_writes) and the
>Vera-
>> Crypt GUI  (under "Settings  --> Preferences")  allow setting this
>mount
>> option.  But
>> 
>>$ mount | grep veracrypt
>> 
>> never shows it,  initially causing me  to erroneously believe  it
>wasn't
>> set and to try finding  on the web another way  of setting it.   By
>pure
>> chance I finally found out that
>> 
>>$ ps -ef | grep veracrypt
>> 
>> lists a  "/usr/sbin/mount.ntfs" task  which shows the  options really
>in
>> effect.  However,  I haven't yet had the time to test the effect of
>this
>> option when writing  plenty of really big files.   I will report on
>that
>> later.
>
>Well,  it's been quite a while,  due to my being almost permanently
>con-
>fronted with more pressing tasks ... :-(
>
>To sum up my experience with my new 128 GB Philips USB 3.0 sticks:
>while
>the Philips sticks  are significantly faster for reading operations
>than
>my old 64 GB Verbatim ones (probably USB 2.0), writing operations to
>the
>Philips sticks  are unbearably slow,  regardless of whether  I created
>a
>normal unencrypted NTFS filesystem on them or an encrypted NTFS
>filesys-
>tem using VeraCrypt.   Writing to  the USB stick  while at the same
>time
>reading from it in a different terminal window caused commands like
>"cd"
>or "ls" to simply stall.  Thus while running
>
>   $ cp --preserve=timestamps -ru $source_dir .
>
>in one terminal window, I ran
>
>   $ while true
>   > do n=$(ps -ef|g 'cp --preserve'|g -v grep)
>   >if [[ "$n" = "${o-}" ]]
>   >then sleep 10
>   >else o="$n"
>   > echo "$n"
>   >fi
>   > done
>
>in another, to get the  wall clock times  when copying a new file
>began.
>That way I found that copying a 30 MB file took about 40 minutes.
>
>So what are my options?
>
>   - Stay away from Philips USB 3.0 sticks?
>
>   - Stay away from Philips USB sticks in general?
>
>   - Stay away from USB 3.0 sticks in general?
>
>  - Stay away from Filesystem in User Space  using a non-stable 5.10 or
> 5.11 kernel (currently I'm using stable 5.4.97)?
>
>   - Stay away from Gentoo?
>
>  - Stay away from Linux in general  and go back to OTOS  (aka the Only
> True Operating System aka Windoze)?
>
>   - ...?
>
>Any ideas and comments welcome ...
>
>Sincerely,
>  Rainer

There are a number of things which might be going on here.
To start with, you can get the kernel, user, and wall clock run times for 
commands by prefixing it with "time".  So:

time cp  

Will get you more precise answers with much less effort.

As for the performance of the USB drive in question, there are a few things 
that might be tripping it up.

Firstly, writing flash memory is significantly slower than reading it.  Some 
drives deal with this by having some kind of internal cache mechanism.  Many 
deal with it by using a pile of smaller chips instead of one big one and 
striping the writes.  If the Phillips drive just used a few large chips 
instead, then it's just slow to write to and there isn't much you can do about 
it.  I've seen a lot of cheaper drives that are that way.

Double check that the alignment and block size are correct for the drive's 
internal structure.  That can cause some pretty massive performance hits if 
it's incorrect.

You can also check the output of the dmesg command for any errors the system is 
encountering with regard to the drive.

I don't know of any reason to stay away from usb 3.0 on Linux, but if you have 
USB 3 devices on both ends and try to hook them together with a USB 2.0 or 1.1 
rated cable that could easily cause some problems...  I assume you're plugging 
the drive straight into the machine's socket.  If you're using the front panel 
though try one of the ones on the back.  There may be something up with the 
case wiring.

I've never had a Phillips USB stick, so maybe do some tests with another brand 
of stick and see if it has the same problem.  Kingston or SanDisk or something. 
 One of the ones where memory is their primary focus.  

You could definitely check performance on a different OS.  There may be 
driver-related performance issues on this model of drive or even this specific 
drive.

Instead of NTFS 

RE: [gentoo-user] Vanishing tab bar in Firefox

[Laurence Perkins]

Try Falkon maybe.

-Original Message-
From: mad.scientist.at.la...@tutanota.com  
Sent: Friday, August 20, 2021 9:45 AM
To: Gentoo User 
Subject: Re: [gentoo-user] Vanishing tab bar in Firefox

I've had the  same frustration.  There is an extension call "custom scroll 
bars".  The problem is due to a "feature" where web pages can decide on what a 
scroll bar looks like and how wide it is.  Apparently some web designers think 
they know what's best for others.  It's a flaw, not a "feature".   Fire fox 
claims this "ability" was sought after by web designers.  Frankly, I'm wishing 
there was an alternative browser, firefox has really gone south.

--"Fascism begins the moment a ruling class, fearing the people may use their 
political democracy to gain economic democracy, begins to destroy political 
democracy in order to retain its power of exploitation and special privilege." 
Tommy Douglas




Aug 20, 2021, 10:26 by pe...@prh.myzen.co.uk:

> On Saturday, 7 August 2021 10:45:26 BST Peter Humphrey wrote:
>
>> Hello list,
>>
>> Far too often, Firefox on this box will hide the tab bar, but behave 
>> as though it were still there. In other words, I have to aim the 
>> mouse pointer 2cm below the thing I want to click.
>>
>> Pressing  reveals the bar but clicking in it just hides it 
>> again. I can still cycle through the tabs with the standard 
>> .
>>
>> Sometimes remerging Firefox fixes it, sometimes rebooting does; 
>> mostly I just have to put up with it.
>>
>> Several versions of Firefox have suffered this, and several versions 
>> of Plasma. I might submit a bug report if I could work out which 
>> component is at fault.
>>
>
> Disabling my three plugins didn't help. I fixed it in the end by 
> removing ~/.mozilla and setting mozilla up again. At least, I hope 
> it's fixed. It's looking good so far.
>
> I didn't realise that Firefox was as vulnerable to local data problems 
> as
> KMail-2 is.  :(
>
> --
> Regards,
> Peter.
>

RE: [gentoo-user] VirtualBox UI

[Laurence Perkins]

-Original Message-
From: p...@xvalheru.org  
Sent: Saturday, August 21, 2021 12:53 PM
To: gentoo-user@lists.gentoo.org
Subject: Re: [gentoo-user] VirtualBox UI

On 2021-08-21 21:11, tastytea wrote:
> On 2021-08-21 20:42+0200 p...@xvalheru.org wrote:
> 
>> Hi,
>> 
>> I didn't use VirtualBox for a while and I find out that 
>> virtualbox-bin package has dissapeard. So I've installed virtualbox
>> 6.1.22 but there's missing VurtualBox command to tun graphical 
>> management. Is there other UI for VirutalBox?


I believe libvirt is capable of managing VirtualBox based VMs as well, but the 
official UI is definitely superior if you're only managing a single host.

If you haven't used it in a while, note that Oracle has changed the license on 
their extensions such that any activity with even the faintest hint of 
educational or commercial purposes now requires purchasing a license.  Last I 
looked the minimum purchase quantity was about $10,000 worth.

LMP

RE: [gentoo-user] python, my nemesis

[Laurence Perkins]

-Original Message-
From: Neil Bothwick  
Sent: Monday, September 20, 2021 8:17 AM
To: gentoo-user@lists.gentoo.org
Subject: Re: [gentoo-user] python, my nemesis

On Mon, 20 Sep 2021 15:56:46 +0200, Gerrit Kuehn wrote:

>> > > ~ # cat /etc/portage/package.use/py
>> > > */* PYTHON_TARGETS: -* python3_9 python3_8
>> > > */* PYTHON_SINGLE_TARGET: -* python3_9
>> > > ---
>> 
>> > You should probably not mess with these variables until after your 
>> > system is 100% updated and consistent. And even then, probably not.
>> > 
>> > With our package manager written in python, you often need old 
>> > python stuff to build the new python stuff, and disabling the old 
>> > python stuff will throw a wrench into that. Even in situations where 
>> > technically some upgrade path exists, the complexity of the python 
>> > dependencies often means that the package manager will give up 
>> > before it finds the solution unless the solution is obvious. By 
>> > tweaking those variables, you make the solution less obvious to it.
>> 
>> Well, this was the suggested way to go, see 
>> https://www.gentoo.org/support/news-items/2021-05-05-python3-9.html
>
>That news item is about going from 3.8 to 3.9, you are on 3.7. I'd try 
>removing the -* items are trying again.
>
>
When upgrading really old systems I find it useful to run:
export PYTHON_TARGETS="$(eix -I | grep -o "python[23]_[0-9]*" | sort -u)"

This will temporarily set the system (for the duration of the current session) 
to build for every version of python that is either available for new builds or 
currently in use.  The dependency resolver seems to have an easier time finding 
a path when it doesn't think it has to throw away all the currently installed 
stuff first.

Note that this will end up rebuilding a lot more than is technically necessary, 
and then you'll need to rebuild again with the python versions you actually 
want and depclean the extras.  Personally though I rather prefer to have the 
computer do more work instead of spending hours using *my* brain to find the 
minimum sized rebuild set to upgrade the system.

LMP

RE: [gentoo-user] python, my nemesis

[Laurence Perkins]

-Original Message-
From: Michael  
Sent: Monday, September 20, 2021 8:21 AM
To: gentoo-user@lists.gentoo.org
Subject: Re: [gentoo-user] python, my nemesis

On Monday, 20 September 2021 15:52:03 BST Gerrit Kuehn wrote:
>> 
>> Just extracting stage3 over everything that is already there?

>No, I move out of the way the config/data files I want to keep and move them 
>back in after untarring the stage 3.

A less traumatic place to start I find is to unpack the stage3 into /tmp or 
wherever it's out of the way and then chroot into it and use quickpkg to bundle 
up newer versions of blocked core utilities.  Not needing build deps can quite 
often simplify the upgrade path enough to get past certain things.

You can also (with a lot of annoyance) use the new portage that's in the 
tarball to get past eapi restrictions and whatnot and tell it to install the 
stuff it builds into your original system's folders.  I've even used that to 
salvage systems that got moved to new hardware with an incompatible set of 
processor flags.  Not straightforward since you have to manage both the build 
environment in the stage3 and the install environment in the original system, 
but there's very little you can't fix with this approach.

Just don't let your system build binary packages for virtual/* under any 
circumstances.  That never ends well.  I really need to write up a request to 
have portage blacklist those by default when buildpkg is enabled...

LMP

RE: [gentoo-user] Re: acrobat reader

[Laurence Perkins]

The fact that the form is completely unsupported on mobile coupled with the way 
many people are moving exclusively to tablet/phone platforms these days might 
be enough to convince them that it's time for an update if you sent them a 
message about it.  They generally like to preserve the illusion that they're 
competent.

The next version may or may not be worse than this one...

LMP

-Original Message-
From: Grant Edwards  
Sent: Wednesday, September 22, 2021 1:13 PM
To: gentoo-user@lists.gentoo.org
Subject: [gentoo-user] Re: acrobat reader

On 2021-09-22, the...@sys-concept.com  wrote:

> This is the form I'm trying to open:
>
> https://cfr.forms.gov.ab.ca/Form/AHC0102.pdf

That's an abomination that use Javascript and dynamic XFA content:

   https://kbpdfstudio.qoppa.com/livecycle-dynamic-xfa-forms/

Whoever designed something like that for use by the general public needs to 
have thier career terminated before they cause any more damage.

--
Grant

RE: [gentoo-user] How to compress lots of tarballs

[Laurence Perkins]

>On Monday, 27 September 2021 14:30:36 BST Peter Humphrey wrote:
>> On Monday, 27 September 2021 02:39:19 BST Adam Carter wrote:
>> > On Sun, Sep 26, 2021 at 8:57 PM Peter Humphrey
>
>> > 
>> > wrote:
>> > > Hello list,
>> > > 
>> > > I have an external USB-3 drive with various system backups. There 
>> > > are
>> > > 350
>> > > .tar files (not .tar.gz etc.), amounting to 2.5TB. I was sure I 
>> > > wouldn't need to compress them, so I didn't, but now I think I'm 
>> > > going to have to.
>> > > Is there a reasonably efficient way to do this?
>> > 
>> > find  -name \*tar -exec zstd -TN {} \;
>> > 
>> > Where N is the number of cores you want to allocate. zstd -T0 (or 
>> > just
>> > zstdmt) if you want to use all the available cores. I use zstd for 
>> > everything now as it's as good as or better than all the others in 
>> > the general case.
>> > 
>> > Parallel means it uses more than one core, so on a modern machine it 
>> > is much faster.
>> 
>> Thanks to all who've helped. I can't avoid feeling, though, that the 
>> main bottleneck has been missed: that I have to read and write on a USB-3 
>> drive.
>> It's just taken 23 minutes to copy the current system backup from 
>> USB-3 to SATA SSD: 108GB in 8 .tar files.
>
>I was premature. In contrast to the 23 minutes to copy the files from USB-3 to 
>internal SSD, zstd -T0 took 3:22 to compress them onto another internal SSD. I 
>watched /bin/top and didn't see more than 250% CPU (this is a 24-CPU box) with 
>next-to-nothing else running. The result was 65G of .tar.zst files.
>
>So, at negligible cost in CPU load*, I can achieve a 40% saving in space. Of 
>course, I'll have to manage the process myself, and I still have to copy the 
>compressed files back to USB-3 - but then I am retired, so what else do I have 
>to do? :)
>
>Thanks again, all who've helped.
>
>*  ...so I can continue running my 5 BOINC projects at the same time.
>
>--
>Regards,
>Peter.

There are also backup tools which will handle the compression step for you.

app-backup/duplicity uses a similar tar file and index system with periodic 
full and then incremental chains.  Plus it keeps a condensed list of file 
hashes from previous runs so it doesn't have to re-read the entire archive to 
determine what changed the way rsync does.

app-backup/borgbackup is more complex, but is very, very good at deduplicating 
file data, which saves even more space.  Furthermore, it can store backups for 
multiple systems and deduplicate between them, so if you have any other 
machines you can have backups there as well, potentially at negligble space 
cost if you have a lot of redundancy.

LMP

RE: [gentoo-user] How to compress lots of tarballs

[Laurence Perkins]

Regular xzutils now does multiple threads with the -T option.

> -Original Message-
> From: Ramon Fischer  
> Sent: Sunday, September 26, 2021 5:23 AM
> To: gentoo-user@lists.gentoo.org
> Subject: Re: [gentoo-user] How to compress lots of tarballs
> 
> In addition to this, you may want to use the parallel implementations of 
> "gzip", "xz", "bzip2" or the new "zstd" (zstandard), which are "pigz"[1], 
> "pixz"[2], "pbzip2"[3], or "zstmt" (within package "app-arch/zstd")[4] in 
> order to increase performance:
> 
> $ cd 
> $ for tar_archive in *.tar; do pixz "${tar_archive}"; done
> 
> -Ramon
> 
> [1]
> * https://www.zlib.net/pigz/
> 
> [2]
> * https://github.com/vasi/pixz
> 
> [3]
> * https://launchpad.net/pbzip2
> * http://compression.ca/pbzip2/
> 
> [4]
> * https://facebook.github.io/zstd/
> 
> 
> On 26/09/2021 13:36, Simon Thelen wrote:
> > [2021-09-26 11:57] Peter Humphrey 
> >> part   text/plain 382
> >> Hello list,
> > Hi,
> >
> >> I have an external USB-3 drive with various system backups. There are 
> >> 350 .tar files (not .tar.gz etc.), amounting to 2.5TB. I was sure I 
> >> wouldn't need to compress them, so I didn't, but now I think I'm 
> >> going to have to. Is there a reasonably efficient way to do this? I 
> >> have 500GB spare space on /dev/sda, and the machine runs constantly.
> > Pick your favorite of gzip, bzip2, xz or lzip (I recommend lzip) and
> > then:
> > mount USB-3 /mnt; cd /mnt; lzip *
> >
> > The archiver you chose will compress the file and add the appropriate 
> > extension all on its own and tar will use that (and the file magic) to 
> > find the appropriate decompresser when you want to extract files later 
> > (you can use `tar tf' to test if you want).
> >
> > --
> > Simon Thelen
> >
> 
> --
> GPG public key: 5983 98DA 5F4D A464 38FD CF87 155B E264 13E6 99BF
> 

RE: [gentoo-user] Iphone and transferring image files, pics and videos.

[Laurence Perkins]

> CAUTION: This is an EXTERNAL email. Do not click links or open attachments 
> unless you recognize the sender and know the content is safe.
> 
> Howdy all,
> 
> My Sis-n-law has a Iphone.  She takes TONS of pics and quite a lot of videos 
> with it.  Since a lot of them are family photos, I'd like to download them to 
> my puter.  I'd prefer to use Digikam since I can tell it to 'download new' 
> next time and not have to worry about duplicates or missing some.  Thing is, 
> it can't access all the photo directories.  I know there is about 4,000 files 
> all together.  Digikam only shows less than 1,000.  I managed to copy them 
> over using Dolphin but it took several tries and some disconnect and 
> reconnecting to do it.
> 
> I did see a error in messages that mtp-probe?? was missing.  I found it and 
> installed it.  That error went away and it did improve things.
> Maybe I'm missing something else but dmesg and messages isn't complaining.
> 
> What do others do to accomplish this?  Is it normal to have issues with this 
> or am I missing something?
> 
> Dale
> 
> :-)  :-)
> 

It's a bit more work than maybe you're looking for, but I set up a Nextcloud 
server and then all our phones have the Nextcloud app and are set to 
automatically upload new pictures whenever they have an appropriate connection.

This does require maintaining a Nextcloud server or similar, but it also makes 
it so that you don't have to plug the phone into anything, or even remember to 
do it yourself.

Furthermore, if you configure the Nextcloud app correctly, then when you run 
low on space, you can tell it to remove the local copies of the older pictures 
and it will keep only the thumbnails on the phone itself, but can seamlessly 
pull them back in should you want to look at them full-size.

Plus it lets you synchronize notes for things like grocery lists at no extra 
charge.

I know a few people who use iPhones with Linux.  It can usually be made to work 
with some trouble, but Apple constantly tries to lock out third party access, 
so it requires regular updates and tweaking.

LMP

RE: [gentoo-user] How to compress lots of tarballs

[Laurence Perkins]

> > On Wed, Sep 29, 2021 at 4:27 AM Peter Humphrey  
> > wrote:
> >> Thanks Laurence. I've looked at borg before, wondering whether I 
> >> needed a more sophisticated tool than just tar, but it looked like 
> >> too much work for little gain. I didn't know about duplicity, but I'm 
> >> used to my weekly routine and it seems reliable, so I'll stick with 
> >> it pro tem. I've been keeping a daily KMail archive since the bad old 
> >> days, and five weekly backups of the whole system, together with 12 
> >> monthly backups and, recently an annual backup. That last may be overkill, 
> >> I dare say.
> > I think Restic might be gaining some ground on duplicity.  I use 
> > duplicity and it is fine, so I haven't had much need to look at 
> > anything else.  Big advantages of duplicity over tar are:
> >
> > 1. It will do all the compression/encryption/etc stuff for you - all 
> > controlled via options.
> > 2. It uses librsync, which means if one byte in the middle of a 10GB 
> > file changes, you end up with a few bytes in your archive and not 10GB 
> > (pre-compression).
> > 3. It has a ton of cloud/remote backends, so it is real easy to store 
> > the data on AWS/Google/whatever.  When operating this way it can keep 
> > local copies of the metadata, and if for some reason those are lost it 
> > can just pull that only down from the cloud to resync without a huge 
> > bill.
> > 4. It can do all the backup rotation logic (fulls, incrementals, 
> > retention, etc).
> > 5. It can prefix files so that on something like AWS you can have the 
> > big data archive files go to glacier (cheap to store, expensive to 
> > restore), and the small metadata stays in a data class that is cheap 
> > to access.
> > 6. By default local metadata is kept unencrypted, and anything on the 
> > cloud is encrypted.  This means that you can just keep a public key in 
> > your keyring for completely unattended backups, without fear of access 
> > to the private key.  Obviously if you need to restore your metadata 
> > from the cloud you'll need the private key for that.
> >
> > If you like the more tar-like process another tool you might want to 
> > look at is dar.  It basically is a near-drop-in replacement for tar 
> > but it stores indexes at the end of every file, which means that you 
> > can view archive contents/etc or restore individual files without 
> > scanning the whole archive.  tar was really designed for tape where 
> > random access is not possible.
> >
> 
> 
> Curious question here.  As you may recall, I backup to a external hard drive. 
>  Would it make sense to use that software for a external hard drive?  Right 
> now, I'm just doing file updates with rsync and the drive is encrypted.  
> Thing is, I'm going to have to split into three drives soon.  So, compressing 
> may help.  Since it is video files, it may not help much but I'm not sure 
> about that.  Just curious. 
> 
> Dale
> 
> :-)  :-) 
> 
> 

If I understand correctly you're using rsync+tar and then keeping a set of 
copies of various ages.

If you lose a single file that you want to restore and have to go hunting for 
it, with tar you can only list the files in the archive by reading the entire 
thing into memory and only extract by reading from the beginning until you 
stumble across the matching filename.  So with large archives to hunt through, 
that could take...  a while...

dar is compatible with tar (Pretty sure, would have to look again, but I 
remember that being one of its main selling-points) but adds an index at the 
end of the file allowing listing of the contents and jumping to particular 
files without having to read the entire thing.  Won't help with your space 
shortage, but will make searching and single-file restores much faster.

Duplicity and similar has the indices, and additionally a full+incremental 
scheme.  So searching is reasonably quick, and restoring likewise doesn't have 
to grovel over all the data.  It can be slower than tar or dar for restore 
though because it has to restore first from the full, and then walk through 
however many incrementals are necessary to get the version you want.  This 
comes with a substantial space savings though as each set of archive files 
after the full contains only the pieces which actually changed.  Coupled with 
compression, that might solve your space issues for a while longer.

Borg and similar break the files into variable-size chunks and store each chunk 
indexed by its content hash.  So each chunk gets stored exactly once regardless 
of how many times it may occur in the data set.  Backups then become simply 
lists of file attributes and what chunks they contain.  This results both in 
storing only changes between backup runs and in deduplication of 
commonly-occurring data chunks across the entire backup.  The database-like 
structure also means that all backups can be searched and restored from in 
roughly equal amounts of time and that backup sets can be deleted in any order. 

RE: [gentoo-user] Re: setcap fails: (Operation not supported)

[Laurence Perkins]

Doesn't it require xattrs?  I vaguely remember running into that at one point 
years ago.  Not sure if the other flags you're using will force xattr support 
on or not, but it's worth checking.

LMP

-Original Message-
From: Grant Edwards  
Sent: Thursday, September 30, 2021 3:00 PM
To: gentoo-user@lists.gentoo.org
Subject: [gentoo-user] Re: setcap fails: (Operation not supported)

On 2021-09-30, Andrew Udvare  wrote:
> On 30/09/2021 13:58, Grant Edwards wrote:

>> Still can't figure out how to get setcap to work

> Not sure if this is it, but do you have CONFIG_EXT4_FS_SECURITY enabled?

No, I don't.

Google has found me information that indicates that SELinux and MAC (Mandatory 
Access Controls) require FS_SECURITY, but Google can't find any indication that 
FS_SECURITY is required for linux file capabilities.

I should try enabling it and see...

Several years ago, I know I could set capabilities on executables (on a 
different Gentoo machine), and I don't remember it being difficult to get 
working at all...

--
Grant

RE: [gentoo-user] 2021-08-24-eudev-retirement

[Laurence Perkins]

> Jack wrote:
> > On 2021.10.05 15:32, the...@sys-concept.com wrote:
> >> I have "eudev" installed but I think it will be obsolete as of
> >> Jan.1/22 according to news: 2021-08-24-eudev-retirement
> >>
> >> Does converting from: sys-fs/eudev
> >> to: sys-fs/udev
> >>
> >> is as simple as: emerge -C sys-fs/eudev emerge sys-fs/udev
> > Have you read the news item?  It's not quite that literal about what 
> > to do, but does imply that should work.  However, on 15 Sep there was 
> > a post on the gentoo-dev list with a link to 
> > https://github.com/eudev-project/eudev (see README.md) which says that 
> > there is a new team taking up maintenance of eudev, so it is not 
> > abandoned, and so it is not clear whether eudev will actually be 
> > dropped or not.
> >
> >
> 
> 
> When I switched from udev to eudev, it was as simple as uninstalling udev and 
> installing eudev.  I'd think as you do the reverse would work now as well but 
> as you point out, it may not.  I'd recommend going to a console to do this 
> and logging out of any GUIs as well.  Of course restarting udev afterwards as 
> well. 
> 
> I also saw the post about new maintainers.  I'm hopeful that it will keep 
> being maintained even tho it can be a headache for devs at times. Thing is, 
> I've read a lot of distros use eudev and avoid the systemd version as much as 
> possible, even tho eudev still has the same code from my understanding.  
> Still, I'd rather stick with what works for me. 
> 
> Maybe we will know pretty soon what the status of this will be. 
> 
> Dale
> 
> :-)  :-) 
> 
> 

I was also glad to hear about the new maintainers.  The day after I heard the 
project had served its purpose and was being abandoned I went to update one of 
my few systems with udev instead of eudev and the build crashed and it took me 
a couple hours to trick it into compiling correctly.  I was a little worried 
that might end up being prophetic.  Whatever the exact differences are, eudev 
still seems to build a little more reliably.  Maybe it's just that the project 
does a better job of informing people about changes to the build requirements...

LMP

RE: [gentoo-user] Hard drive pricing and the near future

[Laurence Perkins]

> -Original Message-
> From: Dale  
> Sent: Tuesday, October 5, 2021 4:32 PM
> To: gentoo-user@lists.gentoo.org
> Subject: [gentoo-user] Hard drive pricing and the near future
> 
> 
> Howdy all,
> 
> I still have quite a bit of drive storage but I've read that prices on drives 
> are on the rise.  Thing is, I don't track them much.  I'm looking at buying a 
> 8TB drive and I've researched to make sure I'm getting a PMR/CMR drive.  I'm 
> avoiding a SMR since it doesn't perform as well in my use case.  I tend to 
> stick with Seagate, WD and other major makers.
> 
> If anyone reading this does track the pricing of drives, are they on the 
> rise, stable, dropping or what?  Is this a good time to expand while it is 
> more cost effective?  I shop around on ebay, Amazon and others before buying. 
>  I'm not opposed to buying used since I can sometimes find one that was 
> pulled and sometimes has only a few hours of use.  I found one once that only 
> had like 10 hours on it.  Still got it too.
> 
> One reason I'm wanting to do this now is price.  However, in a year or so, 
> I'm getting fiber internet, dang fast at that.  It's starts at 200Mb but 
> still over a 100 times faster than current connection.  It goes all the way 
> up to 1Gb.  God help us all.  ROFL
> 
> Thoughts??
> 
> Dale
> 
> :-)  :-)
> 

If you want to avoid SMR, look for the drives built for video surveillance 
systems.  Most manufacturers have a line for that purpose.  For WD it's the 
"purple" ones.

Other option, depending on exactly what your use case is would be to look into 
your choice of filesystem.  SMR doesn't like random writes into one of its 
chunks unless it has enough idle time to go back and straighten it out later is 
all.  There are now format options for ext4 to align its metadata to the SMR 
sections and to make it avoid random writes as much as it can.  Additionally 
BTRFS, ZFS, and NILFS2 are all structured such that they tend to write from one 
end of the disk to the other and then jump back to the beginning, so they see 
little if any degradation from SMR.

LMP

RE: [gentoo-user] Hard drive pricing and the near future

[Laurence Perkins]

> On Wed, Oct 6, 2021 at 12:16 PM Laurence Perkins  wrote:
> >
> > Other option, depending on exactly what your use case is would be to look 
> > into your choice of filesystem.  SMR doesn't like random writes into one of 
> > its chunks unless it has enough idle time to go back and straighten it out 
> > later is all.  There are now format options for ext4 to align its metadata 
> > to the SMR sections and to make it avoid random writes as much as it can.  
> > Additionally BTRFS, ZFS, and NILFS2 are all structured such that they tend 
> > to write from one end of the disk to the other and then jump back to the 
> > beginning, so they see little if any degradation from SMR.
> 
> Unless something has changed, it was ZFS rebuilds that caused a lot of the 
> initial fuss on Linux.  Drives were getting dropped from pools due to 
> timeouts/etc during rebuilds.  I'm not sure how sequential the IO is for ZFS 
> rebuilds.  I think btrfs seems a bit smarter about scrubs in general.
> 
> --
> Rich
> 

Good to know.  I don't use ZFS myself, and I suspect the benchmarks I looked at 
when I was checking on how best to deal with SMR drives didn't try a pool 
rebuild for ZFS.

Scrub on BTRFS is read-only unless there are errors that need correcting, which 
should be rare.  BTRFS balance operations might be affected, but those aren't 
needed as often as they used to be and should still operate linearly on whole 
chunks.  

I think what to look for there would be if there's a way to align the BTRFS 
chunks to the SMR blocks.  But the manufacturers decided to continue 
manufacturing CBR disks for the surveillance industry, so I haven't had to 
worry about it just yet.

LMP

RE: [gentoo-user] Hard drive pricing and the near future

[Laurence Perkins]

> On Wed, Oct 6, 2021 at 3:37 PM Laurence Perkins  wrote:
> > I think what to look for there would be if there's a way to align the BTRFS 
> > chunks to the SMR blocks.
> 
> There are definitely ways to implement filesystems that are more compatible 
> with SMR.  You basically want something like a log-based filesystem.  A COW 
> filesystem is actually a really good candidate as they don't do in-place 
> writes ever, and all you need to do is defer block frees and garbage collect 
> and so on to make it more log-based.

COW is a good compromise.  Log-based filesystems like NILFS2 generally have 
terrible performance.  Their only advantage in this situation is that they 
don't suffer any detectable performance loss from SMR.
EXT4 has new formatting options that tell it to sequentialize the metadata 
writes as much as possible.  Supposedly it helps quite a bit, but I haven't 
tested it myself yet.

> 
> One of the issues though is that these drives obfuscate how they work.
> The filesystem has no way to intentionally write to the CMR vs SMR regions.

It depends on the drive.  There are ATA commands you can use and at least some 
drives will tell you what their zones are and can be told to give the host more 
control over what goes where.
https://zonedstorage.io/getting-started/smr-disk/  has a good description.
Definitely avoid the Drive Managed Interface ones except maybe for cheap, bulk 
storage.  But the Host Aware and Host Managed ones can perform pretty well on 
some IO loads with proper FS tuning.

> 
> This is why drive-managed SMR really shouldn't be a thing.
> Host-managed SMR makes a LOT more sense, because then the filesystem can 
> mitigate most of the issues and not end up fighting the drive firmware, whose 
> behavior isn't even standardized.

Yup.  Only reason for drive-managed is for drop-in replacement of 
low-throughput systems that couldn't otherwise handle it.

> 
> > But the manufacturers decided to continue manufacturing CMR disks for the 
> > surveillance industry, so I haven't had to worry about it just yet.
> 
> There are lots of CMR drives out there.  It is just that you have to be 
> careful as nothing is well-documented and it is all subject to change.  It is 
> like trying to figure out how many channels a DIMM has or what its timing 
> capabilities are.
> 
> I saw a good price on an Exos drive and I believe those are all CMR.
> This is why I use slickdeals - you can set up searches and get alerts when a 
> price drops.  It also picks up stuff like Best Buy who often has some of the 
> best prices on USB enclosures for whatever reason when they go on sale.

Almost everything marketed as being for the surveillance industry is going to 
be CMR.  We often run the drives at 80% or better throughput and a lot of it is 
non-sequential.  SMRs just can't keep up.  So there was a lot of pushback from 
the surveillance industry about not changing the underlying storage technology 
without clearly labeling it as such.

Prior to that the announcement from Seagate and I think WD was that everything 
larger than 12TB was going to be SMR.  (I work on the software side of things, 
I get hardware stuff second hand, so my understanding may be fuzzy.)

But currently the WD Purples and the Seagate Skyhawks should all be CMR.

Obviously I have no power to hold them to that, so do not take this as any kind 
of guarantee.  But if they change it without warning they're likely to have a 
lot of unhappy customers.

> 
> --
> Rich
> 
> 

LMP

RE: [gentoo-user] [OT] how to delete a directory tree really fast

[Laurence Perkins]

> -Original Message-
> From: Rich Freeman  
> Sent: Friday, October 22, 2021 12:29 PM
> To: gentoo-user@lists.gentoo.org
> Subject: Re: [gentoo-user] [OT] how to delete a directory tree really fast
> 
> On Fri, Oct 22, 2021 at 3:21 PM Helmut Jarausch  wrote:
> >
> > Is it possible to have a hard link from one subvolume to a different 
> > one?
> 
> You could do a quick test, but I don't think so.  I haven't used btrfs in 
> years but they're basically separate filesystems as far as most commands are 
> concerned.  I don't think you can create reflinks between subvolumes either.
> 
> The files are already reflinked by design though.  You'd just make a new 
> snapshot and then rsync over it.  Anything that doesn't change will already 
> share space on disk by virtue of the snapshot.  Anything that does change 
> will only be modified on the snapshot you target with rsync.  I'm not sure 
> why you'd want to use a hardlink - it doesn't provide the isolation you 
> already get from the snapshot.
> 
> 
> --
> Rich

So the BTRFS filesystem itself supports hardlinks and reflinks between 
subvolumes because it has to for writable snapshots to work correctly.
The utilities, on the other hand, have not all read that memo so actually 
making it do what you want can sometimes be a bit frustrating.


Note also that all these garbage-collected filesystems are basically doing the 
equivalent of "mv to-delete .deleted ; ionice -c3 rm -rf .deleted"
The files all seem to disappear instantly, but you don't get your space back 
until the garbage collector has had a chance to grovel over all the metadata.
Groveling over the metadata is the part that takes a long time for the rm 
command.  
The advantage to garbage-collected is mainly that if you need to reboot in the 
middle of it it will automatically pick up where it left off when the 
filesystem is mounted again.

But yes, in the future if you're building a massive directory tree that you're 
planning to delete, put it in a subvolume.  That lets you do all kinds of 
useful things with it.

LMP

RE: [gentoo-user] Re: external (NFTS) USB 2TB stick error mount.

[Laurence Perkins]

>> 
>> 
>> -Original Message-
>> From: Grant Edwards  
>> Sent: Monday, November 1, 2021 4:10 PM
>> To: gentoo-user@lists.gentoo.org
>> Subject: [gentoo-user] Re: external (NFTS) USB 2TB stick error mount.
>> 
>> On 2021-11-01, the...@sys-concept.com  wrote:
>> > On 11/1/21 4:47 PM, Grant Edwards wrote:
>> >> On 2021-11-01, the...@sys-concept.com  wrote:
>> >> 
>> >>> I format external nvme SSD (M.2) drive as NTFS on Windows (to store 
>> >>> some pictures etc.)  But when I insert the drive on Linux box (it 
>> >>> has support for NTFS enabled) I get an error:
>> >> 
>> >> Please define what you mean by "it has support for NTFS enabled".
>> >> 
>> >> Are you running a v5.15 kernel with the new in-tree NTFS driver?
>> >> 
>> >> Are you using the ntfs3g FUSE driver?
>> >> 
>> >> Are you using the old, read-only NTFS in-tree driver?
>> >
>> > I'm using kernel:  5.4.72-gentoo
>> >
>> > Under:  File systems:
>> > DOS/FAT/NT Filesystems:
>> > <*> MSDOS fs support 
>> >- <*> VFAT (Windows-95) fs support 
>> >- <*> NTFS file system support 
>> >- [*]   NTFS write support   
>> >  After installing sys-fs/ntfs3g
>> > It keeps telling I don't have:
>> > CONFIG_FUSE_FS: is not set when it should be.
>> 
>> OK, that's what is referred to as the old read-only in-tree driver. It has 
>> had experimental write support for a long time, but people still call it 
>> "the read-only driver". Most people "in the know" seem to advise that the 
>> write support should not be used, and many advise not using that driver it 
>> at all. The ntfs-3g FUSE driver is usually recommended instead.
>> 

Because it is read-only for all practical purposes.  Unless there's been a 
recent improvement its write support technically works, but it can only change 
the contents of existing files and cannot create new ones.  Works fine for a 
read-only driver, but not much point to using it unless you're an insane nutjob 
like me who occasionally needs to boot from NTFS and so needs a driver that can 
be compiled in.

LMP

RE: [gentoo-user] Ethernet card for puter

[Laurence Perkins]

>
>
>-Original Message-
>From: Wol  
>Sent: Sunday, November 7, 2021 1:26 AM
>To: gentoo-user@lists.gentoo.org
>Subject: Re: [gentoo-user] Ethernet card for puter
>
>Only problem was a screw-up over the router - the fibre was terminated at an 
>RJ45 in my house, but apparently needed a dedicated wan port on the router - 
>you can't plug it into a standard port - so I was without internet until they 
>sorted out a new router for me.
>> 

The *fibre* was terminated at an *RJ45*?  Sounds like somebody screwed up 
massively and said you had the wrong router so you wouldn't think they were 
idiots.  Either that or it wasn't actually an RJ45 and you needed a router with 
a fibre port.

LMP

RE: [gentoo-user] Re: Ethernet card for puter

[Laurence Perkins]

>-Original Message-
>From: Grant Edwards  
>Sent: Monday, November 8, 2021 8:48 AM
>To: gentoo-user@lists.gentoo.org
>Subject: [gentoo-user] Re: Ethernet card for puter
>
>On 2021-11-08, Laurence Perkins  wrote:
>>>
>>>
>>>-Original Message-
>>>From: Wol 
>>>Sent: Sunday, November 7, 2021 1:26 AM
>>>To: gentoo-user@lists.gentoo.org
>>>Subject: Re: [gentoo-user] Ethernet card for puter
>>>
>>>Only problem was a screw-up over the router - the fibre was terminated at an 
>>>RJ45 in my house, but apparently needed a dedicated wan port on the router - 
>>>you can't plug it into a standard port - so I was without internet until 
>>>they sorted out a new router for me.
>>
>>
>> The *fibre* was terminated at an *RJ45*?  Sounds like somebody screwed 
>> up massively and said you had the wrong router so you wouldn't think 
>> they were idiots.  Either that or it wasn't actually an RJ45 and you 
>> needed a router with a fibre port.
>
>The fiber is undoubtedly terminated at an ONT which has an RJ45 jack which 
>then needs to be connected to what the ISP usually calls "A Modem". That 
>"modem" is generally a firewall/router and WAP.
>
>The exact Ethernet protocols used on that RJ45 connection to the "modem" 
>varie. Some do PPPoE, some just need some sort of authenticating DHCP client, 
>so do other stuff.
>
>--
>Grant

Yes, that would make far more sense.  A bit disappointing though if they 
couldn't just tell him the authentication protocol to use...

LMP

RE: [gentoo-user] Virtual Desktop with a Virtual Monitor?

[Laurence Perkins]

> -Original Message-
> From: Mark Knecht  
> Sent: Monday, November 8, 2021 11:48 AM
> To: Gentoo User 
> Subject: Re: [gentoo-user] Virtual Desktop with a Virtual Monitor?
> 
> On Mon, Nov 8, 2021 at 12:23 PM Andreas Stiasny 
>  wrote:
> >
> > On 08.11.21 17:34, Mark Knecht wrote:
> > > In the case where the monitor is detached at boot two modules are 
> > > not loaded - fuse & nvmem_rmem. Other than that the module list 
> > > appears identical.
> > >
> >
> > At the moment I can't help you to solve this in software but there may 
> > be a hardware solution. There are fake monitor plugs. This is just a 
> > plug without a cable that makes the device think that there is a 
> > monitor attached even if there isn't. Search for "HDMI dummy plug" or 
> > similar.
> >
> >
> > Andreas
> >
> 
> Actually, if it works and I have little doubt it does, that's a great 
> solution for my needs. I see Amazon has 1920x1080 plugs but also has virtual 
> 4K plugs. All under $10.
> 
> Thanks!
> 
> Mark
> 

So, I don't use the PI variants with GUI or even with a monitor very often, but 
as I recall if the HDMI port isn't attached at boot it then disables it to save 
the video memory.

If you then try to use one of the VNC servers that's designed to share the 
running X session, you'll find that there isn't a running X session.

But what you should still be able to do is install the "vncserver" command to 
fire up an X server that isn't attached to the physical output, only the VNC.
You may need to configure it to also start your preferred display manager, etc 
depending on whether or not the distro you're running does that automatically 
for you.

If you want to be able to plug in a monitor later though you'll probably need 
one of the dummy plugs to prevent the output from being switched off and 
ignored.

LMP

RE: [gentoo-user] Suggestions for NAS appliance?

[Laurence Perkins]

The genpi64 project is supposedly back up and running, so you can get newer 
images that need fewer updates again.

From: Mark Knecht 
Sent: Wednesday, November 10, 2021 8:06 PM
To: Gentoo User 
Subject: Re: [gentoo-user] Suggestions for NAS appliance?


On Wed, Nov 10, 2021, 7:49 PM Alan Grimes 
mailto:alonz...@verizon.net>> wrote:
Hey, my old NAS box croaked the other day. I had to spend $400 ond
hardware and software to recover my data but the issue now is finding a
good new NAS solution.

Use is basically media server + backup provider.

I was happy with my old Netgear ReadyNAS idiot-proof box until it died.
Any experience with products on the market?

Not a recommendation precisely but there's a guy on YouTube named Jeff Geerling 
that's doing a lot of that sort of thing using a Raspberry Pi and multiple SATA 
drives. I've just built my first RP4 box aimed at astrophotography and I'm 
pretty impressed with how well the Pi works. My next project will likely be 
some sort of NAS box using a second Pi4 with an M.2 system drive.

There is a bootable Gentoo image for the Pi4. It was pretty zippy but it's a 
year and a half old so if you're fixated on Gentoo you'd have a lot of work to 
do vs using Ubuntu server or something prebuilt.

HTH,
Mark

RE: [gentoo-user] Suggestions for NAS appliance?

[Laurence Perkins]

> -Original Message-
> From: Peter Humphrey  
> Sent: Friday, November 12, 2021 12:19 AM
> To: gentoo-user@lists.gentoo.org
> Subject: Re: [gentoo-user] Suggestions for NAS appliance?
> 
> On Thursday, 11 November 2021 22:43:17 GMT Mark Knecht wrote:
> > On Thu, Nov 11, 2021 at 3:21 PM Laurence Perkins 
> > 
> wrote:
> > > The genpi64 project is supposedly back up and running, so you can 
> > > get newer images that need fewer updates again.
> > > 
> > > 
> > > 
> > > From: Mark Knecht 
> > > Sent: Wednesday, November 10, 2021 8:06 PM
> > > To: Gentoo User 
> > > Subject: Re: [gentoo-user] Suggestions for NAS appliance?
> > 
> > Do you have a better URL than I have?
> > 
> > https://github.com/sakaki-/gentoo-on-rpi-64bit
> 
> I gogled for 'genpi64 project' and got this: https://github.com/GenPi64 .
> 
> --
> Regards,
> Peter.

Yes, that one.  I think the build.dist is the repo you want.  I don't know if 
they're providing finished images yet or not, but it should at least be capable 
of building them for you on something more powerful than the pi itself.

LMP

RE: [gentoo-user] Switching from eudev to udev, disaster.

[Laurence Perkins]

>> -Original Message-
>> From: tastytea  
>> Sent: Monday, November 29, 2021 9:00 PM
>> To: gentoo-user@lists.gentoo.org
>> Subject: Re: [gentoo-user] Switching from eudev to udev, disaster.
>> 
>> On 2021-11-29 22:47-0600 Dale  wrote:
>> 
>> > Now if I can figure out how to reset the list of /dev/sd* names that 
>> > are lurking about and inconsistent, that would be like striking gold.
>> >  Every time I hook up my external drive, it gets a different sd* name.  
>> > It does the same on the SD cards from my trail cameras too but I can 
>> > auto mount those.
>> 
>> I don't think it is possible to get consistent sd* names for removable 
>> media. But you could use volume labels with `tune2fs -L` or `tune.exfat -L` 
>> or `fatlabel` and then mount them via /dev/disk/by-label/*.
>> 
>> Kind regards, tastytea
>> 
>> --
>> Get my PGP key with `gpg --locate-keys tasty...@tastytea.de` or at 
>> .

Don't even need to do that if you don't care about human beings too much.  
/dev/disk/by-id has similar links based on drive model and serial number and 
/dev/disk/by-uuid I think has them by filesystem uuid.  All of those should be 
pretty consistent.

LMP

RE: [gentoo-user] Switching from eudev to udev, disaster.

[Laurence Perkins]

So the old inconsistency was a super-bad kind of inconsistency.  The interfaces 
got named based on the order in which the devices were discovered.  Which, on a 
lot of systems, meant that every boot was essentially rolling the dice on a 
race condition.  If you only have one device, you're fine.  If your devices 
consistently come up in the same order, you're fine.  If there's jitter though 
then things can easily get messy, and do so unexpectedly.

The new naming scheme names devices based on where they show up on the bus.  
This has its own issues.  It means that USB adapters get different names when 
plugged into different slots.  It means that adding or removing other PCI bus 
devices can change the bus address and therefore the name of your network 
interfaces.  I've seen motherboard firmware updates do the same.  But, at least 
in theory, this inconsistency should be triggered by something you *know* about 
unless hardware is getting added and removed by someone else without your 
knowledge.

If you only have one interface though and tweak your hardware regularly then 
you'll probably be happier to put it back to the old naming scheme because with 
only one device it should always be eth0.

LMP

-Original Message-
From: Grant Taylor  
Sent: Tuesday, November 30, 2021 12:34 PM
To: gentoo-user@lists.gentoo.org
Subject: Re: [gentoo-user] Switching from eudev to udev, disaster.

On 11/30/21 12:58 PM, Dale wrote:
> What I noticed in dmesg is that it takes the old name, eth0 for 
> example, and then renames it to the new name.

I don't know if it's the /kernel/ that does the renaming, or not based on the 
kernel parameter, or if it's something else very early in the boot that does 
the renaming.

> Well, if one moves things around and eth0 becomes eth3 then doesn't 
> that mess up the new name as well?

My understanding is that the new name is -- supposed to be -- based off of some 
property of the device.  I assume that said property is from something akin to 
where lspci gets it's data.  Probably something exposed in /proc and / or /sys 
via the actual driver that ultimately gets feed into the renaming routine.

> That could be why you see the results you have.> It's hard to base a 
> name on something that is changing itself.

My understanding is that the new name is supposed to be completely independent 
from and not derived using the old name.  So the old naming should have no 
influence on the new name.

> It would seem to me that if they were going to change things for real, 
> they would change what the kernel names it in the beginning and it 
> uses the name it was first given based on slot or something else unique.

Agreed.  As in have the driver instantiate the device with the new name from 
the outset.

> In other words, have the kernel assign it enp2s3 or whatever when 
> booting and that is the only name it gets.

Yep.

I don't know /why/ or /where/ the failure is with the new names.  I just know 
that I have seen instability in them.  Seeing as how stability ~> 
predictability is the motivation for the rename, well, that's a failure in my 
opinion.

Besides, it's a LOT easier to /just/ `tcpdump -nni eth0` when logging into a 
machine than it is to have to figure out the interface name first.

That being said, I was okay with what CentOS 6.x did, where the new name was 
matched against the MAC address.  I had eth0 based on MAC for outside and eth1 
based on MAC for inside on a number of systems.



--
Grant. . . .
unix || die

RE: [gentoo-user] LLVM and friends is not compatible.

[Laurence Perkins]

>> 
>> 
>> -Original Message-
>> From: Wol  
>> Sent: Sunday, December 5, 2021 1:59 PM
>> To: gentoo-user@lists.gentoo.org
>> Subject: Re: [gentoo-user] LLVM and friends is not compatible.
>> 
>> On 05/12/2021 21:21, Alan Grimes wrote:
>> > I was playing Superliminal the other day and it wasn't doing too well, 
>> > probably my GPU is too rusty to run it well, and it took down X11. 
>> > Which meant it was time to update packages and reboot
>> > 
>> > Biig mistake...
>> > 
>> > Look, I know a lot of you are in the process of being killed to death 
>> > by the slow acting poison known as the covid 19 vaccine. I'm really 
>> > sorry about that neither myself nor God can do jack shit about that at 
>> > this point.
>> 
>> Let's hope we're not dying from the quick poison known as CoVid-19 (that 
>> said, I do my best to avoid both :-)
>> > 
>> > I also know that one of the several pathologies the poison inflicts is 
>> > a form of brain rot as prion crystals grow and disrupt neural pathways.
>> > =\  It really sucks and I wish things had not gotten to this point.
>> > 
>> > Portage is actually working, to some extent this time so I am actually 
>> > a bit shocked and amazed on that front, it's been ages since it 
>> > actually worked decently well.
>> > 
>> Just make sure you have a working python !!!
>> 
>> > But then a large number of absolutely critical things are now broken.
>> > 
>> > -> Xosview can't find the font it needs (!!! WTF ) None of my 
>> > -> steam games play anymore.
>> > -> The web browser and mail reader that I've been using since the '90s
>> > won't build or run. I've been reduced to using WEBMAIL FOR GOD'S SAKE! 
>> > to post this. I feel unclean.
>> 
>> Well my latest update seems to have fixed Thunderbird (which was randomly 
>> locking up input to the extent it was a Big Red Switch job to recover ...).
>> 
>> > -> Neither python 3.9 nor 3.10 will finish building. It appears they 
>> > -> run
>> > some self-tests which fail and then they just freeze up at 0% CPU 
>> > usage and I never get any clear diagnostic information.
>> > -> GCC-11.2 feels like it is mostly working but might be responsible 
>> > -> for
>> > python failing specific tests as it may have been using trick 
>> > optimizations that were hacks to start with but now don't work with 
>> > the current GCC which is actually working as advertised.
>> 
>> Do you still have Python 3.8? Can you delete 3.9 and 3.10?
>> 
>> Which gcc is default? can you eselect a newer working gcc than your current 
>> version?
>> > 
>> > I think one of the root issues is that llvm-13 deprecated several 
>> > command line options that broke a number of packages that took a great 
>> > many more packages down with them. I first just masked clang-13 but 
>> > that doesn't seem to be enough, I also masked llvm-13 and 
>> > llvm-common-13 and am running --emptytree world AGAIN.
>> > 
>> Well, what I always do then is first an "emerge system" (no extra options), 
>> then an emerge world, then finally an emerge deep newuse world. Just a 
>> little bit each time. Forcing a bit at a time.
>> 
>> The other thing is (as I found out ...) are any default emerge options 
>> screwing you over? I had to comment out my default options to get virtualbox 
>> to emerge update properly, so if you've got any, comment them out and see if 
>> it improves matters ...
>> 
>> I'm going to have more "fun" soon, I think ... I looked at funtoo linux 
>> earlier today, but I think that'll be a no-no - it goes about "wolves owning 
>> their own shit" then craps all over systemd. I get some people don't like 
>> it, but when people say "if you like it we don't want you here" then I am 
>> afraid I don't want to know bigots like that.
>> 
>> I then found SourceMage ... looks very interesting! It's a pretty basic 
>> distro it seems to me, but it also seems to be a distro that doesn't try to 
>> tell its users what to run. So I'll be playing ... and it doesn't give you 
>> all that portage grief! Not that I like the sound of bash ... 
>> :-) but all the basic system tools are written in it. They're very much "no 
>> dependency spaghetti", and that hopefully makes for easier admin...
>> 
>> The only problem is it seems very short-staffed. It's clearly up-to-date as 
>> in there's plenty of recent time stamps, but you have to look for them as 
>> much of the docu is 5, 10 years old ...
>> 
>> Cheers,
>> Wol
>> 
>> 

Source Mage is a spinoff of Sourceror and is kind of the opposite of Gentoo.  

Gentoo is a source-based distro for people who want things to mostly just work 
like with the binary distros, but also want to do customizations and 
optimizations easily.

Source Mage is a distro for Linux From Scratch folks who are tired of 
maintaining their own package manager.  They don't change *anything* from 
upstream in their packages, (which makes it really easy to keep "updated" on 
their end) but the package manager does have a lot of nice features for easily 
stor

RE: [gentoo-user] LLVM and friends is not compatible.

[Laurence Perkins]

>>  
>>  -Original Message-
>>  From: Wols Lists  
>>  Sent: Monday, December 6, 2021 11:02 AM
>>  To: gentoo-user@lists.gentoo.org
>>  Subject: Re: [gentoo-user] LLVM and friends is not compatible.
>>  
>>  On 06/12/2021 17:51, Laurence Perkins wrote:
>>  > Source Mage is a spinoff of Sourceror and is kind of the opposite of 
>> Gentoo.
>>  
>>  Well, I read the philosophy thing where it said it wasn't comparable with 
>> gentoo ...
>>  > 
>>  > Gentoo is a source-based distro for people who want things to mostly just 
>> work like with the binary distros, but also want to do customizations and 
>> optimizations easily.
>>  
>>  Unfortunately, I can be a bit gruff and not suffer fools gladly. Having had 
>> a run in with the bug-wranglers over an issue that completely screwed up my 
>> boot (nothing to do with gentoo, admittedly), but that exposed idiotic 
>> decisions / other bugs in genkernel, I think I want to look elsewhere.
>>  
>>  Let's take a 2x2 truth table - do I have a boot partition, do I have 
>> "automount boot" switched on. Three of the four options stomp all over the 
>> live boot partition. The fourth fatal errors with "wah wah why won't you let 
>> me stomp all over your live boot partition".
>>  
>>  The REASON I don't want it stomping all over that partition is the last 
>> time a distro (SUSE) did it, it completely trashed my boot leading to 
>> several hours debugging and messing about in the systemd rescue shell to get 
>> it bootable again. If anybody is going to trash my live boot, I'd rather it 
>> was me, not an Artificial Stupidity software manager.
>>  
>>  The wranglers' solution was simply to "use the no-install option" - except 
>> that that promptly crashed with "can't find input files". Huh? 
>>  Changing the OUTPUT destination makes the INPUT files disappear? wtf?
>>  > 

Genkernel is pretty...  special...  It's handy if your system is set up the way 
it expects.  If not, well, then its utility drops off quickly.

From what you're describing, my suggestion would be to simply only use it for 
initramfs generation and do the kernel make yourself.  That shouldn't "stomp 
all over" anything except maybe a previous initramfs for the same kernel.

Or else use dracut, it works decently for initramfs generation as well and lets 
you specify output location so you can copy things to /boot manually if you so 
desire.

Or, depending on how complex your system actually is, just creating your own 
initramfs isn't terribly difficult.  Since it seems like you're wanting 
explicit control over everything that might be the option that will keep you 
happiest in the long-run.

Because initramfs generation is really the only complex part of the process.  
The rest is just a wrapper for "make && make install && make modules_install" 
with a few extras of marginal utility like archiving your .config files.



>>  > Source Mage is a distro for Linux From Scratch folks who are tired of 
>> maintaining their own package manager.  They don't change*anything*  from 
>> upstream in their packages, (which makes it really easy to keep "updated" on 
>> their end) but the package manager does have a lot of nice features for 
>> easily storing whatever patches and configuration changes you choose to make 
>> in order to get it running on your system.
>>  
>>  Well, if I have to get into maintaining emerge to get it to behave 
>> sensibly, I might as well try somewhere else and see if it's an improvement.
>>  > 
>>  > If you've always wanted to try LFS but tracking package files and patches 
>> and configs and so-forth seemed daunting then it's definitely an awesome set 
>> of tools.
>>  > 
>>  > Otherwise it's kind of a lot of work...
>>  > 
>>  Well, given that I've got oodles of space (just added 3TB to my mirror to 
>> give myself a 5TB raid-5 /home lvm, along with 1TB root/ lvm, I've got 
>> plenty of space to play with distros. And I was shocked - 32GB of
>>  DD4 was just over £100, so my new system now has 11TB of hard drive, 32GB 
>> of RAM, and a hefty 4-core Ryzen processor :-)
>>  
>>  And the bits from shop screw-up mean the new raid testbed I'm building will 
>> be a reasonably hefty system too - 4x1TB drives for hammering with raid, 3TB 
>> backup drive, 16GB ram - just the thing for learning to kernel program :-)
>>  
>>  And seeing as I won't care about trashing it by mistake, I'll be playing 
>> with KVM, and all those other fancy technologies to try and run multiple 
>> distros stacked on top of each other :-)
>>  
>>  Cheers,
>>  Wol
>>  
>>  

Source-mage is definitely worth experimenting with if you want to get more into 
the nuts-and-bolts of how to create your own distro without necessarily going 
full LFS.  I haven't used it personally in a long time though since I kind of 
need to actually *use* the computer rather than just constantly maintaining it. 
 :D


LMP

RE: [gentoo-user] Steam overlay

[Laurence Perkins]

Nope.  

Check dmesg.  Check syslog.  Try launching the game from a terminal.  Even 
Steam games usually give *some* kind of hint.

LMP

-Original Message-
From: Alan Grimes  
Sent: Monday, December 6, 2021 11:59 PM
To: Gentoo User 
Subject: [gentoo-user] Steam overlay

CAUTION: This is an EXTERNAL email. Do not click links or open attachments 
unless you recognize the sender and know the content is safe.

Anyone have any idea why 90% of my steam games now instantly crash to desktop 
with no error message or anything?



--
Beware of Zombies. =O
#EggCrisis  #BlackWinter
White is the new Kulak.
Powers are not rights.

RE: [gentoo-user] Re: [OT] suggest SSD partitioning

[Laurence Perkins]

>> 
>> 
>> -Original Message-
>> From: Wols Lists  
>> Sent: Friday, December 10, 2021 11:25 AM
>> To: gentoo-user@lists.gentoo.org
>> Subject: Re: [gentoo-user] Re: [OT] suggest SSD partitioning
>> 
>> On 10/12/2021 15:16, Nikos Chantziaras wrote:
>> > If you can't do that, then it doesn't matter much whether you use a 
>> > swap file or partition. On an SSD, both should perform about the same. 
>> > On an HDD, swap files could run into fragmentation issues if you 
>> > resize them or create them incorrectly. On an SSD, fragmentation 
>> > doesn't have much of an impact. A swap file gives you the option to 
>> > resize it later on without having to do filesystem and partition 
>> > resizing, so I'd say a swap file sounds better.
>> 
>> It very much does matter whether you use a swap file or partition in 
>> practice. I've just been reading right now a discussion about systemd 
>> logging and hibernation, and how btrfs handles swap files. It sounds nasty.
>> 
>> If you have a swap file, linux creates an immutable file then uses direct 
>> disk i/o. There's a LOT of unnecessary crap there that could go wrong. Just 
>> avoid all that trouble and give yourself a decent swap partition. (And if 
>> you're running btrfs, a lot of this sounds experimental and dangerous ...)
>> 
>> Cheers,
>> Wol
>> 
>> 

For BTRFS I usually do one partition for the whole system and one partition for 
swap and then use subvolumes for /home and anything else I want to keep 
separate in case of reinstall.

Since BTRFS is a storage pool model, everything else can dynamically resize 
similarly to using LVM.

Swap files in general aren't as reliable if one is planning to hibernate the 
system.  Swap files on BTRFS should go through a loop device unless you set 
them up really carefully.

There's no reason you can't have both swap files and a swap partition.  I 
occasionally end up dynamically adding more when I get a program that uses a 
terabyte of virtual but very little resident at a time or something.

Swap onto zram devices can also be a useful tool if the data being swapped is 
more highly compressible than zswap will take advantage of.

LMP

RE: [gentoo-user] Local mail delivery agent (MDA) wanted

[Laurence Perkins]

>
>
>-Original Message-
>From: Grant Taylor  
>Sent: Tuesday, December 14, 2021 3:34 PM
>To: gentoo-user@lists.gentoo.org
>Subject: Re: [gentoo-user] Local mail delivery agent (MDA) wanted
>
>On 12/13/21 3:12 PM, Frank Steinmetzger wrote:
>>> Using strace, I found out that mail from mailx puts those mail into 
>>> /var/spool/clientmqueue/, one file per mail, but not in a maildir structure.
>
>Yes, the /var/spool/clientmqueue is the mail queue for outgoing messages from 
>clients.  Hence the name "client m(ail) queue".
>
>> OK, I found out that this is the usual outgoing queue which needs to 
>> be processed by sendmail, probably through another cronjob or a 
>> process that itself checks that directory periodically.
>
>Sendmail is quintessentially a daemon that's running all the time.  As such it 
>usually does it's own scheduling and does not depend on external scheduling.
>
>>> In many places I read that system mail—by default—goes into 
>>> /var/spool/mail/, but until now I’ve yet to observe this behavior.
>
>/var/spool/mail/ and /var/mail/ are the quintessential locations 
>for mbox based inbound email storage.
>
>Note:  There are a number of other fancy client mail storage routines that 
>don't use files in this path.
>
>> It’s really not easy to find a description of the default setup of 
>> olden days (or I’m simply using the wrong search terms). Because when 
>> you search for something like unix local mail setup, most results are 
>> about setting up an SMTP server. In hindsight—perhaps that is simply 
>> the way to go. :-/
>You will quite likely need a Mail Transfer Agent to receive the email, either 
>via command (mail(x) / sendmail / etc) or read from a queue location like 
>/var/spool/clientmqueue and then deliver the messages to where they belong.
>
>There /may/ be an alternate "mail" command that does all of this in one 
>function.  But I'd be surprised to learn about such.
>
>Most of the surprise is because it would be combining three distinct parts of 
>the email flow:  the Mail User Agent (a.k.a. MUA) generating the original 
>outgoing message, the Message Transfer Agent (a.k.a. MTA) to receive the 
>original message and do something with it, and the Local Delivery Agent 
>(a.k.a. LDA) to put the message in the proper location.
>
>The originating MUA can frequently be substituted at will with "mail", 
>"mailx", and "nail" being three CLI based that come to mind immediately.
>
>The MTA can frequently be one of many with Sendmail, Postfix, Courier, Exim 
>coming to mind.
>
>The LDA can easily be one of the following; procmail, maildrop, Courier,
>  and something super simple I don't remember the name of because I've not 
> used it in so long.
>
>
>
>--
>Grant. . . .
>unix || die
>
>

So one thing that's annoyed me for a while is that there are several things 
which will pull in nullmailer to accept local mails, but don't pull in anything 
to do local delivery (And I'm not sure if nullmailer can even pass things to 
local delivery) so your local delivery mails by default just stack up in the 
nullmailer outbound queue unless you configure it to pass them off to an 
external mail system.

Since the most commonly used of these programs are things like cron where local 
delivery is probably the only thing most users would care about it might be 
nice if the default configuration were one that does that, and then those who 
want local mail relayed elsewhere still don't have any significant extra setup 
work to do.

LMP

RE: [gentoo-user] Movie editing softeware

[Laurence Perkins]

>>
>>
>>-Original Message-
>>From: Wols Lists  
>>Sent: Tuesday, December 21, 2021 2:03 PM
>>To: gentoo-user@lists.gentoo.org
>>Subject: Re: [gentoo-user] Movie editing softeware
>>
>>On 21/12/2021 19:16, Dale wrote:
>>> Spackman, Chris wrote:
 On 2021/12/21 at 05:13pm, Wols Lists wrote:
> On 21/12/2021 16:07, Spackman, Chris wrote:
>> On 2021/12/20 at 11:17am, William Kenworthy wrote:
>>> Hi, what is a usable piece of software in portage to do a quick 
>>> edit of a movie? (cut start/end and maybe splice a bit in/out of 
>>> the middle?)
>> I've not seen anyone mention OpenShot. It is in portage, but masked 
>> for some reason.
>> As someone else mentioned, you do have to create a project and then 
>> export. Really, though, the "create project, make export choices" 
>> is only like an extra minute or two of your time. I usually don't 
>> even save the project for just some simple trimming.
   
> It may be a minute or two of YOUR time.
>
> For someone who doesn't "DO" video editing, it can easily turn into 
> hours of debugging trying to work out what does (or doesn't) work.
>
> Sorry, but you can't assume we're all video whizzes like you ... :-) 
> That's why we want something dummy-proof!
 Wow, sorry, didn't realize this was such a sore issue. Especially 
 considering we've already discussed several command line programs in 
 this thread.

 To export:

 1. press the red circular "export video" button (or go to File =>
 Export Project => Export Video);

 2a. [optional] change the name of the video in the export window that
  pops up;

 2b. press the "export video" button at the bottom of the window. Here,
  for me, the defaults work fine.

 I did it three times in less than a minute, double checking the 
 accuracy for this post. So, not a huge inconvenience.

 Of course, if the defaults do NOT work for you, then you do not want 
 something "dead easy", you want something that will read your mind 
 and do what you want, somehow, automagically.

>>> 
>>> 
>>> As someone who has experimented with video editing software, I can 
>>> understand Wols on this.  What some of us needs is something similar 
>>> to 'video editing for dummys' except we need the software not the 
>>> book.  At one time, I wanted to remove like 20 or 30 seconds on the 
>>> beginning and about the same on the end of a few videos.  Hours later, 
>>> still couldn't figure it out.  Heaven forbid I wanted to remove 
>>> something in the middle as well or add a second or so of black screen.  
>>> O_O
>>> 
>>> This coming from someone who was able to figure out Kicad and get 
>>> circuit boards made.  Just saying.  LOL
>>> 
>>Yup. This coming from someone who is quite happy with the command line 
>>because when he started THAT'S ALL THERE WAS. My work experience pre-dates 
>>the IBM PC - you know the one - the one with an 8088 inside...
>>
>>Oh - and as for using the command line, it's all very well until you try and 
>>figure out where to tell the command line to cut the video file - I really 
>>don't want to have to run the command line hundreds of times, checking the 
>>output every time, and throwing away the ones that cut in the wrong place. 
>>Oh, and if I use some video editing software to find the exact millisecond I 
>>want to cut, it STILL cuts it in completely the wrong place ...
>>
>>Cheers,
>>Wol
>>
>>

Note that some editing software can only cut at the iframes, and it's also 
fairly common to only be able to cut at the iframes unless it's re-encoding the 
data.  That can be a major source of frustration.

LMP

RE: [gentoo-user] Re: Movie editing softeware

[Laurence Perkins]

>
>-Original Message-
>From: Grant Edwards  
>Sent: Wednesday, December 22, 2021 8:18 AM
>To: gentoo-user@lists.gentoo.org
>Subject: [gentoo-user] Re: Movie editing softeware
>
>On 2021-12-21, Laurence Perkins  wrote:
>
>> Note that some editing software can only cut at the iframes, and it's 
>> also fairly common to only be able to cut at the iframes unless it's 
>> re-encoding the data.
>
>AFAIUI, it's not even theoretically possible to cut anyplace other than the 
>I-frames without decoding and reencoding at least the portion of the stream 
>where the cut is being made. While it might be possible to copy the rest of 
>the stream, I don't know of any editors that will do that.
>
>--
>Grant
>
>

Oh, you can do it, it'll just mess up every frame between where you cut it and 
the next iframe so sensible software that doesn't want to reencode it refuses 
to do so.
But the playback decoders won't generally choke on it or anything.

It's basically the same as a full+incremental backup scheme, only with video 
frames.  Chop some diffs out of the middle and the stuff between there and the 
next full goes... strange.

But what I meant was that cutting to the whole GOP is ALL that some software 
can do and it won't even try to reencode.

LMP

RE: [gentoo-user] Synchronous writes over the network.

[Laurence Perkins]

>> 
>> -Original Message-
>> From: Wols Lists  
>> Sent: Thursday, December 23, 2021 2:29 PM
>> To: gentoo-user@lists.gentoo.org
>> Subject: Re: [gentoo-user] Synchronous writes over the network.
>> 
>> On 23/12/2021 21:50, Mark Knecht wrote:
>> > In the case of astrophotography I will have multiple copies of the 
>> > original photos. The process of stacking the individual photos can 
>> > create gigabytes of intermediate files but as long as the originals 
>> > are safe then it's just a matter of starting over. In my 
>> > astrophotography setup I create about 50Mbyte per minute and take 
>> > pictures for hours so a set of photos coming in at 1-2GB and up to 
>> > maybe 10GB isn't uncommon. I might create 30-50GB of intermediate 
>> > files which eventually get deleted but they can reside on the server 
>> > while I'm working. None of that has to be terribly fast.
>> 
>> :-)
>> 
>> Seeing as I run lvm, that sounds a perfect use case. Create an LV, dump the 
>> files on it, when you're done unmount and delete the LV.
>> 
>> I'm thinking of pulling the same stunt with wherever gentoo dumps its build 
>> files etc. Let it build up til I think I need a clearout, then create a new 
>> lv and scrap the old one.
>> 
>> Cheers,
>> Wol
>> 
>> 

The locations where Portage drops build files, package files, and source 
archives are set in make.conf if you really want to do this.
But the build files get deleted automatically when finished unless there was an 
error or you specifically told Portage not to, and the "eclean" tool will clean 
up the stale things in the other locations without deleting stuff that's 
actually still useful.

LMP

RE: [gentoo-user] Synchronous writes over the network.

[Laurence Perkins]

>> -Original Message-
>> From: Wols Lists  
>> Sent: Thursday, December 23, 2021 9:54 AM
>> To: gentoo-user@lists.gentoo.org
>> Subject: Re: [gentoo-user] Synchronous writes over the network.
>> 
>> > As always I'm interested in your comments about what works or 
>> > doesn't work about this sort of setup.
>> > 
>> My main desktop/server currently has two 4TB drives split 1TB/3TB. The two 
>> 3TB partitions are raid-5'd with a 3TB drive to give me 6TB of /home space.
>> 
>> I'm planning to buy an 8TB drive as a backup. The plan is it will go into a 
>> test-bed machine, that will be used for all sorts of stuff, but it will at 
>> least keep a copy of my data off my main machine.
>> 
>> But you get the idea. If you get two spare drives you can back up on to 
>> them. I don't know what facilities ZFS offers for sync'ing filesystems, but 
>> if you're go somewhere regularly, where you can stash a hard disk (even a 
>> shed down the bottom of the garden :-), you back up onto disk 1, swap it for 
>> disk 2, back up on to disk 1, swap it for disk 2 ...
>> 
>> AND YOUR BACKUP IS OFF SITE!
>> 
>> Cheers,
>> Wol

Data does not exist unless it exists in at least three places.  Assume your 
most recent backup will also have a problem and plan accordingly with regard to 
time and number of copies.

Back in the day the rule of thumb was that whatever losing the data would cost 
you, you should probably spend about a third of that on redundancy.

LMP

RE: [gentoo-user] Synchronous writes over the network.

[Laurence Perkins]

>> -Original Message-
>> From: Rich Freeman  
>> Sent: Thursday, December 23, 2021 9:50 AM
>> To: gentoo-user@lists.gentoo.org
>> Subject: Re: [gentoo-user] Synchronous writes over the network.
>> 
>> On Thu, Dec 23, 2021 at 12:39 PM Mark Knecht  wrote:
>> >
>> > I'll respond to Rich's points in a bit but on this point I think 
>> > you're both right - new SSDs are very very reliable and I'm not overly 
>> > worried, but it seems a given that forcing more and more writes to an 
>> > SSD has to up the probability of a failure at some point. Zero writes 
>> > is almost no chance of failure, trillions of writes eventually wears 
>> > something out.
>> >
>> 
>> Every SSD has a rating for total writes.  This varies and the ones that cost 
>> more will get more writes (often significantly more), and wear pattern 
>> matters a great deal.  Chia fortunately seems to have died off pretty 
>> quickly but there is still a ton of data from those who were speculating on 
>> it, and they were buying high end SSDs and treating them as expendable 
>> resources - and plotting Chia is actually a fairly ideal use case as you 
>> write a few hundred GB and then you trim it all when you're done, so the 
>> entirety of the drive is getting turned over regularly.  People plotting 
>> Chia were literally going through cases of high-end SSDs due to write wear, 
>> running them until failure in a matter of weeks.
>> 
>> Obviously if you just write something and read it back constantly then wear 
>> isn't an issue.
>> 
>> Just googled the Samsung Evo 870 and they're rated to 600x their capacity in 
>> writes, for example.  If you write 600TB to the 1TB version of the drive, 
>> then it is likely to fail on you not too long after.
>> 
>> Sure, it is a lot better than it used to be, and for typical use cases I 
>> agree that they last longer than spinning disks.  However, a ZIL is not a 
>> "typical use case" as such things are measured.
>> 
>> --
>> Rich
>> 
>> 

This is also why the video surveillance industry still uses spinning rust for 
anything beyond a very minimal capacity.  

Rotating drives wear out primarily based on time run as long as you're not 
thrashing the heads all the time by running Windows (Windows 10 seems to have 
given up even trying to optimize read and write on non-SSD media.)

SSD wears out primarily based on write throughput, and anything with a large 
turnover can easily wear one out in a matter of months.

LMP

RE: [gentoo-user] TLD for home LAN?

[Laurence Perkins]

-Original Message-
From: tastytea  
Sent: Saturday, January 15, 2022 2:54 AM
>To: gentoo-user@lists.gentoo.org
>Subject: Re: [gentoo-user] TLD for home LAN?
>
>On 2022-01-15 10:33+ Peter Humphrey  wrote:
>
>> Hello list,
>> 
>> Rich F said recently, "I'd avoid using the .local TLD due to RFC 
>> 6762."
>> 
>

Glancing at that RFC, if you want to be compliant (which, it's *your* network, 
and this is all link-local only, so there's no reason you *have* to be), all 
you need to do is make sure that any .local addresses you assign are resolvable 
via multicast DNS.

The standard does not prohibit the names being resolvable via unicast DNS as 
well, though it does recommend that you make sure the two resolution paths 
return consistent results since most systems will take the first response they 
get.

If you assign .local addresses which aren't resolvable via mdns then there is 
the possibility that some piece of software will only check mdns and not 
regular dns before grabbing a name and cause a conflict.  But as long as it's 
your network and your devices that's not terribly hard to sort out should it 
ever happen.

LMP

RE: [gentoo-user] TLD for home LAN?

[Laurence Perkins]

>>-Original Message-
>>From: Rich Freeman  
>>Sent: Tuesday, January 18, 2022 11:41 AM
>>To: gentoo-user@lists.gentoo.org
>>Subject: Re: [gentoo-user] TLD for home LAN?
>>
>>On Tue, Jan 18, 2022 at 12:28 PM Laurence Perkins  
>>wrote:
>>>
>>> The standard does not prohibit the names being resolvable via unicast DNS 
>>> as well, though it does recommend that you make sure the two resolution 
>>> paths return consistent results since most systems will take the first 
>>> response they get.
>>
>>If a host queries DNS first, and obtains an NXDOMAIN from an authoritative 
>>name server, I'm not sure most would even check mDNS.  I think I had that 
>>issue back when I was using .local before I heard of zeroconfig.
>>

Right.  If you have .local names registered with your DNS, but not resolvable 
via mDNS, then if you plug in a device which tries to assign a conflicting name 
and it only checks mDNS for conflicts (the standard only *says* to check mDNS 
for conflicts, even though checking DNS as well would seem to always be a good 
idea) you could end up with DNS and mDNS returning different results.  This can 
be confusing if different devices have different resolver preference orders.

So, if you're going to have your DNS resolve .local names it's a really good 
idea to provide those names via mDNS as well.  At least if there's any 
significant chance of a conflict arising.

LMP

RE: [gentoo-user] TLD for home LAN?

[Laurence Perkins]

>
>-Original Message-
>From: Rich Freeman  
>Sent: Tuesday, January 18, 2022 12:50 PM
>To: gentoo-user@lists.gentoo.org
>Subject: Re: [gentoo-user] TLD for home LAN?
>
>On Tue, Jan 18, 2022 at 3:12 PM Laurence Perkins  wrote:
>>
>>
>>
>> >>-Original Message-
>> >>From: Rich Freeman 
>> >>Sent: Tuesday, January 18, 2022 11:41 AM
>> >>To: gentoo-user@lists.gentoo.org
>> >>Subject: Re: [gentoo-user] TLD for home LAN?
>> >>
>> >>On Tue, Jan 18, 2022 at 12:28 PM Laurence Perkins  
>> >>wrote:
>> >>>
>> >>> The standard does not prohibit the names being resolvable via unicast 
>> >>> DNS as well, though it does recommend that you make sure the two 
>> >>> resolution paths return consistent results since most systems will take 
>> >>> the first response they get.
>> >>
>> >>If a host queries DNS first, and obtains an NXDOMAIN from an authoritative 
>> >>name server, I'm not sure most would even check mDNS.  I think I had that 
>> >>issue back when I was using .local before I heard of zeroconfig.
>> >>
>>
>> Right.  If you have .local names registered with your DNS, but not 
>> resolvable via mDNS...
>
>No, I'm talking about the opposite situation.  I'm talking about you have 
>foo.local resolvable via mDNS, but not DNS - then there is a chance you won't 
>be able to access the host.  Basically having an authoritative nameserver for 
>.local may disable mDNS on your network for some devices.
>
>--
>Rich
>
Yeah, I can see that also being a possibility, and it's likely to be annoying 
to deal with since different clients can have different preferred resolver 
orders and may or may not take the authoritative part seriously and any program 
hard-coded to use mDNS will work fine, so there could be a big wad of 
inconsistent behaviour that obscures what's going on.

LMP

RE: [gentoo-user] Handling a sizable amount of spam and Dovecote question

[Laurence Perkins]

> -Original Message-
> From: Marco Rebhan  
> Sent: Thursday, January 20, 2022 6:07 AM
> To: gentoo-user@lists.gentoo.org
> Subject: Re: [gentoo-user] Handling a sizable amount of spam and Dovecote 
> question
> 
> On Thursday, 20 January 2022 14:22:02 CET Dale wrote:
> > What do others do with spam to minimize it?
> 
> Hi Dale,
> 
> I'm not sure if you're talking about self-hosted mail because you mention 
> dovecot, if you do:
> 
> I use postfix's smtp_recipient_restrictions to block mail coming from servers 
> marked as spam by RBLs:
> 
> smtpd_recipient_restrictions =
> reject_rbl_client zen.spamhaus.org,
> reject_rbl_client bl.spamcop.net
> 
> I'm not sure if this is doable through Dovecot configuration, but without an 
> MTA like Postfix you can't receive mail anyway. This alone seems to block 
> most of the spam I get. And additionally I have rspamd and some filters set 
> up because some stuff does get through. I think Postfix also lets you block 
> IP ranges directly in case you get spammed by some network that isn't listed 
> on spamhaus but I haven't needed that yet.
> 
> > I'm considering setting
> > up another email address and switching then closing current account. 
> > Yea, it's that annoying.  Is there someone I can report them too?  Is 
> > there something I'm not recognizing in the message headers that I can 
> > use to report them too?
> 
> In case you're using some other email provider (is this about your Gmail 
> address?), what you can do regardless is set up filters (ideally server- side 
> if they provide the capability...) filtering mails by e.g. From header (I 
> have a whole list of those), and if they have a well configured spam filter 
> you should be able to move the spam mail into your spam folder and it should 
> eventually start to classify similar mail as spam automatically. (but this is 
> specific to the service so I can't tell you a way that works everywhere)
> 
> > By the way, I have dovecote set up and the service seems to start. 
> > What do I do after getting the service to start to set up where to get 
> > email etc?
> 
> If you aren't self-hosting your mail but want to (be warned, it's a fair bit 
> of setup connecting it all together), you first need a domain, a server with 
> a static IP address (don't use some box in your home) that has the correct 
> rDNS record set in addition to DNS, so the hostname can be resolved from its 
> public IP. To actually be able to receive mail, you also need an MTA. Dovecot 
> is just a way to access a mailbox, it doesn't actually handle receiving mail 
> from other servers or sending mail. 
> Postfix is what I use for that, they work well together. I followed these 
> wiki articles among some others I can't find right now to set it up initially:
> https://wiki.archlinux.org/title/Mail_server
> https://wiki.archlinux.org/title/Postfix
> https://wiki.archlinux.org/title/Dovecot
> 
> I hope this helps!
> 
> -Marco
> 

Note that you don't technically need a static IP address, and a box in your 
home works just fine as long as you have sufficient bandwidth.  But most 
residential ISPs in the USA at least block port 25 due to a technically flawed 
recommendation from the federal government that was made in the '90s when 
everybody was on dialup.  The recommendation, of course, is that *everybody* 
should block both incoming and outgoing port 25 *all* the time...  Which 
*would* meet the stated goal of eliminating spam email...  

Fortunately you can generally pay an ISP an extra fee to get port 25 unblocked. 
 This usually also includes getting a static IP address.  But the system will 
work with dynamic IP addresses just fine as long as your dynamic DNS client is 
configured correctly.

Note that if you're going to host your own email you need to test the security 
of your DNS provider.  There have been some notable cases where a few of them 
were more than willing to help a scammer with vague knowledge of their victim's 
personal info guess their way into the account and then start intercepting 
mail.  Specifically password reset emails.  So find a DNS provider that will 
let you set up strong security on the account, like all password reset requests 
needing to be notarized or something.

LMP

RE: [gentoo-user] Crypt Ease Series

[Laurence Perkins]

Anybody know if these are phishing attempts or just cluelessness?  They all 
seem to be coming from the same email address, so if we think it's the former 
we should probably block it.

If it's the latter, then would the sender please give us some context about 
what you're hoping to get for a reply?  None of these things are particularly 
interesting to much of anyone on this list I expect.

LMP

-Original Message-
From: xbx  
Sent: Thursday, January 20, 2022 11:27 AM
To: gentoo-user@lists.gentoo.org
Subject: [gentoo-user] Crypt Ease Series

CAUTION: This is an EXTERNAL email. Do not click links or open attachments 
unless you recognize the sender and know the content is safe.

Hello Gentoo Users Developers!

I would like to share, contribute my Encryption / Decryption algorithm.

It uses + -, and bitwise xor to encrypt data.

See the attached files.

The last is the CryptEaseSuper-MAX the use 8 x ( 8 x 2) 2 ^ 64 bit integer to 
encrypt / decrypt information.

Sincerely xbx.

RE: [gentoo-user] Crypt Ease Series

[Laurence Perkins]

Yeah, the only reason I even bother to ask instead of just trashing them all is 
that it's doing a decent job of seeming like a clueless noob who barely knows 
English trying to start a discussion.  So if that's the case then they should 
take this one chance to say so clearly and distinctly.

But it's probably just a phishing program by someone who thinks they understand 
computers.

Is there a virus scanner checking the list attachments for blatant things at 
least?

LMP

-Original Message-
From: Wol  
Sent: Thursday, January 20, 2022 12:14 PM
To: gentoo-user@lists.gentoo.org
Subject: Re: [gentoo-user] Crypt Ease Series

Given that anybody with half a clue knows NEVER to open unsolicited 
attachments, I'm verging on thinking it's phishing. Anyways, I'm THAT close to 
setting up an auto-delete rule on anything from him. The email address feels 
spammy, too ...

Cheers,
Wol

On 20/01/2022 19:57, Laurence Perkins wrote:
> Anybody know if these are phishing attempts or just cluelessness?  They all 
> seem to be coming from the same email address, so if we think it's the former 
> we should probably block it.
> 
> If it's the latter, then would the sender please give us some context about 
> what you're hoping to get for a reply?  None of these things are particularly 
> interesting to much of anyone on this list I expect.
> 
> LMP
> 
> -Original Message-
> From: xbx 
> Sent: Thursday, January 20, 2022 11:27 AM
> To: gentoo-user@lists.gentoo.org
> Subject: [gentoo-user] Crypt Ease Series
> 
> CAUTION: This is an EXTERNAL email. Do not click links or open attachments 
> unless you recognize the sender and know the content is safe.
> 
> Hello Gentoo Users Developers!
> 
> I would like to share, contribute my Encryption / Decryption algorithm.
> 
> It uses + -, and bitwise xor to encrypt data.
> 
> See the attached files.
> 
> The last is the CryptEaseSuper-MAX the use 8 x ( 8 x 2) 2 ^ 64 bit integer to 
> encrypt / decrypt information.
> 
> Sincerely xbx.
> 

RE: [gentoo-user] gnu-screen split region can't invoke shell

[Laurence Perkins]

Note also that “regions” aren’t actually full children of sessions.  If you set 
up a bunch of regions in a session in one terminal, and then attach to that 
same session in a different terminal, the only evidence of the regions will be 
the odd sizes of some of the windows.  So you can have the same screen session 
attached from different places with different region layouts simultaneously.

LMP

From: russian sky 
Sent: Tuesday, March 1, 2022 5:21 AM
To: gentoo-user@lists.gentoo.org
Subject: Re: [gentoo-user] gnu-screen split region can't invoke shell

> As far as I know you need to use 'Ctrl-a n' to move the screen focus > into 
> the next region of the split screen and then 'Ctrl-a c' to start > a window 
> in this split region with a shell in it. Otherwise it > remains empty. > > 
> You could also use 'Ctrl-a 0' to display the content of the first > region of 
> the screen in the second split region - mirroring what the > first region 
> shows. > > 'Ctrl-a tab' switches focus between regions. 'Ctrl-a n' switches 
> the > displayed window within a region to the next window which has an > 
> active shell in it, within the screen session. Instead of 'n' for > next, or 
> 'p' for previous, you can enter the number of the window, > with 0 being the 
> first window in the screen session. > > I'm not sure if I explained it an 
> understandable way, but I think > with a bit of experimentation you'll soon 
> understand how screen > sessions, windows with shells and split regions work.

After rereading the info(gnu-screen) manual, which indeed clarifies that

'Ctrl-a S' will generate a blank window.

It works after following your advice, thanks



The gnu-screen runs a litte bit different from tmux which make me

confusing. Good thing is the structure

(sessions --->>> regions --->>> windows) becomes clear, thanks again

RE: [gentoo-user] planning a new machine : comments welcome

[Laurence Perkins]

With regard to SMR drives, note that there are three basic types:

Some completely hide the fact that they are SMR.  These suck, hands down.  
Performance is unpredictable and random.
Some at least advertise that they're SMR, and expose basic counters about where 
they are in their maintenance cycles.  These still suck, but at least you can 
kind of predict when they're about to get really slow.

The best ones actually advertise what the shingled ranges are, at which point a 
new enough kernel and filesystem can keep the writes to those ranges as 
sequential as possible, and you can use the big, cheap drives with very little 
performance loss.

There are a couple articles explaining how to determine what you've got and 
optimize it.  I don't have my bookmarks to hand, but it was in a discussion on 
this list a few months ago.

LMP

-Original Message-
From: Dale  
Sent: Wednesday, March 2, 2022 2:01 PM
To: gentoo-user@lists.gentoo.org
Subject: Re: [gentoo-user] planning a new machine : comments welcome

Wols Lists wrote:
> On 02/03/2022 19:39, Philip Webb wrote:
>
>>
>> [ Memory : Kingston has always performed well &  16 GB  sb adequate ]
>>
>>    220222 Memory : MEKIT00581 : Kingston : 2 x 8 = 16 GB   
>> $ 110
>>     DDR4 3600 MHz CL17 dual channel
>
> What's the largest chip a slot takes? If you can afford it, buy two of 
> those. My new mobo takes 16GB, and they were only GB£50 each - £100.
> If you ever want to upgrade the ram, you won't need to "chuck and 
> replace".

I agree with this because I've made this mistake, twice.  Find out the largest 
size that will fit in each slot and buy to enough of them to cover your needs.  
When you need to expand, just plug in a extra stick, or two. 

Depending on what you do, 16GBs may be enough.  I have 32GBs on mine and quite 
often during updates I hit 20GBs of memory in use.  I'm not including cache or 
anything, actually memory in use by the system.  If you need to save on cash 
and upgrade later, one could likely adjust the memory used during large updates 
by using load options etc with emerge. I don't know why but it seems I get hit 
with a Firefox, libreoffice and some other large package at the same time quite 
often.  It takes up a lot of memory doing those.  I just thought it worth a 
mention but you may have a different demand than I do. 

>>
>> [ HDD : for back-up + alternative OS (Mint).
>> ANB5 has Seagate : anyone prefer W Digital & if so, why ? ]
>>
>> * 220222 HDD : HDSEA00144 : Seagate : 2 TB $  55
>>  SATA 3,5" 7200 RPM
>
> BarraCuda? That's an SMR drive. Look at the FireCuda instead or even 
> better an IronWolf. Only snag is, it'll push the price up. Umm ... the 
> firecuda costs a lot more than I thought - a 4TB IronWolf is £84.

On this, good point.  I have a SMR drive, didn't know it when I bought it, and 
I use it for a external backup that is done weekly.  Even with that, it is a 
bit annoying.  Even when it is done, it isn't really done yet.  I would not use 
this for anything where you might have a lot of large writes.  Once it hits a 
certain point, varies by drive, it gets slow and can get really slow.  I've 
seen old IDE drives that could be faster.  If unsure, post the model and I'm 
sure someone can tell you if it is CMR or SMR.  CMR = good, SMR = bad in some, 
maybe most, circumstances.  I don't know how true this is but I've read SMR and 
RAID is bad, btfrs(sp?) and similar file systems can also be bad.  Research and 
weigh based on what will work for you.  SMR can save you some cash, sometimes. 

>>
>> [ Case : ANB5 has Deepcool, which seems a good price ]
>>
>> * 220223 Case : CSDCL00019 : Deepcool : $  55
>>   E-Shield 120 mm Fan  Radiator Support
>>   E-ATX/ATX/MicroATX/MiniITX
>>  00032 : Matrexx
>> 50    $  60
>>  00035 : Matrexx 55
>> MESH   $  70
>>   PSU Shroud
>>  00044 : Macube 310P
>> WH    $  80
>>   Magnetic/CableManagement/FAN HUB Pre-install
>>
>> [ Power : ANB5 has Thermaltake : is  700 W  likely to be adequate ? ]
>
> Crumbs. For the system you're spec'ing, 250W is probably adequate.
> That said, at $60 I wouldn't skimp on a decent supply. A stressed 
> supply is more likely (a) to blow, and (b) to take out components with 
> it. That $60 is money well spent. My PSUs are Corsairs, for no 
> particular reason.

I agree with this.  A badly designed power supply can really cost you if it 
goes out.  I think mine is a Thermaltake but only a few models are really good. 
 There is a thread on overclockers forum that lists power supplies by model and 
if they passed the tests they ran.  It also explains how good the protection is 
when the power supply goes bad.  I don't know how up to date that list is but 
don't skimp to much on this. I had a fan go out on one of my 

RE: [gentoo-user] planning a new machine : comments welcome

[Laurence Perkins]

> 
> 
> -Original Message-
> From: Dale  
> Sent: Wednesday, March 2, 2022 5:08 PM
> To: gentoo-user@lists.gentoo.org
> Subject: Re: [gentoo-user] planning a new machine : comments welcome
> 
> Laurence Perkins wrote:
> > With regard to SMR drives, note that there are three basic types:
> >
> > Some completely hide the fact that they are SMR.  These suck, hands down.  
> > Performance is unpredictable and random.
> > Some at least advertise that they're SMR, and expose basic counters about 
> > where they are in their maintenance cycles.  These still suck, but at least 
> > you can kind of predict when they're about to get really slow.
> >
> > The best ones actually advertise what the shingled ranges are, at which 
> > point a new enough kernel and filesystem can keep the writes to those 
> > ranges as sequential as possible, and you can use the big, cheap drives 
> > with very little performance loss.
> >
> > There are a couple articles explaining how to determine what you've got and 
> > optimize it.  I don't have my bookmarks to hand, but it was in a discussion 
> > on this list a few months ago.
> >
> > LMP
> >
> 
> That's some new info.  I tend to follow threads, even started one ages ago 
> about my hard drive doing a bumpy thing for a long time after I updated my 
> backups.  Rich plus others informed me I unknowingly bought a SMR drive.  I 
> think mine has about a 15 or 20GB CMR section.  I've noticed if my updates go 
> to about that much or more, it gets slow. Either way, it does the bumpy thing 
> for a good while after my backups are done and I've unmounted the drive.  I 
> just let it sit there until it gets done.  If I don't, it just slows down 
> faster the next time because it starts out behind on moving the files from 
> CMR to SMR and doing its rewrite thing. 
> 
> My biggest point for the OP, look at its use and pick what works as expected. 
>  I've read, and Wol seems to confirm this, that RAID and SMR do not go 
> together well.  I've read some have hosed RAID thingys when they put in a SMR 
> drive and didn't know it. 
> 
> The biggest problem I have is when they don't let us know when a drive is 
> SMR.  I don't like a company that sells me something that isn't as good 
> without telling me.  It sort of rubs me the wrong way. 
> 
> To the OP tho, research first, then buy.  Know what you getting and that it 
> will work for your needs.  As I said, for the most part, my backup drive 
> being SMR is mostly a little annoying.  It does work.  I just won't do it 
> again tho.
> 
> Dale
> 
> :-)  :-) 
> 
> 

Yeah, A lot of RAID controllers will see the bumpy performance and error out.  
Some vendors are starting to update their firmware though.
You're not alone in being upset about manufacturers trying to sneak SMR in 
without telling anyone.  The CCTV industry kind of took them to the woodshed 
over it.
BTRFS got patches for detecting and optimizing zoned drives in 2020.  
EXT4 has a format-time option for it at this point.  
F2FS and NILFS2 are both designed to write a disk end-to-end sequentially for 
wear-levelling purpose and while they're not the fastest, they also don't see 
significant performance degradation from SMR except during schedulable 
maintenance operations.
https://zonedstorage.io/ has a good list of reference material for figuring out 
what you've got and getting it configured in the best way.

LMP

RE: [gentoo-user] planning a new machine : comments welcome

[Laurence Perkins]

There’s nothing actually wrong conceptually with SMR drives in RAID.  The write 
order used by the RAID system simply needs to be appropriate for such a drive.  
The early SMR drives tried to hide what they were, and simply didn’t have 
sufficient cache area for non-sequential workloads in any volume.  This is 
frankly a QA failure on the part of WD, Seagate, and Toshiba.  They assumed 
that most RAID systems were for bulk storage, SMR drives are great for getting 
more bulk storage in a smaller space, why wouldn’t that be awesome?  They 
obviously never did more than cursory testing.  The fact that some of these 
drive-managed units also didn’t actually succeed at hiding what they were and 
would return strange errors under certain circumstances probably didn’t help 
things.  Basically alpha technology pushed out in secret and it failed to 
perform.  There’s been at least one lawsuit about it along false advertising 
lines.

However, over the last year, there has been a lot of work done on updating 
filesystems, raid controllers, kernels, and everything else to handle zoned 
storage, I’ve seen firmware updates going past for a lot of hardware RAID 
controllers, along with updates for mdraid, BTRFS, ZFS, etc.  So as long as you 
get drives that are either host-aware or host-managed and have the very latest 
software for your setup they should no longer crash and burn on a RAID rebuild.

I’d probably wait another year for all the bugs to work their way out of the 
system before trying it on purpose, but if you have non-critical systems to 
play with, well, the SMR drives are rather a lot cheaper…  I got a few for one 
of my server chassis because they can cram 5TB of storage into a 2.5” drive 
that fits in the bay for only $120.

LMP

From: Julien Roy 
Sent: Thursday, March 3, 2022 2:55 PM
To: Gentoo User 
Subject: Re: [gentoo-user] planning a new machine : comments welcome

Even of WD's official page for WD Reds, they are still advertised for RAID: 
https://www.westerndigital.com/products/internal-drives/wd-red-sata-hdd#WD20EFAX

For instance :
●  Reliability: The always-on environment of a NAS or RAID is a hot one, and 
desktop drives aren’t typically designed and tested under those conditions like 
WD Red™ is.
●  Error recovery controls: WD Red™ NAS hard drives are specifically designed 
with RAID error recovery control to help reduce failures within the NAS system.

Or:

Designed with SMR technology for workloads associated with personal and home 
office, such as storing, archiving and sharing, in RAID-optimized NAS systems 
with up to 8 bays.
This isn't just NewEgg or Amazon puting outdated information on the product 
page, it is WD still advertising today that the SMR WD Reds are appropriate for 
RAIDs, while they are, in fact, not.

Regarding the blogpost: I don't think a blogpost is appropriate for a company 
to correct information about deffective product, when the advertising for the 
said product remains uncorrected.
Nevertheless, even in their blogpost, they aren't being entirely honest, since 
they still claim that WD Reds are appropriate for RAIDs:

"In a RAID rebuild scenario using a typical Synology or QNAP (non-ZFS) 
platform, WD Red DMSMR drives perform as well as CMR drives or show slightly 
longer RAID rebuild times, depending on the condition of the drive and extent 
of rebuild required."
They double down on this further in the blogpost.

From some quick research, this is only true in a RAD1 array. In any other type 
of array, it is sometimes impossible to expand or rebuild the array with SMR 
drives.
See: https://raid.wiki.kernel.org/index.php/Timeout_Mismatch


Julien



Mar 3, 2022, 16:27 by markkne...@gmail.com:
On Thu, Mar 3, 2022 at 2:11 PM Julien Roy 
mailto:jul...@jroy.ca>> wrote:

The WD Reds are still marketted as RAID compatible to this day, despite the 
fact that they are SMRs.

Julien

18 months ago WD put out this statement:

https://blog.westerndigital.com/wd-red-nas-drives/

If by 'marketed' you mean Amazon or Newegg or some other seller is telling
people that the SMR is a good RAID solution I wouldn't say that's on WD
but rather the vendor.

Just my 2 cents,
Mark

RE: [gentoo-user] planning a new machine : comments welcome

[Laurence Perkins]

>
>
>-Original Message-
>From: Wols Lists  
>Sent: Friday, March 4, 2022 1:51 PM
>To: gentoo-user@lists.gentoo.org
>Subject: Re: [gentoo-user] planning a new machine : comments welcome
>
>On 04/03/2022 21:18, Laurence Perkins wrote:
>> I’d probably wait another year for all the bugs to work their way out 
>> of the system before trying it on purpose, but if you have 
>> non-critical systems to play with, well, the SMR drives are rather a 
>> lot cheaper…  I got a few for one of my server chassis because they 
>> can cram 5TB of storage into a 2.5” drive that fits in the bay for only $120.
>
>But are they?
>
>Okay, this is a 3.5", but I paid £180 I think for 8TB in a decent CMR package 
>(Toshiba N300). It was a backup drive, I was open to SMR, but didn't think 
>they were value for money.
>
>Cheers,
>Wol
>
>

3.5" gives a lot more platter area. Pi-r-squared and all that.  I couldn't find 
anything larger than 1TB CMR in a 2.5" package from anyone, and those were old 
stock.  Seagate doesn't make *any* CMR 2.5" anymore, not even the 500GB ones.
Even flash devices are going zoned it seems.  Simplifies the controller and 
lets you cram more storage onto the chips.

Unfortunately the chassis in question only takes 2.5" or smaller.  Otherwise I 
would definitely have preferred some 3.5", 12TB CMR surveillance-grade drives.

LMP

RE: [gentoo-user] Re: Root can't write to files owned by others?

[Laurence Perkins]

>On 09/03/2022 20:28, Dr Rainer Woitok wrote:
>> until recently my system behaves sort of strangely:
>> 
>> $ echo x | sudo tee /tmp/file
>> Password:
>> tee: /tmp/file: Permission denied
>> [...]
>> 
>> Since when can't root write to files  it doesn't own?   And not even, if
>> the file has write permission for everybody?
>
>This is normal, at least when using systemd. To disable this behavior, you 
>have to set:
>
>   sysctl fs.protected_regular=0
>
>But you should know what this means when it comes to security. See:
>
>   https://www.spinics.net/lists/fedora-devel/msg252452.html
>
>

And they chose to have systemd set that instead of putting it in sysctl.conf or 
the default kernel settings where it belongs?  Good grief!  

I guess if you're going to use systemd you need to subscribe to the Fedora 
mailing lists so you get at least a little notice before they break things.

LMP

RE: [gentoo-user] Re: Root can't write to files owned by others?

[Laurence Perkins]

>
>
>-Original Message-
>From: Dr Rainer Woitok  
>Sent: Thursday, March 10, 2022 9:51 AM
>To: gentoo-user@lists.gentoo.org; Nikos Chantziaras 
>Subject: [gentoo-user] Re: Root can't write to files owned by others?
>
>Nikos,
>
>On Thursday, 2022-03-10 12:21:36 +0200, you wrote:
>
>> ...
>> Are you sure that:
>> 
>> sysctl fs.protected_regular=0
>> 
>> does not help? I can reproduce it here on my system with kernel 
>> 5.15.27, and setting that sysctl to 0 fixes it immediately.
>
>No,  I'm not at all sure.   Since you mentioned  in your first mail that
>this is normal  when using  "systemd",  I did not pursue  this route any 
>further, because I'm using "openrc".
>
>I'll search the web for "fs.protected_regular"  to get a feeling for the 
>consequences and then perhaps set this when I'll again boot kernel vers- ion 
>5.15.26.
>
>Thanks for being persistent :-)
>
>Sincerely,
>  Rainer
>
>

Basically the idea is to keep other users from being able to trick root into 
writing sensitive data to something they control.
It's a "systemd thing" because, apparently, the systemd developers decided to 
have systemd enable it instead of leaving it in the bailiwick of the distros' 
configurations.
But if the default setting changed in a later kernel as well, that would 
potentially affect everyone, so a quick check of what it's set to wouldn't be 
amiss.

LMP

RE: [gentoo-user] Re: Root can't write to files owned by others?

[Laurence Perkins]

>-Original Message-
>From: Neil Bothwick  
>Sent: Friday, March 11, 2022 6:59 AM
>To: gentoo-user@lists.gentoo.org
>Subject: Re: [gentoo-user] Re: Root can't write to files owned by others?
>
>On Fri, 11 Mar 2022 12:38:48 +0100, Dr Rainer Woitok wrote:
>
>> No.   My "/tmp/" directory  is not mounted at all,  it is just a genuine
>> directory in "/".   And that root CAN overwrite a file it doesn't own in
>> other directories, is due to most directories  not having the sticky 
>> bit set  (which is a  (wanted) particularity  of "/tmp/" and 
>> "/var/tmp/", in that it prevents normal users from (re)moving other people's 
>> files):
>
>It's not the sticky bit per se from what I've read, but the new default 
>prevents root from overwriting a file if the file and the directory containing 
>it have different owners. In most cases, the file has the same directory as 
>the owner so this does not happen, but the sticky bit allows users that don't 
>own the directory to create files in it.
>
I was just looking at the patch.  In at least one of its modes I think it's 
specifically looking for the sticky bit on the directory.  I didn't think to 
pay attention to what the new default setting ends up doing for which specific 
mode it goes with.

LMP

RE: [gentoo-user] Any way to run multiple commands from single script in parallel?

[Laurence Perkins]

If you don't want to do thread management yourself in bash then you can use 
something like GNU Parallel (in the repo) to handle forking and collating 
processes for you.

Parallel in particular has the additional advantage that it's capable of 
shipping tasks off to other machines via SSH, so if you get to the point where 
you need a whole cluster to do your processing it's just a matter of adding a 
couple of arguments.

LMP

-Original Message-
From: Ramon Fischer  
Sent: Monday, March 14, 2022 3:37 AM
To: gentoo-user@lists.gentoo.org
Subject: Re: [gentoo-user] Any way to run multiple commands from single script 
in parallel?

Hello Joost,

I suppose, that you are talking about Bash scripts.

If so, you may put each individual command in a subshell by using an ampersand 
("&") at the end of the line.

This example[1] shows it nicely.

-Ramon

[1] 3. Parallelize running commands by grabbing PIDs.: 
https://will-keleher.com/posts/5-Useful-Bash-Patterns.html

On 14/03/2022 11:13, J. Roeleveld wrote:
> Hi,
>
> I often put multiple commands into a single file/script to be run in sequence.
> (each line can be executed individually, there is no dependency)
>
> Is there a tool/method to execute multiple lines/commands 
> simultaneously? Like having 3 or 4 run together and when 1 is 
> finished, it will grab the next one in the list?
>
> I would prefer this over simply splitting the file as the different 
> lines/ commands will not take the same amount of time.
>
> Thanks,
>
> Joost
>
>
>

--
GPG public key: 5983 98DA 5F4D A464 38FD CF87 155B E264 13E6 99BF

RE: [gentoo-user] KDE, sddm etc security. Plus LVM question.

[Laurence Perkins]

>>
>>
>>-Original Message-
>>From: Rich Freeman  
>>Sent: Sunday, March 20, 2022 11:03 AM
>>To: gentoo-user@lists.gentoo.org
>>Subject: Re: [gentoo-user] KDE, sddm etc security. Plus LVM question.
>>
>>On Sun, Mar 20, 2022 at 12:05 PM Daniel Frey  wrote:
>>>
>>> They don't even need to defeat a password. If they have root, it's 
>>> trivial to unlock a locked session without knowing the password - just FYI.
>>> ...
>>> The screen locks in linux are security by obscurity, if something is 
>>> that sensitive, don't stay logged in all the time.
>>
>>If somebody has root access to your box, then they are going to be able to 
>>get at your data.  They don't have to unlock your session to do it - they 
>>have access to the memory of all your processes, everything on disk, and so 
>>on.  If you're using encryption at the account level and it is 
>>well-implemented then root probably can't get at your data while you aren't 
>>logged in, but they certainly can get it the next time you log in.
>>
>>It is true though that linux screensavers are often not well-implemented.  
>>Honestly, I'm not sure if any of them are - it seems to be more of an 
>>afterthought in the design layered on top.  I haven't made a study of them, 
>>so maybe there are some which are, but something like this really needs to be 
>>designed into the system to be secure, and some of that needs to be treated 
>>as security-critical code.
>>
>>Now, if you want to make an argument for leaving systems powered down except 
>>when needed if they contain sensitive data that would certainly reduce the 
>>opportunity for intrusion, but you still need the OS to keep people from 
>>gaining root in the first place.
>>
>>As others have mentioned at the start of the thread, if you're concerned with 
>>physical security then full disk encryption (or at least encryption of data 
>>combined with airtight authentication of the
>>OS) has to be part of the solution.  In 99% of linux-based solutions that 
>>requires entering a password at boot.  In theory the linux kernel has support 
>>for TPM verified boot, so you could implement something like Bitlocker/etc on 
>>Linux, but I'm not aware of any distros that have done so (unless you want to 
>>count something like ChromeOS).  For a desktop system a boot password isn't 
>>as much of a problem, but if you want an unattended server to be able to boot 
>>on power restoration then a TPM-based solution would be better.  It certainly 
>>is prettier on the desktop, and allows for more recovery options, which is 
>>why just about all corporate laptops I've seen do it this way.  Of course 
>>without a boot password you're only as secure as your OS, as any attacker can 
>>still boot the OS and attack it while it is running, which they can't do if 
>>the disk requires a password to decrypt it.
>>
>>If you're running Windows on a system with a TPM the simplest solution to all 
>>this stuff is to turn on Bitlocker, though this is not available on the Home 
>>edition of Win10.
>>
>>--
>>Rich
>>
>>

There was the ORWL project a few years ago.  Self-encrypting SSD drive with a 
TPM that would unlock it only in the presence of an encrypted RFID tag plus 
tapping in a code on the keypad, with all the sensitive bits wrapped in an 
active mesh system that would destroy the data if it detected any tampering.

Plus it could be set to lock down if the accelerometer went off, or after 
inactivity, which would disable whatever set of external ports was set.

Don't seem to be made any more, which is a pity.  But the design was getting to 
be a bit dated.  Schematics are theoretically still available somewhere if you 
want to build one.

LMP

RE: [gentoo-user] KDE, sddm etc security. Plus LVM question.

[Laurence Perkins]

>-Original Message-
>From: Rich Freeman  
>Sent: Monday, March 21, 2022 11:07 AM
>To: gentoo-user@lists.gentoo.org
>Subject: Re: [gentoo-user] KDE, sddm etc security. Plus LVM question.
>
>On Mon, Mar 21, 2022 at 12:17 PM Laurence Perkins  wrote:
>>
>> There was the ORWL project a few years ago.  Self-encrypting SSD drive with 
>> a TPM that would unlock it only in the presence of an encrypted RFID tag 
>> plus tapping in a code on the keypad, with all the sensitive bits wrapped in 
>> an active mesh system that would destroy the data if it detected any 
>> tampering.
>
>While I can see this being useful if for some reason you don't have support 
>for encryption on the software side, something like this seems like it 
>wouldn't actually solve the unattended boot problem, since you have to enter a 
>PIN.  If you don't require the PIN and leave the RFID tag sitting next to the 
>drive all the time, then anybody can walk in and take the drive and the tag 
>and then read the data off the drive bypassing the OS.  So it offers at best 
>the same protection as a LUKS passphrase entered at boot, and at worst no 
>protection at all.  It would have the advantage that you wouldn't be able to 
>attack the passphrase itself as no doubt the PIN only offers limited attempts 
>and would be very difficult to bypass.
>
>The advantage of the TPM in the computer is that you can do unattended 
>verified boot, so the disk can only be decrypted if the OS boots normally 
>without tampering.  Obviously you're still open to OS vulnerabilities, but the 
>drive itself cannot be accessed except via the OS.  The TPM chip can actually 
>supervise the boot process.
>
>Still an interesting product though.  I could see it being useful if you had 
>to run some specific OS that doesn't support disk encryption natively.
>
>--
>Rich
>
>

As I recall there were several security modes related to the battery-backed 
arm-based TPM, (which also had source code available, so it could probably do 
anything you wanted.)

Having it remain unlocked and capable of rebooting unless the accelerometer 
showed movement I think was an option since the TPM kept monitoring even if the 
mains power was interrupted.  

And there's still the standard BIOS-level passwords and secureboot stuff to 
keep someone from switching the OS.  As long as you use your own keys and 
remove the Microsoft one anyway.

Any attempt to open the case would disrupt the active mesh and cause it to dump 
the keys, so that takes care of attacking the drive itself or the system memory.

Main problem was that it was just too expensive.  Could put a standard computer 
in a safe for far less, and the group of people who need paranoid security on 
the move just wasn't enough to support ongoing updates to the design.

Could probably do something similar these days with one of those $3 bluepill 
boards and one of those new 3d printers capable of embedding metal though.

LMP

RE: [gentoo-user] KDE, sddm etc security. Plus LVM question.

[Laurence Perkins]

>-Original Message-
>From: Rich Freeman  
>Sent: Monday, March 21, 2022 1:08 PM
>To: gentoo-user@lists.gentoo.org
>Subject: Re: [gentoo-user] KDE, sddm etc security. Plus LVM question.
>
>On Mon, Mar 21, 2022 at 2:30 PM Laurence Perkins  wrote:
>>
>> Having it remain unlocked and capable of rebooting unless the accelerometer 
>> showed movement I think was an option since the TPM kept monitoring even if 
>> the mains power was interrupted.
>>
>
>Yeah, there might still be ways to accomplish it with features like this.
>
>>
>> Could probably do something similar these days with one of those $3 bluepill 
>> boards and one of those new 3d printers capable of embedding metal though.
>
>Or you could just use the TPM that is probably already in your computer...  :)
>
>-- 
>Rich
>
>
The TPM in most computers doesn't dump the keys if someone tries to open the 
case to install hardware sniffers.  

Not that it isn't sufficient for the average person's security needs, but it 
definitely still has a lot of attack vectors.

A case with active mesh and a customizable TPM module with extra sensors, 
hardware overrides, and that could take standard-sized desktop components might 
be useful for small business applications where they have sensitive data but 
can't afford a whole secured data center.

LMP

RE: [gentoo-user] KDE, sddm etc security. Plus LVM question.

[Laurence Perkins]

> -Original Message-
> From: Rich Freeman  
> Sent: Monday, March 21, 2022 5:21 PM
> To: gentoo-user@lists.gentoo.org
> Subject: Re: [gentoo-user] KDE, sddm etc security. Plus LVM question.
> 
> On Mon, Mar 21, 2022 at 8:03 PM Laurence Perkins  wrote:
> >
> > The TPM in most computers doesn't dump the keys if someone tries to open 
> > the case to install hardware sniffers.
> >
> 
> That's a good point, though if somebody with the ability to sniff the RAM or 
> (to a lesser degree) GPU traffic is after you, then you probably want to be 
> on the lookout for rubber hose decryption.
> 
> If you're a big spender the AMD Secure Memory Encryption feature would 
> probably help there, assuming they ever get it working on Linux.
> 
> --
> Rich
> 
> 

There are a lot of circumstances where the attacker doesn't want you to know 
you've been compromised.  At least not right away.  Tamper detection can be 
useful for avoiding that.

Along similar lines there's a kernel module available, the project name slips 
my mind, which can be set to wipe keys, memory, etc. if the system is booted 
without a particular USB device attached, or if the device is removed.  Gives 
one a way to quickly "decommission" a system.

LMP

RE: [gentoo-user] How to run X11 apps remotely?

[Laurence Perkins]

>-Original Message-
>From: Grant Edwards  
>Sent: Tuesday, March 22, 2022 9:42 AM
>To: gentoo-user@lists.gentoo.org
>Subject: [gentoo-user] How to run X11 apps remotely?
>
>CAUTION: This is an EXTERNAL email. Do not click links or open attachments 
>unless you recognize the sender and know the content is safe.
>
>How does one run "modern" X11 apps remotely?
>
>Using ssh -X or ssh -Y works fine for older applications, but not for things 
>that use "modern" toolkits. Modern tookit designers appear to have adopted a 
>life mission to maximize the number of client-server round-trips required for 
>even a trivial event like a keystroke in a text box.
>
>As a result, even with a 5-10Mbps remote connection, it takes several minutes 
>to enter a string of even a few characters. A mouseclick on a button can take 
>a minute or two to get processed. Resizing a window pretty much means it's 
>time for a cuppa.
>
>Opening chrome and loading a web page can take 10-15 minutes. No activity at 
>all on the screen, but the network connection to the remote machine is 
>saturated at 5Mbps for minutes at a time. WTF?
>
>Something like LibreOffice is completely unusable.
>
>Even something "lightweight" like atril is so slow it's barely usable.
>
>I do not want a "remote desktop". I just want to run a single application on a 
>remote machine and have its window show up locally.
>
>Back in the day, I used to run X11 apps remotely through dial-up connections, 
>and most of them were a little sluggish but still actually usable...
>
>X11 transparent network support was its killer feature, but for all practical 
>purpopses, that feature seems to have been killed.
>
>--
>Grant
>

As you mentioned, it's a lot of extra round-trips.  Which means that it's not 
primarily your bandwidth that's the limiting factor, it's the latency.

Unfortunately, the speed of light being what it is, there are practical limits 
to what you can do about latency depending on how far apart the systems in 
question are.

But, check for and mitigate any bufferbloat issues you may have, that will 
spike your latency quite a bit.

The key back in the day was that people used X11 primitives directly.  But the 
X11 primitives are ugly, and there weren't any tools for making them pretty.  
So rather than add those mechanisms all the toolkit authors just did their own 
thing and now everything is just bitmaps and practically no processing can be 
done locally.

Some programs like gVim will detect that they're running over SSH and fall back 
to basic X11 for the speed factor.  Not sure what browsers might do that.

LMP

RE: [gentoo-user] Sound not sounding

[Laurence Perkins]

> -Original Message-
> From: Mark Knecht  
> Sent: Wednesday, March 23, 2022 3:14 PM
> To: Gentoo User 
> Subject: Re: [gentoo-user] Sound not sounding
> 
> On Wed, Mar 23, 2022 at 3:05 PM Wol  wrote:
> >
> > On 23/03/2022 20:31, Mark Knecht wrote:
> 
>  Yeah, ain't that the truth! I'd likely be a Gentoo user if anyone 
> built and maintained a 'standard', stable distro, something delivered in 
> primarily binary files. I just lost patience dealing with all the issues you 
> guys deal with daily.
> 

I believe that's called Sabayon.  Standard, provided configurations and 
supported apps with a binary package server just like it was a binary distro, 
but if you really want to customize something it's still Gentoo under the fancy 
GUI.

I've not actually used the distro itself, but their overlay has been quite 
handy in the past.

ChromeOS is supposedly similar, at least it as at one point, but I don't trust 
them any farther than I can throw them, so I didn't investigate much and just 
ripped all their stuff out of my Chromebook and replaced it with real Gentoo.

LMP

RE: [gentoo-user] LVM and moving things around

[Laurence Perkins]

> -Original Message-
> From: Dale  
> Sent: Monday, April 4, 2022 5:42 AM
> To: gentoo-user@lists.gentoo.org
> Subject: Re: [gentoo-user] LVM and moving things around
> 
> Bill Kenworthy wrote:
> > Rsync has a bwlimit argument which helps here. Note that rsync copies 
> > the whole file on what it considers local storage (which can be 
> > mounted network shares) ... this can cause a real slowdown.
> > BillK
> >
> >
> 
> I ended up just letting it do its thing.  I didn't want to slow it down by 
> much, just make my desktop able to respond better.  I used nice and ionice to 
> do this with emerge and it works great.  I just thought I was missing some 
> option for that command that google didn't help with.  I went and helped my 
> sis-n-law with some garden stuff.  That helped.  ;-)
> 
> As it stands now, I've copied enough over to get a free 8TB drive.  I set up 
> LUKS, which includes LVM, on the drive and am copying some more files onto 
> the newly encrypted drive.  Once everything is transferred, I'll then see if 
> I need the other drive added or not.  I may not at the moment.  Of course, 
> once fiber internet gets here, that may change pretty soon. 
> 
> If someone is really knowledgeable about LVM and LUKS and how to set up a 
> encrypted hard drive, not a whole install but just a data drive, a howto for 
> this would be really nice.  I had to use a LUKS howto and a LVM howto and 
> sort of merge commands until I figured out how to get the two together.  Even 
> tho I got it working, I'm still not real clear on how one part of it works.  
> I'm just not clear enough on it to write one myself.  A Gentoo wiki would be 
> nice.  There's one for the two separately but not together.  One posted 
> anywhere google can find it would be great tho.
> 
> Now to find something to do while rsync copies over some 6TBs of files. O_O
> 
> Dale
> 
> :-)  :-) 
> 
A little late to the party, but the other setting to look at when you're doing 
this kind of thing is "sysctl vm.dirty_ratio".
Basically it's what percentage of the disk cache can be dirty before the system 
forces all IO operations to be synchronous.

Setting it higher lets the system keep more data up-in-the-air while you're 
shuffling lots of stuff around.  Of course, it also risks losing more if the 
system crashes in the middle of it all, so use it judiciously.

Setting dirty_ratio dirty_background_ratio, and dirty_writeback_centisecs 
appropriately when doing things with large amounts of data can significantly 
improve system responsiveness and, with rotational drives, throughput.

LMP

RE: [gentoo-user] LVM and moving things around

[Laurence Perkins]

>-Original Message-
>From: Dale  
>Sent: Monday, April 4, 2022 4:37 PM
>To: gentoo-user@lists.gentoo.org
>Subject: Re: [gentoo-user] LVM and moving things around
>
>
>One thing that annoys me, it trying to use swap.  I don't want to disable it 
>because on occasion Firefox goes nuts and starting hogging memory really bad.  
>I have swappiness set to like 5 or something which means it shouldn't use it 
>but when using rsync, it creeps some in.  When it does, that results in some 
>slowness.  I have a little script thing that clears all that but still, I may 
>set it to 3 or maybe 2 for a bit. Me ponders the thought. 
>
>I'm making progress.  Feel sorry for those hard drives tho. ;-)
>
>Dale
>
>:-)  :-) 
>
>

I'm told that you can use cgroups for dealing with that kind of thing such 
that, for example, only Firefox is allowed to be swapped.  I haven't had time 
to dig into it, but it seems like a useful tool.

Also, the compressed swap and zram swap devices with backing stores offer a 
fairly significant boost to the speed of swap so long as the data being swapped 
is compressible.

LMP

RE: [SUSPECTED SPAM] Re: [gentoo-user] Choose a wireless access point

[Laurence Perkins]

> -Original Message-
> From: William Kenworthy  
> Sent: Monday, April 4, 2022 8:05 PM
> To: gentoo-user@lists.gentoo.org
> Subject: [SUSPECTED SPAM] Re: [gentoo-user] Choose a wireless access point
> 
> 
> On 4/4/22 23:12, Jack wrote:
> > On 4/4/22 01:31, William Kenworthy wrote:
> >> Is there a way force openrc and wpa_supplicant to map a particular 
> >> access point to an interface or fail?
> >>
> >> I have two AP's (each on a different ssid) to connect to so have two 
> >> wifi interfaces - unfortunately they are not equal so I want wlan0 to 
> >> connect to only one particular AP, and wlan1 to the other ... reliably!
> >> I can manually force it to connect but invariably at the first glitch 
> >> they both end up connected to the same AP (usually the strongest 
> >> which is often not what I want :(
> >>
> >> BillK
> >
> > I don't know about wpa-supplicant, but I'm using open-rc and KDE, and 
> > KDE's systemsettings Network / Connections screen lets you restrict a 
> > network connection so a specific device.  Not sure if this helps you 
> > any, but it would indicate that what you want is possible.
> >
> > Jack
> >
> Hi Jack, unfortunately its a headless, wifi only system which is why getting 
> openrc to behave is important!
> 
> BillK

The bit where specifying the SSID in conf.d/net doesn't work sounds like a bug 
to me, but one that may take a while to be fixed since I'm not sure how many 
people use netifrc with wireless.

If you're open to experimenting, NetworkManager will let you specify that 
connections may only be used with specific adapters.  While normally considered 
a GUI tool it does have nmcli and nmtui for configuring it on headless systems.

LMP

RE: [gentoo-user] LVM and moving things around

[Laurence Perkins]

> -Original Message-
> From: Rich Freeman  
> Sent: Tuesday, April 5, 2022 11:59 AM
> To: gentoo-user@lists.gentoo.org
> Subject: Re: [gentoo-user] LVM and moving things around
> 
> On Tue, Apr 5, 2022 at 11:10 AM Wols Lists  wrote:
> >
> > I don't know how you take advantage of it, but linux by default caches 
> > disk i/o. You can tell it to "don't cache" and apparently it makes a 
> > major difference. Given that rsync reads once and then never uses it 
> > again, you don't want it cached.
> >
> 
> I suggest reading:
> man  posix_fadvise
> https://insights.oetiker.ch/linux/fadvise/
> http://rdiez.shoutwiki.com/wiki/The_Linux_Filesystem_Cache_is_Braindead
> https://lwn.net/Articles/806980/
> https://bugzilla.samba.org/show_bug.cgi?id=9560
> 
> There might be something more recent, but my overall impression is that this 
> problem is less solved than it probably ought to be.
> 
> --
> Rich
> 
I remember seeing something more with regard to "please cache/please 
discard/this can be immediately swapped because I won't need it for a while" 
stuff for like the 5.15 kernel or something, but the actual system programs 
will have to be updated to use it before it'll make any difference.

LMP

RE: [gentoo-user] Hard drive error from SMART

[Laurence Perkins]

> -Original Message-
> From: Dale  
> Sent: Monday, April 11, 2022 6:28 PM
> To: gentoo-user@lists.gentoo.org
> Subject: [gentoo-user] Hard drive error from SMART
> 
> Given the low number and it showing it corrected that error, and then passed 
> a short and long test, is this drive "safe enough" to keep in service?  I 
> have backups just in case but just curious what others know from experience.  
> At least this isn't one of those nasty messages that the drive will die 
> within 24 hours.  I got one of those ages ago and it didn't miss it by much.  
> A little over 30 hours or so later, it was a door stop.  It would spin but it 
> couldn't even be seen by the BIOS.
> Maybe drives are getting better and SMART is getting better as well.
> 
> Thoughts.  Replace as soon as drive arrives or wait and see?
> 
> Dale
> 
> :-)  :-)
> 
When it's just one or two errors like that and they don't keep going up I tend 
to treat it as an isolated incident, but the drive still goes into the pool I 
use with RAID just in case.

Preferably a setup where you can lose more than one disk without losing the 
data.

Note that, depending on where the bad sector is, when it gets remapped the 
extra seek necessary to read that logical address could slow the drive down 
substantially.  Make sure your filesystem's root inode or something doesn't end 
up on top of it.

Sometimes I miss the old drives where all this was handled by the OS and so you 
knew exactly what sector was bad and your filesystem could be told to just not 
use it.  Made scanning for bad sectors more annoying, but deciding how bad the 
drive was rather easier.

LMP

RE: [gentoo-user] Hard drive error from SMART

[Laurence Perkins]

> -Original Message-
> From: Dale  
> Sent: Tuesday, April 12, 2022 10:08 AM
> To: gentoo-user@lists.gentoo.org
> Subject: Re: [gentoo-user] Hard drive error from SMART
> 
> Rich Freeman wrote:
> > On Mon, Apr 11, 2022 at 9:27 PM Dale  wrote:
> >> Thoughts.  Replace as soon as drive arrives or wait and see?
> >>
> > So, first of all just about all my hard drives are in a RAID at this 
> > point, so I have a higher tolerance for issues.
> >
> > If a drive is under warranty I'll usually try to see if they will RMA 
> > it.  More often than not they will, and in that case there is really 
> > no reason not to.  I'll do advance shipping and replace the drive 
> > before sending the old one back so that I mostly have redundancy the 
> > whole time.
> >
> > If it isn't under warranty then I'll scrub it and see what happens.
> > I'll of course do SMART self-tests, but usually an error like this 
> > won't actually clear until you overwrite the offline sector so that 
> > the drive can reallocate it.  A RAID scrub/resilver/etc will overwrite 
> > the sector with the correct contents which will allow this to happen.
> > (Otherwise there is no way for the drive to recover - if it knew what 
> > was stored there it wouldn't have an error in the first place.)
> >
> > If an error comes back then I'll replace the drive.  My drives are 
> > pretty large at this point so I don't like keeping unreliable drives 
> > around.  It just increases the risk of double failures, given that a 
> > large hard drive can take more than a day to replace.  Write speeds 
> > just don't keep pace with capacities.  I do have offline backups but I 
> > shudder at the thought of how long one of those would take to restore.
> >
> 
> 
> Sadly, I don't have RAID here but to be honest, I really need to have it 
> given the data and my recent luck with hard drives.  Drives used to get 
> dumped because they were just to small to use anymore.  Nowadays, they seem 
> to break in some fashion long before their usefulness ends their lives. 
> 
> I remounted the drives and did a backup.  For anyone running up on this, just 
> in case one of the files got corrupted, I used a little trick to see if I can 
> figure out which one may be bad if any.  I took my rsync commands from my 
> little script and ran them one at a time with --dry-run added.  If a file was 
> to be updated on the backup that I hadn't changed or added, I was going to 
> check into it before updating my backups.  It could be that the backup file 
> was still good and the file on my drive reporting problems was bad.  In that 
> case, I would determine which was good and either restore it from backups or 
> allow it to be updated if needed.  Either way, I should have a good file 
> since the drive claims to have fixed the problem.  Now let us pray.  :-D 
> 
> Drive isn't under warranty.  I may have to start buying new drives from 
> dealers.  Sometimes I find drives that are pulled from systems and have very 
> few hours on them.  Still, warranty may not last long.  Saves a lot of money 
> tho. 
> 
> USPS claims drive is on the way.  Left a distribution point and should update 
> again when it gets close.  First said Saturday, then said Friday.  I think 
> Friday is about right but if the wind blows right, maybe Thursday. 
> 
> I hope I have another port and power cable plug for the swap out.  At least 
> now, I can unmount it and swap without a lot of rebooting.  Since it's on 
> LVM, that part is easy.  Regretfully I have experience on that process.  :/
> 
> Thanks to all. 
> 
> Dale
> 
> :-)  :-) 
> 
> 
You can get up to 16X SATA PCI-e cards these days for pretty cheap.  So as long 
as you have the power to run another drive or two there's not much reason not 
to do RAID on the important stuff.  Also, the SATA protocol allows for port 
expanders, which are also pretty cheap.  

One of my favorite things about BTRFS is the data checksums.  If the drive 
returns garbage, it turns into a read error.  Also, if you can't do real RAID, 
but have excess space you can tell it to keep two copies of everything.  
Doesn't help with total drive failure, but does protect against the occasional 
failed sector.  If you don't mind writes taking twice as long anyway.

LMP

RE: [gentoo-user] Hard drive error from SMART

[Laurence Perkins]

> -Original Message-
> From: Frank Steinmetzger  
> Sent: Tuesday, April 12, 2022 10:39 AM
> To: gentoo-user@lists.gentoo.org
> Subject: Re: [gentoo-user] Hard drive error from SMART
> 
> 
> I actually developed a tool for that. It creates and checks md5 checksums 
> recursively and *per directory*. Whenever I copy stuff from somewhere, like a 
> music album, I do an immediate md5 run on that directory. And when I later 
> copy that stuff around, I simply run the tool again on the copy (after the FS 
> cache was flushed, for example by unmounting and remounting) to see whether 
> the checksums are still valid.
> 
> You can find it on github: https://github.com/felf/dh It’s a single-file 
> python application, because I couldn’t be bothered with the myriad ways of 
> creating a python package. ;-)
> 
> --
> Grüße | Greetings | Salut | Qapla’
> Please do not share anything from, with or about me on any social network.
> 
> A horse comes into a bar.
> Barkeep: “Hey!”
> Horse:   “Sure.”
> 
There's also app-crypt/md5deep

Does a number of hashes, is threaded, has options for piecewise hashing and a 
matching mode for using the hashes to find duplicates.  Also a number of input 
and output filters for those cases where you don't want to hash everything.

Also can output a number of formats, but reformatting is generally trivial.

LMP

RE: [gentoo-user] Hard drive error from SMART

[Laurence Perkins]

>-Original Message-
>From: Dale  
>Sent: Tuesday, April 12, 2022 11:22 AM
>To: gentoo-user@lists.gentoo.org
>Subject: Re: [gentoo-user] Hard drive error from SMART
>
>Laurence Perkins wrote:
>>> -Original Message-
>>> From: Dale 
>>> Sent: Tuesday, April 12, 2022 10:08 AM
>>> To: gentoo-user@lists.gentoo.org
>>> Subject: Re: [gentoo-user] Hard drive error from SMART
>>>
>>> Rich Freeman wrote:
>>>> On Mon, Apr 11, 2022 at 9:27 PM Dale  wrote:
>>>>> Thoughts.  Replace as soon as drive arrives or wait and see?
>>>>>
>>>> So, first of all just about all my hard drives are in a RAID at this 
>>>> point, so I have a higher tolerance for issues.
>>>>
>>>> If a drive is under warranty I'll usually try to see if they will 
>>>> RMA it.  More often than not they will, and in that case there is 
>>>> really no reason not to.  I'll do advance shipping and replace the 
>>>> drive before sending the old one back so that I mostly have 
>>>> redundancy the whole time.
>>>>
>>>> If it isn't under warranty then I'll scrub it and see what happens.
>>>> I'll of course do SMART self-tests, but usually an error like this 
>>>> won't actually clear until you overwrite the offline sector so that 
>>>> the drive can reallocate it.  A RAID scrub/resilver/etc will 
>>>> overwrite the sector with the correct contents which will allow this to 
>>>> happen.
>>>> (Otherwise there is no way for the drive to recover - if it knew 
>>>> what was stored there it wouldn't have an error in the first place.)
>>>>
>>>> If an error comes back then I'll replace the drive.  My drives are 
>>>> pretty large at this point so I don't like keeping unreliable drives 
>>>> around.  It just increases the risk of double failures, given that a 
>>>> large hard drive can take more than a day to replace.  Write speeds 
>>>> just don't keep pace with capacities.  I do have offline backups but 
>>>> I shudder at the thought of how long one of those would take to restore.
>>>>
>>>
>>> Sadly, I don't have RAID here but to be honest, I really need to have it 
>>> given the data and my recent luck with hard drives.  Drives used to get 
>>> dumped because they were just to small to use anymore.  Nowadays, they seem 
>>> to break in some fashion long before their usefulness ends their lives. 
>>>
>>> I remounted the drives and did a backup.  For anyone running up on 
>>> this, just in case one of the files got corrupted, I used a little 
>>> trick to see if I can figure out which one may be bad if any.  I took 
>>> my rsync commands from my little script and ran them one at a time 
>>> with --dry-run added.  If a file was to be updated on the backup that 
>>> I hadn't changed or added, I was going to check into it before 
>>> updating my backups.  It could be that the backup file was still good 
>>> and the file on my drive reporting problems was bad.  In that case, I 
>>> would determine which was good and either restore it from backups or 
>>> allow it to be updated if needed.  Either way, I should have a good 
>>> file since the drive claims to have fixed the problem.  Now let us 
>>> pray.  :-D
>>>
>>> Drive isn't under warranty.  I may have to start buying new drives from 
>>> dealers.  Sometimes I find drives that are pulled from systems and have 
>>> very few hours on them.  Still, warranty may not last long.  Saves a lot of 
>>> money tho. 
>>>
>>> USPS claims drive is on the way.  Left a distribution point and should 
>>> update again when it gets close.  First said Saturday, then said Friday.  I 
>>> think Friday is about right but if the wind blows right, maybe Thursday. 
>>>
>>> I hope I have another port and power cable plug for the swap out.  At 
>>> least now, I can unmount it and swap without a lot of rebooting.  
>>> Since it's on LVM, that part is easy.  Regretfully I have experience 
>>> on that process.  :/
>>>
>>> Thanks to all. 
>>>
>>> Dale
>>>
>>> :-)  :-)
>>>
>>>
>> You can get up to 16X SATA PCI-e cards these days for pretty cheap.  So as 
>> long as you have the power to run another drive or two there's not much 
>> reason not to do RAID on the important s

RE: [gentoo-user] Hard drive error from SMART

[Laurence Perkins]

>-Original Message-
>From: Wol  
>Sent: Tuesday, April 12, 2022 2:51 PM
>To: gentoo-user@lists.gentoo.org
>Subject: Re: [gentoo-user] Hard drive error from SMART
>
>On 12/04/2022 20:41, Laurence Perkins wrote:
>> LVM is good for being able to swap out drives easily but with the modern, 
>> huge drives you really want data checksums if you can get them.  Otherwise 
>> all it takes is a flipped bit somewhere to wreck your data and drive 
>> firmware doesn't always notice.  I think you can do that with LVM, but I've 
>> never looked into it for certain.
>
>Look at that link for my system that I posted. I use dm-integrity, so a 
>flipped bit will trigger a failure at the raid-5 level and recover.
>
>For those people looking at btrfs - note that parity-raid (5 or 6) is not a 
>wise idea at the moment so you don't get two-failure protection ...

Specifically if the system crashes or has a power failure there may be some 
data left hanging until it can complete a scrub.  Disk failures during that 
period may lose some of said data.

How much of a risk that is depends on the stability of your power and kernel 
and how much data turnover you have.  I only use it on systems with UPS power 
and additional backups.  Needs careful monitoring of the drives too since 
system crashes due to drive failures can leave you in rather a sticky mess.

>
>Cheers,
>Wol
>
>

RE: [gentoo-user] doing gentoo install disrupted

[Laurence Perkins]

> -Original Message-
> From: Dale  
> Sent: Thursday, April 14, 2022 8:29 AM
> To: gentoo-user@lists.gentoo.org
> Subject: Re: [gentoo-user] doing gentoo install disrupted
> 
> Jude DaShiell wrote:
> > I got down to emerge-webrsync and discovered an error in 
> > portage/make.conf on line 21 non-existent server then emerge-webrsync 
> > tried to get a time stamp and then speakup died.
> > I have a single ssd slot in this machine and may be able to attach the 
> > gentoo ssd to the computer by usb so I can recover contents of my 
> > make.conf file and find out what happened in there.  It will be a long 
> > way back to chroot before I can try this again and that's not 
> > happening today due to space weather.  The computer will be off until 
> > tomorrow and disconnected.
> >
> >
> >
> 
> 
> You can pick up where you left off at pretty much any point in a Gentoo 
> install.  If you have to shutdown or unmount your install, just follow the 
> mounting and chroot sections again.  Once you have that done, pick up where 
> you left off.  In the past, I've had to do that before and it does fine as 
> long as you remember where you left off. 
> 
> Hope that helps.
> 
> Dale
> 
> :-)  :-) 
> 
Even if you don't remember where you left off, just check if each step has 
already been done before you do it.

And unpacking the stage3 I think is the only one likely to actually hurt 
anything if you do it again.  The others would just waste a little time is all.

LMP

RE: [gentoo-user] Reinstall

[Laurence Perkins]

And sometimes if you use --binpkg-respect-use=n and/or --with-bdeps=n you can 
jostle it into using more of the binaries on both passes.

Additionally, you can use the ebuild command directly to force it to just 
install things without checking all the dependencies, that's sometimes handy 
for breaking cycles too.

Do pay careful attention to the merge order though.  Make sure any updates to 
glibc happen first or else you'll wedge your system pretty badly.  Having 
static-compiled busybox installed as a backup is often a good idea.

Alternatively, fully update the system before putting in your world file, and 
then instead of copying in the world file all at once just run a loop to emerge 
the lines in it one at a time.

LMP

-Original Message-
From: David Palao  
Sent: Wednesday, May 11, 2022 5:26 AM
To: gentoo-user@lists.gentoo.org
Subject: Re: [gentoo-user] Reinstall

Hi,

What I would suggest is to try yo emerge @world first with a reduced list of 
USE flags, maybe the default, and after success you could introduce back the 
wanted USE flags and emerge @world once more.

It could be a bit too much compilation, but if you have already binary 
packages, it will not be so expensive the second round, IMHO.

Best

On 11/5/22 11:57, Francisco Ares wrote:
> Hello
>
> After a main HD failure, I'll have to reinstall Gentoo from almost 
> zero - I have a full and recent copy of the /etc directory and the 
> file /var/lib/portage/world in a secondary HD (along many personal 
> backups).
>
> Installation basics done, now it is time for an emerge world.
>
> Although the emerge lists is as huge as expected, it doesn't even 
> start, portage says there are cyclic USE flags that I should avoid at 
> the first moment, but may restore afterwards.
>
> But it doesn't say which are those USE flags that block each other.
>
> Is there any way to find those better than brute force?
>
> By the way, I also have a copy of all binary packages (I always use 
> the -b flag while emerging any package) in that second disk. But that 
> didn't help so far, even trying to use the -K flag. I thought on 
> un-tar'ing those binary packages by hand, but portage will be unaware 
> of this, not knowing the packages are installed.
>
> Any hint will be greatly appreciated!
>
> Thanks
>
> Francisco

RE: [gentoo-user] Reinstall

[Laurence Perkins]

> -Original Message-
> From: Neil Bothwick  
> Sent: Wednesday, May 11, 2022 11:35 AM
> To: gentoo-user@lists.gentoo.org
> Subject: Re: [gentoo-user] Reinstall
> 
> On Wed, 11 May 2022 16:45:31 +0000, Laurence Perkins wrote:
> 
> > Alternatively, fully update the system before putting in your world 
> > file, and then instead of copying in the world file all at once just 
> > run a loop to emerge the lines in it one at a time.
> 
> Now you mention it, that's what I did last time, although my loop emerge ten 
> lines at a time to cut down on the number of dependency recalculations.
> 
Well, and have it check the exit status and spit any lines that fail into 
another file to try again later.  Then shuffle the new file so the bad ones 
move around, or do the second pass one at a time.

LMP

RE: [gentoo-user] chromium shortcuts?

[Laurence Perkins]

Compile time for Chromium currently seems to run about four hours on an i7-9700.

-Original Message-
From: Jorge Almeida  
Sent: Sunday, May 29, 2022 2:26 PM
To: gentoo-user@lists.gentoo.org
Subject: Re: [gentoo-user] chromium shortcuts?

On Sun, May 29, 2022 at 10:16 PM w...@op.pl  wrote:
>
> If non-free software is not of any concern for you, then maybe try 
> Vivaldi browser? It is based on the same engine and feels quite like 
> chromium but with many many more options (USE=widevine for netflix ;) 
> )
>
I have no concerns regarding licensing et al., but I once tried Vivaldi and was 
not impressed (I don't remember why). I'm into chrom* only because it provides 
a smooth experience re youtube & gmail. I don't think other browsers can do it, 
but I wouldn't mind being wrong.

BTW: chromium also has a "widevine" flag. I don't know how much time does it 
take to compile chromium nowadays, but I don't have netflix anyway :(

RE: [gentoo-user] *sob*

[Laurence Perkins]

>-Original Message-
>From: Neil Bothwick  
>Sent: Sunday, June 12, 2022 4:28 AM
>To: gentoo-user@lists.gentoo.org
>Subject: Re: [gentoo-user] *sob*
>
>On Sun, 12 Jun 2022 00:42:56 -0400, Alan Grimes wrote:
>
>> All of the followning statemens are true:
>> 
>> 1. I am never not running seamonkey, it is always open at all times on 
>> my left hand monitor, If at any point it was not on my left hand 
>> monitor either I must have been going through a power failure or 
>> something had broken it, because it is working at the moment I have 
>> decided not to use emerge update for the next six months, two months 
>> in to that period at this point.
>> 
>> 2. Seamonkey is my default browser.
>> 
>> 3. Seamonkey cannot be launched twice, ie it cannot be launched on my 
>> right hand monitor because the monitors are using separte X servers 
>> and it would need to be launched twice to come up on that monitor.
>
>The last statement is not quite true. It cannot be launched twice with the 
>same profile. This was the first hit when googling "seamonkey launch two 
>instances"
>
>http://kb.mozillazine.org/Opening_a_new_instance_of_your_Mozilla_application_with_another_profile
>
>Is there a particular reason for running two X servers rather than one desktop 
>on two screens? The latter completely avoids the problem you are having.
>
>
>--
>Neil Bothwick
>
>If God can't help you, how about Mr. Coffee?

I can think of a few ways around this mess actually...

1.  Use one X server for both screens, and should you happen to need a separate 
X server to contain something use xnest, xephyr, or a headless VNC server to 
run another X that you can have there as a window.
2.  Since you're using two X instances, and therefore two instances of your 
window manager, etc., create two profiles for all that which use separate 
seamonkey profile folders for their default browser.
3.  Go all the way and run your two X servers as two completely different user 
accounts.  I presume you're already using separate keyboard and mouse for each 
anyway since I'm not sure how it would work otherwise.
4.  Update your default browser string to explicitly set which X display to 
talk to.  Then it will open on your left monitor no matter where it's launched 
from.
5.  You can't really get away from having a default browser, and you probably 
don't want to since it'll break other things later, but you could make it 
launch a wrapper script that looks at what display it's on and behaves 
accordingly.
6.  There used to be something called "libteleport" that would let you transfer 
X programs from one display to another.  It was rarely used because programs 
had to be built against it, and most binary distros didn't...  But on Gentoo 
that should theoretically be easy...
7.  Several other browsers will automatically open new windows on whatever 
display their running instance is on (even over SSH...  long story...)  Since 
you don't want to switch to one of those you could port that functionality to 
Seamonkey.

I've probably missed a few options, that's just what I can think of off the top 
of my head.  

LMP

RE: [gentoo-user] Re: Searching the list archives

[Laurence Perkins]

>-Original Message-
>From: Peter Humphrey  
>Sent: Sunday, June 12, 2022 2:17 AM
>To: gentoo-user@lists.gentoo.org
>Subject: Re: [gentoo-user] Re: Searching the list archives
>
>On Sunday, 12 June 2022 09:04:30 BST Nuno Silva wrote:
>> On 2022-06-12, Peter Humphrey wrote:
>> > Hello list,
>> > 
>> > Does any site out there offer a search function over its whole archive?
>> > Going through one month at a time is going to take for ever.
>> 
>> I think the search feature at marc.info goes through the entire 
>> archive at once:
>> 
>> https://marc.info/?l=gentoo-user
>
>Certainly does - thanks Nuno.
>
>--
>Regards,
>Peter.
>
>
Now if only there were a way for people to find this message without having to 
search for it a month at a time...  :P

LMP

RE: [Possible Malware Fraud]Re: [gentoo-user] PySimpleBuild + MariaDB C Client

[Laurence Perkins]

Xbx shows up every once in a while, seems to be a young kid who doesn’t read 
English terribly well or know how mailing lists work and isn’t subscribed so 
doesn’t get list replies.

The couple I’ve picked apart didn’t seem malicious, just not actually necessary 
or particularly useful.  But I’d hate to discourage someone who’s trying to 
learn.

Of course, that could be a double-bluff of some kind just waiting for us to 
trust him.  Who knows.

LMP

From: coa...@tuta.io 
Sent: Saturday, June 11, 2022 12:21 AM
To: gentoo-user@lists.gentoo.org
Subject: [Possible Malware Fraud]Re: [gentoo-user] PySimpleBuild + MariaDB C 
Client



WARNING: Your email security system has determined the message below may be a 
potential threat.

It may trick victims into clicking a link and downloading malware. Do not open 
suspicious links.

If you do not know the sender or cannot verify the integrity of the message, 
please do not respond or click on links in the message. Depending on the 
security settings, clickable URLs may have been modified to provide additional 
security.



Dear xbx7336,

I'd like to thank you for sharing this little program with us,nevertheless I 
need you to understand that you just posted an unidentifiable zip package in a 
security and simplicity and modular linux distro mailchain, bypassing the fact 
you just used a .zip rather .tar.* I feel like I should state that simplicity 
of usage, security of though and modularity are all reasons >>Git<<
 exists.

I don't know what PySimpleBuild is but MariaDB has been released under a GPL 
license so there should NOT be a reason not to post your "zip's" contents onto 
a github, gitlab or any other third party or your own git instance.

Lets not even mention the security risks of downloading something from any 
(even trusted) email due to any and all email protocols having weaker security 
than a tamagotchi.

Jun. 7, 2022, 11:37 by xbx7...@gmail.com:
Dear All!

I would like to share this little program.

xbx.

[gentoo-user] netfilter partial MAC filtering

[Laurence Perkins]

I am designing a small system with a switch and an uplink.  It needs to be able 
to forward traffic from trusted, and only trusted, devices connected to the 
switch out through the uplink.

Since all potential trusted devices will have the same MAC OUI prefix in this 
case, the immediately obvious course of action would be to base the decision on 
that.

Unfortunately, there doesn't seem to be a good way to do so.  There was 
https://serverfault.com/questions/877576/shorewall-wildcard-filter-by-source-mac-address
 from a few years ago, with the answer being "You can't."

While I didn't bother to test it, I'm guessing that adding about 16 million MAC 
filtering rules to the firewall won't be good for performance.  I briefly 
thought I could use the string matching or the U32 filters, but unfortunately 
it appears that they can't access anything prior to the start of the IP 
section, so picking bytes out of the ethernet header isn't possible.

I did find 
https://martin.uy/blog/wildcard-support-for-mac-addresses-in-netfilter-linux-kernel-and-iptables/
   But it's old, and has something of a glaring flaw with regard to false 
wildcard matches.

I can think of a few ways to do this, mostly involving somehow monitoring 
incoming packets and noting the MAC addresses which have the correct prefix, 
and then having a little daemon pick up those addresses and add rules to let 
them through.

Either that, or try to write a custom netfilter module.

None of this seems particularly "fun" to sort out.  Does anybody know of any 
common solutions for doing packet matching based on just part of a MAC address 
on Linux?  Failing that, some advice about whether the system daemon and packet 
inspection route or the netfilter module route is more likely to be stable and 
maintainable would be appreciated.

Thanks,
LMP

RE: [gentoo-user] netfilter partial MAC filtering

[Laurence Perkins]

> -Original Message-
> From: Samuraiii  
> Sent: Thursday, June 16, 2022 9:48 PM
> To: gentoo-user@lists.gentoo.org
> Subject: Re: [gentoo-user] netfilter partial MAC filtering
> 
> On Fri, 2022-06-17 at 01:32 +0000, Laurence Perkins wrote:
> > I am designing a small system with a switch and an uplink.  It needs 
> > to be able to forward traffic from trusted, and only trusted, devices 
> > connected to the switch out through the uplink.
> >  
> > Since all potential trusted devices will have the same MAC OUI prefix 
> > in this case, the immediately obvious course of action would be to 
> > base the decision on that.
> >  
> > Unfortunately, there doesn't seem to be a good way to do so.  There 
> > was
> > https://serverfault.com/questions/877576/shorewall-wildcard-filter-
> > by-source-mac-address from a few years ago, with the answer being "You 
> > can't."
> >  
> > While I didn't bother to test it, I'm guessing that adding about 16 
> > million MAC filtering rules to the firewall won't be good for 
> > performance.  I briefly thought I could use the string matching or the 
> > U32 filters, but unfortunately it appears that they can't access 
> > anything prior to the start of the IP section, so picking bytes out of 
> > the ethernet header isn't possible.
> >  
> > I did find
> > https://martin.uy/blog/wildcard-support-for-mac-addresses-in-netfilter
> > -linux-kernel-and-iptables/
> >But it's old, and has something of a glaring flaw with regard to 
> > false wildcard matches.
> >  
> > I can think of a few ways to do this, mostly involving somehow 
> > monitoring incoming packets and noting the MAC addresses which have 
> > the correct prefix, and then having a little daemon pick up those 
> > addresses and add rules to let them through.
> >  
> > Either that, or try to write a custom netfilter module.
> >  
> > None of this seems particularly "fun" to sort out.  Does anybody know 
> > of any common solutions for doing packet matching based on just part 
> > of a MAC address on Linux?  Failing that, some advice about whether 
> > the system daemon and packet inspection route or the netfilter module 
> > route is more likely to be stable and maintainable would be 
> > appreciated.
> >  
> > Thanks,
> > LMP
> Hi,
> I would recommend to look into nftables and its set feature...
> It should perform better with one rule for multiple matches.
> I bet no one had tried it with 16M items, but it is the best, as far as I 
> know.
> Cheers
> S
> 
> 
> https://wiki.nftables.org/wiki-nftables/index.php/Sets
> https://developers.redhat.com/blog/2017/04/11/benchmarking-nftables#the_first_test
> 
I guess it's worth a shot.  If it has enough intelligence to collapse the 
search list so it only has to look for the prefix once then it would at least 
reject forbidden connections quickly...

I'll generate out the whole list and see what happens and let you guys know.

LMP