Re: Why LVM

[David Christensen]

On 4/8/24 16:54, Stefan Monnier wrote:

If I have a hot-pluggable device (SD card, USB drive, hot-plug SATA/SAS
drive and rack, etc.), can I put LVM on it such that when the device is
connected to a Debian system with a graphical desktop (I use Xfce) an icon
is displayed on the desktop that I can interact with to display the file
systems in my file manager (Thunar)?


In the past: definitely not.  Currently: no idea.
I suspect not, because I think the behavior on disconnection is still
poor (you want to be extra careful to deactivate all the volumes on the
drive *before* removing it, otherwise they tend to linger "for ever").

I guess that's one area where partitions are still significantly better
than LVM.


 Stefan "who doesn't use much hot-plugging of mass storage"



Thank you for the clarification.  :-)


David

Re: Why LVM

[Stefan Monnier]

> If I have a hot-pluggable device (SD card, USB drive, hot-plug SATA/SAS
> drive and rack, etc.), can I put LVM on it such that when the device is
> connected to a Debian system with a graphical desktop (I use Xfce) an icon
> is displayed on the desktop that I can interact with to display the file
> systems in my file manager (Thunar)?

In the past: definitely not.  Currently: no idea.
I suspect not, because I think the behavior on disconnection is still
poor (you want to be extra careful to deactivate all the volumes on the
drive *before* removing it, otherwise they tend to linger "for ever").

I guess that's one area where partitions are still significantly better
than LVM.


Stefan "who doesn't use much hot-plugging of mass storage"

Re: Why LVM

[David Christensen]

On 4/8/24 14:08, Stefan Monnier wrote:

David Christensen [2024-04-08 11:28:04] wrote:

Why LVM?


Personally, I've been using LVM everywhere I can (i.e. everywhere
except on my OpenWRT router, tho I've also used LVM there back when my
router had an HDD.  I also use LVM on my 2GB USB rescue image).

To me the question is rather the reverse: why not?
I basically see it as a more flexible form of partitioning.

Even in the worst cases where I have a single LV volume, I appreciate
the fact that it forces me to name things, isolating me from issue
linked to predicting the name of the device and the issues that plague
UUIDs (the fact they're hard to remember, and that they're a bit too
magical/hidden for my taste, so they sometimes change when I don't want
them to and vice versa).


 Stefan



If I have a hot-pluggable device (SD card, USB drive, hot-plug SATA/SAS 
drive and rack, etc.), can I put LVM on it such that when the device is 
connected to a Debian system with a graphical desktop (I use Xfce) an 
icon is displayed on the desktop that I can interact with to display the 
file systems in my file manager (Thunar)?



David

Re: Why LVM (was: HDD long-term data storage with ensured integrity)

[DdB]

Am 08.04.2024 um 23:08 schrieb Stefan Monnier:
> David Christensen [2024-04-08 11:28:04] wrote:
>> Why LVM?
> 
> Personally, I've been using LVM everywhere I can (i.e. everywhere
> except on my OpenWRT router, tho I've also used LVM there back when my
> router had an HDD.  I also use LVM on my 2GB USB rescue image).
> 
> To me the question is rather the reverse: why not?
> I basically see it as a more flexible form of partitioning.

As an LVM-newbie (never used it before, i am more familar with ZFS), i
did already collect quite a bit of misconceptions of mine/design
problems with lvm. Therefore i would rather renew the question: Why?

Just one example:
In order to be able to use thin snapshots on my root partition, i did
every thing i could, to have it inside a thinpool... until i noticed
some weird problems booting from it (attributed to grub), so i setup a
/boot outside, but the problems stayed (due to lvm's limitations).

I came to use it to gain some flexibility (although it is an experiment)
and found myself setting up zfs for its data integrity + flexibility,
just to have a quality backup of the lvm-volume(s) on a zfs pool.

> 
> Even in the worst cases where I have a single LV volume, I appreciate
> the fact that it forces me to name things, isolating me from issue
> linked to predicting the name of the device and the issues that plague
> UUIDs (the fact they're hard to remember, and that they're a bit too
> magical/hidden for my taste, so they sometimes change when I don't want
> them to and vice versa).

Even GPT brings you the chance to name hings (like part_label), only it
does not force you. But i have been using that for 10+ years as a routine.

DdB

Why LVM (was: HDD long-term data storage with ensured integrity)

[Stefan Monnier]

David Christensen [2024-04-08 11:28:04] wrote:
> Why LVM?

Personally, I've been using LVM everywhere I can (i.e. everywhere
except on my OpenWRT router, tho I've also used LVM there back when my
router had an HDD.  I also use LVM on my 2GB USB rescue image).

To me the question is rather the reverse: why not?
I basically see it as a more flexible form of partitioning.

Even in the worst cases where I have a single LV volume, I appreciate
the fact that it forces me to name things, isolating me from issue
linked to predicting the name of the device and the issues that plague
UUIDs (the fact they're hard to remember, and that they're a bit too
magical/hidden for my taste, so they sometimes change when I don't want
them to and vice versa).


Stefan

Installing? Why debian?

[yxcv]

Debian still wants interesteing user to take netinstall.iso or any other 
*iso because of Debian installation, but recommends the US-Shit GRUB. it is 
usefull to study Heise.de before and absolut all from Ubuntu about the 
GReatly Ugly and UnUseabe pile of sht B. This shit in combination with other 
maschines in  the net kill all other bootprogramms on all maschiens. This 
isn's just mad, it is criminal typical US-Manner. And Debian is it stupid 
partner just like Ukraine.
When interested for control. kill all ethernet, therefore wireless is just 
so much suppoerted. And Debian is the stupid partner. Everybody is warned.


All answers never will be read

Re: Why is /var/lib/apt/lists not in /var/cache?

[tomas]

On Mon, Dec 18, 2023 at 09:05:22AM -0500, Stefan Monnier wrote:
> > I would imagine that it's due to the FHS (Filesystem Hierarchy Standard)
> > which defines what the various directories on a "typical Linux system" are
> > for. "man hier", for example, tells me that:
> >
> > * /var/cache - Data cached for programs.
> >
> > * /var/lib - Variable state information for programs.
> 
> These leave open the question of what "data cached for the program" mean
> and what "Variable state" means.
> 
> Another way to look at it is in terms of what it implies.
> For me, `/var/lib` data is data that is important to keep, e.g. data you
> definitely don't want to delete/lose without a very good reason, whereas
> `/var/cache` is data that's handy to have but not indispensable because
> we can reconstruct it (or something close enough) from other sources,
> which means that it doesn't have to be included in backups and it can
> be deleted if it strikes your fancy (typically when you're in urgent need
> of a tiny bit more disk space).

[...]

Most probably you are right, especially since TLDP only says you can
delete /var/cache's content safely across reboots [1]. The apt lists
will be regenerated after doing an apt-get update, so...

OTOH it'd be only a true-real-honestly-promised cache if every apt
command did an apt-get update whenever it found that file missing.

Cheers

[1] https://tldp.org/LDP/Linux-Filesystem-Hierarchy/html/var.html
-- 
t


signature.asc
Description: PGP signature

Re: Why is /var/lib/apt/lists not in /var/cache?

[Stefan Monnier]

> I would imagine that it's due to the FHS (Filesystem Hierarchy Standard)
> which defines what the various directories on a "typical Linux system" are
> for. "man hier", for example, tells me that:
>
> * /var/cache - Data cached for programs.
>
> * /var/lib - Variable state information for programs.

These leave open the question of what "data cached for the program" mean
and what "Variable state" means.

Another way to look at it is in terms of what it implies.
For me, `/var/lib` data is data that is important to keep, e.g. data you
definitely don't want to delete/lose without a very good reason, whereas
`/var/cache` is data that's handy to have but not indispensable because
we can reconstruct it (or something close enough) from other sources,
which means that it doesn't have to be included in backups and it can
be deleted if it strikes your fancy (typically when you're in urgent need
of a tiny bit more disk space).

> system, so apt's state is more like the state of the repositories.

But both the "files that contain the list of .deb files" and "the .deb
files" are part of the state of the repositories :-)

Or, stated in another way, this way to look at the problem degenerates
into philosophical questions.  So I think the question "does it have
to be included in the backup" is a better guide.


Stefan

Re: Why is /var/lib/apt/lists not in /var/cache?

[Darac Marjal]

On 16/12/2023 15:59, Stefan Monnier wrote:

AFAICT, all of `/var/lib/apt/lists` is made of files fetched from
repositories, which APT will re-fetch if missing.
So, it sounds to me like it belongs in `/var/cache/apt/lists`, really.
What am I missing?  Or is it just a historical accident?


 Stefan "whose `/var/lib/apt/lists` is a symlink into /`var/cache`"


I would imagine that it's due to the FHS (Filesystem Hierarchy Standard) 
which defines what the various directories on a "typical Linux system" 
are for. "man hier", for example, tells me that:


* /var/cache - Data cached for programs.

* /var/lib - Variable state information for programs.

So, apt seems to be doing the right thing here. /var/cache is where the 
cached packages are stores, but /var/lib is where the state of the 
repositories is stored. Admittedly, the state of the repositories is 
cached information, but recall that apt is essentially just a network 
fetcher/cacher. dpkg would be the ultimate arbiter of the state of the 
system, so apt's state is more like the state of the repositories.




OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: Why is /var/lib/apt/lists not in /var/cache?

[Stefan Monnier]

>> That was a typo.  it's `/var/cache/plocate/plocate.db`, sorry.
> My plocate.db is in /var/lib/plocate/, as is bookworm's.
> Is that changing in the future?

Hmm... I could swear that I saw it in /var/cache but every machine
I look at has it in /var/lib, indeed.
[ GNU locate puts its DB in `/var/cache/locate/`, apparently, OTOH.  ]

Anyway, thanks for all the suggestions.  From what I can tell, there is
no strong reason to put the lists in `/var/lib` rather than
`/var/cache`.  I suspect this is mostly a historical accident (at some
point, erasing `/var/lib/apt/lists` caused problems because that dir
wasn't automatically recreated if missing, so you'd have to `mkdir`
manually as well as create the lock inside of it).


Stefan

Re: Why is /var/lib/apt/lists not in /var/cache?

[David Wright]

On Sun 17 Dec 2023 at 15:33:30 (-0500), Stefan Monnier wrote:
> >> That seems similar to things like `locate` failing if you remove
> >> `/var/log/plocate/plocate.db` (until that DB is rebuilt).
> >
> > It's tricky to discern your point as /var/log/ is not involved.
> 
> That was a typo.  it's `/var/cache/plocate/plocate.db`, sorry.

My plocate.db is in /var/lib/plocate/, as is bookworm's.
Is that changing in the future?

  $ ls -l /var/lib/*locate/
  /var/lib/mlocate/:
  total 19344
  -rw-r- 1 root mlocate 19807421 Aug 28 07:43 mlocate.db

  /var/lib/plocate/:
  total 20632
  -rw-r--r-- 1 root root 183 Nov 17  2021 CACHEDIR.TAG
  -rw-r- 1 root plocate 21121629 Dec 16 16:06 plocate.db
  $ 

Cheers,
David.

Re: Why is /var/lib/apt/lists not in /var/cache?

[Stefan Monnier]

>> That seems similar to things like `locate` failing if you remove
>> `/var/log/plocate/plocate.db` (until that DB is rebuilt).
>
> It's tricky to discern your point as /var/log/ is not involved.

That was a typo.  it's `/var/cache/plocate/plocate.db`, sorry.


Stefan

Re: Why is /var/lib/apt/lists not in /var/cache?

[David Wright]

On Sun 17 Dec 2023 at 01:06:28 (-0500), Stefan Monnier wrote:
> > Some packages will stay the same for years, but in the past week
> > I can see four occasions when changes in list contents have occurred
> > on oldstable. So there's little similarity.
> 
> The question is not really whether "apt/lists" is similar to
> "apt/archives", but whether the content of "apt/lists" is appropriate
> for `/var/cache`.

I didn't bring up apt/archives, but my first words posted were:
"This may not answer your question".

> AFAIK nothing bad will happen to your system if you delete
> `/var/lib/apt/lists`.  It will still run as before and you can still
> install packages as before.

As you said.

> The only thing that seems to be impacted is
> that some APT operations will temporarily work in a kind of "degraded"
> mode until the next `apt update`.

As I said.

On how significant that difference is, we beg to differ.

> That seems similar to things like `locate` failing if you remove
> `/var/log/plocate/plocate.db` (until that DB is rebuilt).

It's tricky to discern your point as /var/log/ is not involved.

Cheers,
David.

Re: Why is /var/lib/apt/lists not in /var/cache?

[Stefan Monnier]

> Some packages will stay the same for years, but in the past week
> I can see four occasions when changes in list contents have occurred
> on oldstable. So there's little similarity.

The question is not really whether "apt/lists" is similar to
"apt/archives", but whether the content of "apt/lists" is appropriate
for `/var/cache`.

AFAIK nothing bad will happen to your system if you delete
`/var/lib/apt/lists`.  It will still run as before and you can still
install packages as before.  The only thing that seems to be impacted is
that some APT operations will temporarily work in a kind of "degraded"
mode until the next `apt update`.

That seems similar to things like `locate` failing if you remove
`/var/log/plocate/plocate.db` (until that DB is rebuilt).


Stefan

Re: Why is /var/lib/apt/lists not in /var/cache?

[David Wright]

On Sat 16 Dec 2023 at 12:50:51 (-0500), Stefan Monnier wrote:
> David Wright [2023-12-16 11:30:01] wrote:
> > On Sat 16 Dec 2023 at 10:59:48 (-0500), Stefan Monnier wrote:
> >> AFAICT, all of `/var/lib/apt/lists` is made of files fetched from
> >> repositories, which APT will re-fetch if missing.
> >> So, it sounds to me like it belongs in `/var/cache/apt/lists`, really.
> >> What am I missing?  Or is it just a historical accident?
> >
> > This may not answer your question, but when you fetch new lists,
> > they aren't the same as what was there before, and don't actually
> > match your system until you upgrade it.
> 
> AFAIK those lists aren't supposed to "match your system", are they?
> IIUC they just keep a local copy of the repository's list.

Whether it's considered necessary or desirable depends on the
user and what they use the lists for (apart from the obvious).

> Admittedly if I `apt update` today, and then a week from now I delete
> the lists and do `apt update` again I won't get the same files.
> But similar, if I delete one of the files in
> `/var/cache/apt/archive`, there's no guarantee that I'll be able to
> re-download this exact file next week (tho I guess
> https://snapshot.debian.org/ might still make it possible if you want it
> badly enough).

Some packages will stay the same for years, but in the past week
I can see four occasions when changes in list contents have occurred
on oldstable. So there's little similarity.

Cheers,
David.

Re: Why is /var/lib/apt/lists not in /var/cache?

[Stefan Monnier]

Max Nikulin [2023-12-17 09:10:29] wrote:
> On 16/12/2023 22:59, Stefan Monnier wrote:
>> AFAICT, all of `/var/lib/apt/lists` is made of files fetched from
>> repositories, which APT will re-fetch if missing.
>> So, it sounds to me like it belongs in `/var/cache/apt/lists`, really.
> APT running by a regular user is unable to write to /var/cache/apt.

But neither is it able to write to `/var/lib/apt/lists`, AFAICT.


Stefan

Re: Why is /var/lib/apt/lists not in /var/cache?

[Max Nikulin]

On 16/12/2023 22:59, Stefan Monnier wrote:

AFAICT, all of `/var/lib/apt/lists` is made of files fetched from
repositories, which APT will re-fetch if missing.
So, it sounds to me like it belongs in `/var/cache/apt/lists`, really.


APT running by a regular user is unable to write to /var/cache/apt. E.g. 
DNF creates cache in the user home directory in such cases. I like APT 
approach since (at least with default settings) search queries work 
without delay while repository data is being updated. (I am realizing 
that I may get stale results.)


So for APT lists are a bit more than just cache.

Re: Why is /var/lib/apt/lists not in /var/cache?

[Stefan Monnier]

David Wright [2023-12-16 11:30:01] wrote:
> On Sat 16 Dec 2023 at 10:59:48 (-0500), Stefan Monnier wrote:
>> AFAICT, all of `/var/lib/apt/lists` is made of files fetched from
>> repositories, which APT will re-fetch if missing.
>> So, it sounds to me like it belongs in `/var/cache/apt/lists`, really.
>> What am I missing?  Or is it just a historical accident?
>
> This may not answer your question, but when you fetch new lists,
> they aren't the same as what was there before, and don't actually
> match your system until you upgrade it.

AFAIK those lists aren't supposed to "match your system", are they?
IIUC they just keep a local copy of the repository's list.

Admittedly if I `apt update` today, and then a week from now I delete
the lists and do `apt update` again I won't get the same files.
But similar, if I delete one of the files in
`/var/cache/apt/archive`, there's no guarantee that I'll be able to
re-download this exact file next week (tho I guess
https://snapshot.debian.org/ might still make it possible if you want it
badly enough).


Stefan

Re: Why is /var/lib/apt/lists not in /var/cache?

[David Wright]

On Sat 16 Dec 2023 at 10:59:48 (-0500), Stefan Monnier wrote:
> AFAICT, all of `/var/lib/apt/lists` is made of files fetched from
> repositories, which APT will re-fetch if missing.
> So, it sounds to me like it belongs in `/var/cache/apt/lists`, really.
> What am I missing?  Or is it just a historical accident?

This may not answer your question, but when you fetch new lists,
they aren't the same as what was there before, and don't actually
match your system until you upgrade it.

Cheers,
David.

Why is /var/lib/apt/lists not in /var/cache?

[Stefan Monnier]

AFAICT, all of `/var/lib/apt/lists` is made of files fetched from
repositories, which APT will re-fetch if missing.
So, it sounds to me like it belongs in `/var/cache/apt/lists`, really.
What am I missing?  Or is it just a historical accident?


Stefan "whose `/var/lib/apt/lists` is a symlink into /`var/cache`"

Re: why would "tr --complement --squeeze-repeats ..." append the substitution char once more? ...

[Albretch Mueller]

On 12/11/23, Greg Wooledge  wrote:
> 1) Many implementations of echo will interpret parts of their argument(s),
>in addition to processing options like -n.  If you want to print a
>variable's contents to standard output without *any* interpretation,
>use printf.
>
> printf %s "$myvar"
> printf '%s\n' "$myvar"
>

 I will use "printf ..." from now on.

> 2) As tomas already told you, the square brackets in
>
> tr -c -s '[A-Za-z0-9.]' _
>
>are literal.  You're using a command which will keep left and right
>square brackets in the input, *not* replacing them with underscores.
>This may not be what you want.

 My mistake, even though it didn't get in the way of what I was trying
to do. I replaced :alnum: by what I thought it meant and left the
brackets.

> 3) In locales other than C or POSIX, ranges like A-Z are *not* necessarily
>synonyms for [:upper:].  As I've already mentioned, GNU tr is known to
>contain bugs, so you're getting lucky here.  The bugs in GNU tr happen
>to work the way you're expecting, so that A-Z is treated like [:upper:]
>when it should not be.  If at some point in the future GNU tr is fixed
>to conform to POSIX, your script may break.
>
>The correct tr command you should be using if you want to retain
>accented letters (as defined in your locale) is:
>
> tr -c -s '[:alnum:].' _
>
>If you want to discard accented letters, then either of these is OK:
>
> LC_COLLATE=C tr -c -s '[:alnum:].' _
> LC_COLLATE=C tr -c -s 'A-Za-z0-9.' _
>

 I like your second one liner much better (LC_COLLATE=C tr -c -s 'A-Za-z0-9.' _)

 I tend to avoid '[:alnum:].' because the intended meaning of
"ALphabetic et NUMeric" characters, even though it depends on the
locale has a strong ASCII accent to it.

> Thus, we come full circle.

 Yes, we did. Thank you, lbrtchx

Re: why would "tr --complement --squeeze-repeats ..." append the substitution char once more? ...

[debian-user]

Albretch Mueller  wrote:
> echo "abc123" > file.txt
> ftype=$(file --brief file.txt)
> echo "// __ \$ftype: |${ftype}|"
> ftypelen=${#ftype}
> echo "// __ \$ftypelen: |${ftypelen}|"
> 
> # removing spaces ...
> ftype2=$(echo "${ftype}" | tr --complement --squeeze-repeats
> '[A-Za-z0-9.]' '_');
> echo "// __ \$ftype2: |${ftype2}|"
> ftype2len=${#ftype2}
> echo "// __ \$ftype2len: |${ftype2len}|"
> 
> lbrtchx

Short answer. tr doesn't append anything. echo does output a linefeed
at the end of the string, unless you stop it. tr dutifully translates
that to an underscore.

Re: "echo" literally in sh scripts (was: why would "tr --complement --squeeze-repeats ..." append the substitution char once more? ...)

[Greg Wooledge]

On Mon, Dec 11, 2023 at 10:16:35AM -0500, Stefan Monnier wrote:
> > 1) Many implementations of echo will interpret parts of their argument(s),
> >in addition to processing options like -n.  If you want to print a
> >variable's contents to standard output without *any* interpretation,
> >use printf.
> >
> > printf %s "$myvar"
> > printf '%s\n' "$myvar"
> 
> Interesting.  I used the following instead:
> 
> bugit_echo () {
> # POSIX `echo` has all kinds of "features" we don't want, such as
> # handling of \c and -n.
> cat < $*
> ENDDOC
> }

That requires an external command (one fork), plus whatever overhead is
used by the << implementation (temp file or pipe, depending on shell and
version).  It's not wrong, but an implementation using nothing but
builtins is usually preferable.

echo() { printf '%s\n' "$*"; }

It's also worth mentioning that both of these rely on the expansion of $*
with a default or nearly-default IFS variable.  If you want it to work
when IFS may have been altered, you can do this in bash:

echo() { local IFS=' '; printf '%s\n' "$*"; }

In sh, you'd need to fork a subshell:

echo() { (IFS=' '; printf '%s\n' "$*"); }

Or if you're a golfer:

echo() (IFS=' '; printf '%s\n' "$*")

I *really* dislike that syntax, but that's just me.  Some people use it.

"echo" literally in sh scripts (was: why would "tr --complement --squeeze-repeats ..." append the substitution char once more? ...)

[Stefan Monnier]

> 1) Many implementations of echo will interpret parts of their argument(s),
>in addition to processing options like -n.  If you want to print a
>variable's contents to standard output without *any* interpretation,
>use printf.
>
> printf %s "$myvar"
> printf '%s\n' "$myvar"

Interesting.  I used the following instead:

bugit_echo () {
# POSIX `echo` has all kinds of "features" we don't want, such as
# handling of \c and -n.
cat <

Re: why would "tr --complement --squeeze-repeats ..." append the substitution char once more? ...

[tomas]

On Mon, Dec 11, 2023 at 09:55:54AM -0500, Greg Wooledge wrote:

[...]

Greg, your analyses are always impressive. And enjoyable.

Thanks for this

cheers
-- 
t


signature.asc
Description: PGP signature

Re: why would "tr --complement --squeeze-repeats ..." append the substitution char once more? ...

[Greg Wooledge]

On Mon, Dec 11, 2023 at 02:00:49PM +, Albretch Mueller wrote:
>  Ach, yes! I forgot echo by default appends a new line character at
> the end of every string it spits out. In order to suppress it you need
> to use the "n" option: "echo -n ..."
> 
> _FL_TYPE="   abc  á é í ó ú ü ñ Á É Í Ó Ú Ü Ñ 123 birdiehere ¿ ¡ §
> ASCII  ä ö ü ß Ä Ö Ü Text"
> echo "// __ \$_FL_TYPE: |${_FL_TYPE}|"
> _FL_TYPE=$(echo "${_FL_TYPE}" | xargs)
> echo "// __ \$_FL_TYPE: |${_FL_TYPE}|"
> _FL_TYPE=$(echo -n "${_FL_TYPE}" |  tr --complement --squeeze-repeats
> '[A-Za-z0-9.]' '_');
> echo "// __ \$_FL_TYPE: |${_FL_TYPE}|"
> 
> // __ $_FL_TYPE: |   abc  á é í ó ú ü ñ Á É Í Ó Ú Ü Ñ 123 birdiehere
> ¿ ¡ § ASCII  ä ö ü ß Ä Ö Ü Text|
> // __ $_FL_TYPE: |abc á é í ó ú ü ñ Á É Í Ó Ú Ü Ñ 123 birdiehere ¿ ¡
> § ASCII ä ö ü ß Ä Ö Ü Text|
> // __ $_FL_TYPE: |abc_123_birdie_here_ASCII_Text|

OK.  Tomas's analysis was better than mine in this case.  Looks like CR
was not the issue this time around.  I do have some comments, though.

1) Many implementations of echo will interpret parts of their argument(s),
   in addition to processing options like -n.  If you want to print a
   variable's contents to standard output without *any* interpretation,
   use printf.

printf %s "$myvar"
printf '%s\n' "$myvar"

2) As tomas already told you, the square brackets in

tr -c -s '[A-Za-z0-9.]' _

   are literal.  You're using a command which will keep left and right
   square brackets in the input, *not* replacing them with underscores.
   This may not be what you want.

3) In locales other than C or POSIX, ranges like A-Z are *not* necessarily
   synonyms for [:upper:].  As I've already mentioned, GNU tr is known to
   contain bugs, so you're getting lucky here.  The bugs in GNU tr happen
   to work the way you're expecting, so that A-Z is treated like [:upper:]
   when it should not be.  If at some point in the future GNU tr is fixed
   to conform to POSIX, your script may break.

   The correct tr command you should be using if you want to retain
   accented letters (as defined in your locale) is:

tr -c -s '[:alnum:].' _

   If you want to discard accented letters, then either of these is OK:

LC_COLLATE=C tr -c -s '[:alnum:].' _
LC_COLLATE=C tr -c -s 'A-Za-z0-9.' _

4) The xargs command, which you used above, uses quotation mark characters
   as well as whitespace to define input words.  Your example worked only
   because your input does not contain any single or double quotes.

Here's a demonstration of A-Z not equating to [:upper:] using GNU sed,
which is behaving correctly:

unicorn:~$ x='   abc  á é í ó ú ü ñ Á É Í Ó Ú Ü Ñ 123 birdiehere ¿ '
unicorn:~$ printf '%s\n' "$x" | sed 's/[A-Z]//g'
   abc  á é í ó ú ü ñ123 birdiehere ¿ 
unicorn:~$ printf '%s\n' "$x" | LC_COLLATE=C sed 's/[A-Z]//g'
   abc  á é í ó ú ü ñ Á É Í Ó Ú Ü Ñ 123 birdiehere ¿ 

The meaning of [A-Z] in the sed command depends on the locale.  In my
locale, which is en_US.utf8, characters like Á are part of the A-Z range.
In the C locale, they aren't, as seen in the last command above.

The use of [A-Z] in regular expressions and globs is a *very* heavily
debated topic, and I'm only scratching the surface here.  Honestly, you
really should avoid using it.  It's just too unpredictable.

Here's an example of xargs failing when its input contains a quote:

unicorn:~$ echo 'foo "bar' | xargs
xargs: unmatched double quote; by default quotes are special to xargs 
unless you use the -0 option
foo

You can't use xargs to normalize whitespace safely.  In fact, the proper
way to normalize whitespace is...

unicorn:~$ printf 'foo "bar \t\t \t  baz  \n' | tr -s ' \t' ' '
foo "bar baz 

Thus, we come full circle.

Re: why would "tr --complement --squeeze-repeats ..." append the substitution char once more? ...

[Max Nikulin]

On 11/12/2023 21:00, Albretch Mueller wrote:

// __ $_FL_TYPE: |abc á é í ó ú ü ñ Á É Í Ó Ú Ü Ñ 123 birdiehere ¿ ¡
§ ASCII ä ö ü ß Ä Ö Ü Text|
// __ $_FL_TYPE:|abc_123_birdie_here_ASCII_Text|


https://pypi.org/project/Unidecode/
should be more friendly to languages other than English.

Re: why would "tr --complement --squeeze-repeats ..." append the substitution char once more? ...

[Albretch Mueller]

 Ach, yes! I forgot echo by default appends a new line character at
the end of every string it spits out. In order to suppress it you need
to use the "n" option: "echo -n ..."

_FL_TYPE="   abc  á é í ó ú ü ñ Á É Í Ó Ú Ü Ñ 123 birdiehere ¿ ¡ §
ASCII  ä ö ü ß Ä Ö Ü Text"
echo "// __ \$_FL_TYPE: |${_FL_TYPE}|"
_FL_TYPE=$(echo "${_FL_TYPE}" | xargs)
echo "// __ \$_FL_TYPE: |${_FL_TYPE}|"
_FL_TYPE=$(echo -n "${_FL_TYPE}" |  tr --complement --squeeze-repeats
'[A-Za-z0-9.]' '_');
echo "// __ \$_FL_TYPE: |${_FL_TYPE}|"

// __ $_FL_TYPE: |   abc  á é í ó ú ü ñ Á É Í Ó Ú Ü Ñ 123 birdiehere
¿ ¡ § ASCII  ä ö ü ß Ä Ö Ü Text|
// __ $_FL_TYPE: |abc á é í ó ú ü ñ Á É Í Ó Ú Ü Ñ 123 birdiehere ¿ ¡
§ ASCII ä ö ü ß Ä Ö Ü Text|
// __ $_FL_TYPE: |abc_123_birdie_here_ASCII_Text|

 Thank you,
 lbrtchx

Re: why would "tr --complement --squeeze-repeats ..." append the substitution char once more? ...

[Greg Wooledge]

On Mon, Dec 11, 2023 at 02:11:46PM +0100, to...@tuxteam.de wrote:
> On Mon, Dec 11, 2023 at 07:42:10AM -0500, Greg Wooledge wrote:
> > Looks like GNU tr in Debian 12 still doesn't handle multibyte characters
> > correctly:
> > 
> > unicorn:~$ echo 'mañana' | tr ñ X
> > maXXana
> 
> Hey, you just gave us a handy way to count how many encoding units
> a character takes:
> 
>   tomas@trotzki:~$ echo 'birdiehere' | tr -c 'a-z' X
>   birdiehereX

Cute as that is, there are better ways.

unicorn:~$ x=ñ; (echo "${#x}"; LC_ALL=C; echo "${#x}")
1
2

Re: why would "tr --complement --squeeze-repeats ..." append the substitution char once more? ...

[tomas]

On Mon, Dec 11, 2023 at 07:42:10AM -0500, Greg Wooledge wrote:
> On Mon, Dec 11, 2023 at 09:37:42AM +0100, to...@tuxteam.de wrote:
> >  2. This is tr, not regexp, so '[A-Za-z0-9.]' isn't doing what you
> >think it does. It will match '[', 'A' to 'Z', 'a' to 'z','.' and
> >']'. I guess you want to say 'A-Za-z0-9.'
> 
> Well spotted.
> 
> >  3. As a convenience, tr has char classes. Perhaps [:alnum:] is for
> >you. No idea whether this is a GNU extension
> 
> It's POSIX.  100% portable, as long as you ignore any bugs in GNU tr.
> 
> Looks like GNU tr in Debian 12 still doesn't handle multibyte characters
> correctly:
> 
> unicorn:~$ echo 'mañana' | tr ñ X
> maXXana
> 
> So... as long as you're working in the C locale, where [:alnum:] is
> just the ASCII capital and lowercase letters and digits, you should be
> fine.

Hey, you just gave us a handy way to count how many encoding units
a character takes:

  tomas@trotzki:~$ echo 'birdiehere' | tr -c 'a-z' X
  birdiehereX

;-)

Cheers
-- 
t


signature.asc
Description: PGP signature

Re: why would "tr --complement --squeeze-repeats ..." append the substitution char once more? ...

[Greg Wooledge]

On Mon, Dec 11, 2023 at 09:37:42AM +0100, to...@tuxteam.de wrote:
>  2. This is tr, not regexp, so '[A-Za-z0-9.]' isn't doing what you
>think it does. It will match '[', 'A' to 'Z', 'a' to 'z','.' and
>']'. I guess you want to say 'A-Za-z0-9.'

Well spotted.

>  3. As a convenience, tr has char classes. Perhaps [:alnum:] is for
>you. No idea whether this is a GNU extension

It's POSIX.  100% portable, as long as you ignore any bugs in GNU tr.

Looks like GNU tr in Debian 12 still doesn't handle multibyte characters
correctly:

unicorn:~$ echo 'mañana' | tr ñ X
maXXana

So... as long as you're working in the C locale, where [:alnum:] is
just the ASCII capital and lowercase letters and digits, you should be
fine.

Re: why would "tr --complement --squeeze-repeats ..." append the substitution char once more? ...

[Greg Wooledge]

On Mon, Dec 11, 2023 at 11:25:13AM +, Albretch Mueller wrote:
> In the case of: "ASCII text"
>  what should come out of it is: "ASCII_text"
>  not: "ASCII_text_"
>  no underscore at the end. That is the question I have.

OK, here's my guess.

The lines of code that you showed us are not actually in a script.
They're just in a FILE, and you're running a command like this:

sh myfile

Furthermore, I am guessing that the lines of code in this file have
Microsoft CR+LF line endings.  Therefore, when you do a variable
assignment like

ftype=$(file --brief "$whatever")

you end up with a Carriage Return character at the end of the variable's
content (because there is one at the end of this command).

Since you never actually SHOWED US the command you ran, or the output that
was produced, which could have made this really, really obvious, we're
forced to guess.  My guess might be right, or wrong.  But it's the best
guess I have with the limited information you've chosen to share with us.

What I mean by "obvious" is this.  Here's part of your code:

echo "abc123" > file.txt
ftype=$(file --brief file.txt)
echo "// __ \$ftype: |${ftype}|"

If my guess is correct, you got output that looks like this:

|/ __ $ftype: |ASCII text

Showing this would have made it immediately clear that a CR is involved.

Re: why would "tr --complement --squeeze-repeats ..." append the substitution char once more? ...

[tomas]

On Mon, Dec 11, 2023 at 11:25:13AM +, Albretch Mueller wrote:
>  "tr --complement --squeeze-repeats ..." makes sure that the replaced
> characters only appear once (that it doesn't immediately repeat). Say
> you have something like "  " (two spaces) or "?$|" (three characters)
> which will be replaced by just an underscore.

...which would change the length, as I wrote.

> In the case of: "ASCII text"
>  what should come out of it is: "ASCII_text"
>  not: "ASCII_text_"
>  no underscore at the end. That is the question I have.

That depends on whether your "ASCII text" has some thingy at the end
which you don't see. A newline, perchance?

>  I use such constructs as: "[A-Za-z0-9.]" to make explicit to myself
> and other people what I mean. I work in corpora research dealing with
> text based various alphabets not just in ASCII so I avoid any kinds of
> linguistic/cultural shortcuts and abbreviations.

What has this to do with how tr works? It will treat [ and ] as characters
not to substitute. I pointed that out, because it might have been unintended:

  echo -n 'This is a  text with [some brackets] in   it' | tr -cs 
"[A-Za-z0-9.]" "_"
  This_is_a_text_with_[some_brackets]_in_it

(Note this "-n" on the echo, btw? Without it, I'd be getting a "_" at the
end, the transliterated newline).

Do whatever you want :-)

Cheers
-- 
t


signature.asc
Description: PGP signature

Re: why would "tr --complement --squeeze-repeats ..." append the substitution char once more? ...

[Albretch Mueller]

 "tr --complement --squeeze-repeats ..." makes sure that the replaced
characters only appear once (that it doesn't immediately repeat). Say
you have something like "  " (two spaces) or "?$|" (three characters)
which will be replaced by just an underscore.

In the case of: "ASCII text"
 what should come out of it is: "ASCII_text"
 not: "ASCII_text_"
 no underscore at the end. That is the question I have.

 I use such constructs as: "[A-Za-z0-9.]" to make explicit to myself
and other people what I mean. I work in corpora research dealing with
text based various alphabets not just in ASCII so I avoid any kinds of
linguistic/cultural shortcuts and abbreviations.

 lbrtchx

On 12/11/23, to...@tuxteam.de  wrote:
> On Mon, Dec 11, 2023 at 08:04:06AM +, Albretch Mueller wrote:
>> On 12/11/23, Greg Wooledge  wrote:
>> > Please tell us ...
>>
>>  OK, here is what I did as a t-table
>
> [...]
>
> Your style is confusing, to say the least. Why not play with minimal
> examples and work your way up from that?
>
>> the two strings are not the same length even though your are just
>> replacing ASCII characters, why did:
>> echo "${ftype}" | tr --complement --squeeze-repeats '[A-Za-z0-9.]' '_'
>> place a character at the end?
>
> Two things stick out:
>
>  1. with --squeeze-repeats you are challenging tr to output less
>characters than the input has:
>
>trotzki:~$ echo -n "this is a #   string ###" | tr -cs 'a-z' '_'
>=> this_is_a_string_
>
>(I allowed myself to simplify things a bit) See? tr is squeezing
>repeats (repeated matches), the space-plus-three-hashes at the
>end gets squeezed to just one _, thus changing the length.
>If your strings contain more than one non-alphanumeric (something
>I don't feel like even trying a guess at), this is bound to happen.
>You ordered it.
>
>  2. This is tr, not regexp, so '[A-Za-z0-9.]' isn't doing what you
>think it does. It will match '[', 'A' to 'Z', 'a' to 'z','.' and
>']'. I guess you want to say 'A-Za-z0-9.'
>
>  3. As a convenience, tr has char classes. Perhaps [:alnum:] is for
>you. No idea whether this is a GNU extension
>
>  4. In case of doubt, read the man page :)
>
> Cheers
> --
> t
>

Re: why would "tr --complement --squeeze-repeats ..." append the substitution char once more? ...

[tomas]

On Mon, Dec 11, 2023 at 08:04:06AM +, Albretch Mueller wrote:
> On 12/11/23, Greg Wooledge  wrote:
> > Please tell us ...
> 
>  OK, here is what I did as a t-table

[...]

Your style is confusing, to say the least. Why not play with minimal
examples and work your way up from that?

> the two strings are not the same length even though your are just
> replacing ASCII characters, why did:
> echo "${ftype}" | tr --complement --squeeze-repeats '[A-Za-z0-9.]' '_'
> place a character at the end?

Two things stick out:

 1. with --squeeze-repeats you are challenging tr to output less
   characters than the input has:

   trotzki:~$ echo -n "this is a #   string ###" | tr -cs 'a-z' '_'
   => this_is_a_string_

   (I allowed myself to simplify things a bit) See? tr is squeezing
   repeats (repeated matches), the space-plus-three-hashes at the
   end gets squeezed to just one _, thus changing the length.
   If your strings contain more than one non-alphanumeric (something
   I don't feel like even trying a guess at), this is bound to happen.
   You ordered it.

 2. This is tr, not regexp, so '[A-Za-z0-9.]' isn't doing what you
   think it does. It will match '[', 'A' to 'Z', 'a' to 'z','.' and
   ']'. I guess you want to say 'A-Za-z0-9.'

 3. As a convenience, tr has char classes. Perhaps [:alnum:] is for
   you. No idea whether this is a GNU extension

 4. In case of doubt, read the man page :)

Cheers
-- 
t


signature.asc
Description: PGP signature

Re: why would "tr --complement --squeeze-repeats ..." append the substitution char once more? ...

[Albretch Mueller]

On 12/11/23, Greg Wooledge  wrote:
> Please tell us ...

 OK, here is what I did as a t-table

echo "abc123" > file.txt  # obvious text file
ftype=$(file --brief file.txt)  # got its type as reported by the "file" utility
echo "// __ \$ftype: |${ftype}|"
ftypelen=${#ftype}  # length of the string containing the file type
echo "// __ \$ftypelen: |${ftypelen}|"

# removing spaces et any other char which is not '[A-Za-z0-9.]'
replacing with underscores ...
# here is what I think to be an error happened instead of just
replacing ... by underscores
# it adds an underscore at the end?
ftype2=$(echo "${ftype}" | tr --complement --squeeze-repeats
'[A-Za-z0-9.]' '_');
echo "// __ \$ftype2: |${ftype2}|"
ftype2len=${#ftype2}
echo "// __ \$ftype2len: |${ftype2len}|"

the two strings are not the same length even though your are just
replacing ASCII characters, why did:
echo "${ftype}" | tr --complement --squeeze-repeats '[A-Za-z0-9.]' '_'
place a character at the end?
Probably echo and tr are not dancing well together. echo might be
tailgating an end of string character which tr then replaces with an
underscore.
which option do I use with echo for that not to happen?
SHould I probably play with IFS ...?

lbrtchx


On 12/11/23, Greg Wooledge  wrote:
> On Mon, Dec 11, 2023 at 02:53:07AM +, Albretch Mueller wrote:
>> echo "abc123" > file.txt
>> ftype=$(file --brief file.txt)
>> echo "// __ \$ftype: |${ftype}|"
>> ftypelen=${#ftype}
>> echo "// __ \$ftypelen: |${ftypelen}|"
>>
>> # removing spaces ...
>> ftype2=$(echo "${ftype}" | tr --complement --squeeze-repeats
>> '[A-Za-z0-9.]' '_');
>> echo "// __ \$ftype2: |${ftype2}|"
>> ftype2len=${#ftype2}
>> echo "// __ \$ftype2len: |${ftype2len}|"
>
> Please tell us:
>
>  * What you are trying to do.
>
>  * What you did (is the previous code all in a script?  if so, this is a
>good answer for this part).
>
>  * What result you got.
>
>  * What you expected to get.

Re: why would "tr --complement --squeeze-repeats ..." append the substitution char once more? ...

[Greg Wooledge]

On Mon, Dec 11, 2023 at 02:53:07AM +, Albretch Mueller wrote:
> echo "abc123" > file.txt
> ftype=$(file --brief file.txt)
> echo "// __ \$ftype: |${ftype}|"
> ftypelen=${#ftype}
> echo "// __ \$ftypelen: |${ftypelen}|"
> 
> # removing spaces ...
> ftype2=$(echo "${ftype}" | tr --complement --squeeze-repeats
> '[A-Za-z0-9.]' '_');
> echo "// __ \$ftype2: |${ftype2}|"
> ftype2len=${#ftype2}
> echo "// __ \$ftype2len: |${ftype2len}|"

Please tell us:

 * What you are trying to do.

 * What you did (is the previous code all in a script?  if so, this is a
   good answer for this part).

 * What result you got.

 * What you expected to get.

why would "tr --complement --squeeze-repeats ..." append the substitution char once more? ...

[Albretch Mueller]

echo "abc123" > file.txt
ftype=$(file --brief file.txt)
echo "// __ \$ftype: |${ftype}|"
ftypelen=${#ftype}
echo "// __ \$ftypelen: |${ftypelen}|"

# removing spaces ...
ftype2=$(echo "${ftype}" | tr --complement --squeeze-repeats
'[A-Za-z0-9.]' '_');
echo "// __ \$ftype2: |${ftype2}|"
ftype2len=${#ftype2}
echo "// __ \$ftype2len: |${ftype2len}|"

lbrtchx

Re: Why is bullseye-backports recommended on bookworm?

[David Wright]

On Mon 20 Nov 2023 at 11:12:03 (+0100), Vincent Lefevre wrote:
> On 2023-11-18 23:43:34 -0600, David Wright wrote:
> > On Sat 18 Nov 2023 at 23:33:59 (+0100), Vincent Lefevre wrote:
> > > On 2023-11-18 09:18:56 -0500, Greg Wooledge wrote:
> > > > The "6.1.0-" part comes from the upstream release series.  All the
> > > > kernel images containing "6.1.0-" in this section should come from the
> > > > same upstream series (6.1.x), and should have basically the same feature
> > > > set, with no major changes.
> > > 
> > > BTW, since this is for 6.1.x, I've always wondered why Debian uses the
> > > "6.1.0-" prefix instead of "6.1-". The "6.1.0" is a bit confusing.
> > 
> > So as not to confuse and break software that's hardwired to expect
> > three numbers in any linux kernel version.
> 
> But these numbers in the package name are not the correct ones.
> If the 3 numbers matter, this could yield a potential breakage too.

There aren't really three numbers, and it's the software that's wrong,
perhaps written back in the days of 2.6.26 (lenny) or earlier.

Cheers,
David.

Re: Why is bullseye-backports recommended on bookworm?

[Vincent Lefevre]

On 2023-11-18 23:43:34 -0600, David Wright wrote:
> On Sat 18 Nov 2023 at 23:33:59 (+0100), Vincent Lefevre wrote:
> > On 2023-11-18 09:18:56 -0500, Greg Wooledge wrote:
> > > The "6.1.0-" part comes from the upstream release series.  All the
> > > kernel images containing "6.1.0-" in this section should come from the
> > > same upstream series (6.1.x), and should have basically the same feature
> > > set, with no major changes.
> > 
> > BTW, since this is for 6.1.x, I've always wondered why Debian uses the
> > "6.1.0-" prefix instead of "6.1-". The "6.1.0" is a bit confusing.
> 
> So as not to confuse and break software that's hardwired to expect
> three numbers in any linux kernel version.

But these numbers in the package name are not the correct ones.
If the 3 numbers matter, this could yield a potential breakage too.

-- 
Vincent Lefèvre  - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Why is bullseye-backports recommended on bookworm?

[David Wright]

On Sat 18 Nov 2023 at 23:24:25 (+0100), Vincent Lefevre wrote:
> On 2023-11-18 00:20:25 -0600, David Wright wrote:
> > On Fri 17 Nov 2023 at 13:30:32 (+0100), Vincent Lefevre wrote:
> > > On 2023-11-16 14:04:29 -0600, David Wright wrote:
> > > > On Thu 16 Nov 2023 at 13:02:28 (+0100), Vincent Lefevre wrote:
> > > > > In any case, if a package is renamed (which particularly applies to
> > > > > unstable, I don't know about backports), I would expect reportbug
> > > > > to also consider the new name for a newer version of the package.
> > > > > In short, its search for newer versions should be based on the
> > > > > source package rather than the binary package.
> > > > 
> > > > As I said above, I don't know whether they apply any fuzziness to the
> > > > version numbers in view of the multiplicity of linux-image versions
> > > > (and sources). As far as a 'rename' is concerned, I don't think that
> > > > linux-image has changed name since it was kernel-image in sarge.
> > > 
> > > The name of the binary package frequently changes. This is why Tixy
> > > said "Because it's a different package?".
> > 
> > Tixy said that because the bookworm-backports packages are
> > called "linux-image-6.4.0…" which are all from a different kernel
> > source.
> 
> He didn't explain. So I thought that he meant the usual rename of
> the binary packages from the same kernel source.

As I wrote elsewhere, I think we're using the 'rename' differently.

> > I would call linux-image-x.y.z-386 → linux-image-x.y.z-486
> > and suchlike a name change.
> > 
> > > > > Note that for the Packages files, reportbug just uses the files from
> > > > > the /var/lib/apt/lists directory, but I don't have anything matching
> > > > > *bullseye* there.
> > > > 
> > > > I didn't know that, and at least one post in this thread suggests
> > > > otherwise.
> > > 
> > > I'm wondering why you think that.
> > 
> > Only because Greg wrote ‘What it said was "Hey, I looked on the
> > internets and I saw this other kernel that might be newer than the
> > one you're running, so maybe you wanna check this other kernel first
> > and see if it's still got the same bug, before you report this."’
> 
> This does not necessarily mean that it fetches Packages files there.
> There are various means to get package information on the web.

I wasn't implying that reportbug downloaded Packages files from
anywhere. You might have thought I did because /I/ downloaded the
backports Packages file, but that was because I don't know how to
use your 'various means' to get package information from the web.

My point was that Greg's post suggested reportbug could range much
further afield than your computer's APT lists and sources.list.

Cheers,
David.

Re: Why is bullseye-backports recommended on bookworm?

[David Wright]

On Sat 18 Nov 2023 at 23:33:59 (+0100), Vincent Lefevre wrote:
> On 2023-11-18 09:18:56 -0500, Greg Wooledge wrote:
> > The "6.1.0-" part comes from the upstream release series.  All the
> > kernel images containing "6.1.0-" in this section should come from the
> > same upstream series (6.1.x), and should have basically the same feature
> > set, with no major changes.
> 
> BTW, since this is for 6.1.x, I've always wondered why Debian uses the
> "6.1.0-" prefix instead of "6.1-". The "6.1.0" is a bit confusing.

So as not to confuse and break software that's hardwired to expect
three numbers in any linux kernel version.

Cheers,
David.

Re: Why is bullseye-backports recommended on bookworm?

[David Wright]

On Sat 18 Nov 2023 at 15:29:51 (+0100), steve wrote:
> Le 18-11-2023, à 09:18:56 -0500, Greg Wooledge a écrit :
> > On Sat, Nov 18, 2023 at 12:24:30AM -0600, David Wright wrote:
> > > On Fri 17 Nov 2023 at 14:07:54 (+), Tixy wrote:
> > > > At time of writing, that depended on package in stable is called
> > > > 'linux-image-6.1.0-13-amd64' and the version of that package is
> > > > '6.1.55-1'. This is the kernel installed on my machine.
> > > 
> > > And AIUI that version is the upstream source version, and a Debian
> > > counter for that source. The counter is rarely used, AFAICT, and can
> > > cause consternation when it is, because it means the kernel gets
> > > upgraded 'in place', making it tricky to revert if you wanted to.
> > > (That shouldn't normally be necessary.) And I'm sure you know all
> > > this, or something like it.
> > 
> > Debian kernel images have a complex naming system, to be sure.  Let's
> > look at the package name first: linux-image-6.1.0-13-amd64
> > 
> > The "linux-image-" part is obvious.  That's static.
> > 
> > The "6.1.0-" part comes from the upstream release series.  All the
> > kernel images containing "6.1.0-" in this section should come from the
> > same upstream series (6.1.x), and should have basically the same feature
> > set, with no major changes.
> > 
> > The "13" is the ABI (Application Binary Interface) identifier.  This
> > gets incremented each time the kernel's internal structures change in
> > a way that would require kernel modules to be recompiled.
> > 
> > And finally, the "-amd64" part is the architecture.

I used the term flavour, to encompass the variety within the i386
architecture: {3,4,5,6}86[-pae][-smp] and so on. I don't follow
other architectures enough to know whether a similar variety
exists elsewhere.

> > Next, look at the package version string: 6.1.55-1
> > 
> > The "6.1.55" part is the upstream release number.  In this case, this
> > is the 55th point release in the upstream 6.1.x series.
> > 
> > The "-1" indicates that this is the first Debian package built from
> > this upstream release, by the Debian kernel image maintainers.
> > 
> > Now, let's say a major bug is found in this kernel, and the maintainers
> > decide to release a new kernel package built from the same upstream
> > source, but with a fix.  Depending on the changes they make, one of two
> > things can happen:
> > 
> > 1) If the fix doesn't require an ABI change (old modules can be loaded
> >   by the new kernel), then they only have to increment the package
> >   version number.  So they'll release package linux-image-6.1.0-13-amd64
> >   version 6.1.55-2.  (Or if it were the security team doing it, then
> >   the version number would be something like 6.1.55-1+deb12u1 instead.)

We saw that happening in July with linux-image-5.10.0-23-amd64
(bullseye), when there were three versions (sources 5.10.179-{1,2,3})
in use over a period of 12 weeks (6 CVEs).

> > 2) If the fix requires an ABI change, then a new package name has to
> >   be created.  In this case, they'll release a new package
> >   linux-image-6.1.0-14-amd64 with version 6.1.55-1 (or something
> >   like 6.1.55-0+deb12u1 maybe, although the security team is much
> >   less likely to invoke an ABI change).
> > 
> > In practice, though, new kernel images are most likely to be released
> > after a whole bunch of upstream point releases have occurred, and
> > will roll up all of those upstream changes into one gigantic change.
> > So we would most likely jump from linux-image-6.1.0-13-amd64 version
> > 6.1.55-1 to linux-image-6.1.0-14-amd64 version 6.1.72-1 (or something
> > along those lines).  Because so many changes get amalgamated together,
> > it's vanishingly rare for the ABI counter *not* to increment.
> 
> Thanks Greg for the precise explanation. I would suggest to put it in the
> Debian Wiki for futur reference.

Cheers,
David.

Re: Why is bullseye-backports recommended on bookworm?

[Tim Woodall]

On Tue, 14 Nov 2023, Vincent Lefevre wrote:


To my surprise, reportbug asks me to use bullseye-backports
(= oldstable-backports) on my bookworm (= stable) machine:

Your version (6.1.55-1) of linux-image-6.1.0-13-amd64 appears to be out of date.
The following newer release(s) are available in the Debian archive:
 bullseye-backports (backports-policy): 6.1.55+1~bpo11+1
Please try to verify if the bug you are about to report is already addressed by 
these releases.  Do you still want to file a report [y|N|q|?]?

Why?



I'm not exactly sure how numbering works with unstable but
6.1.55+1~bpo11+1 comes after 6.1.55-1 but before 6.1.55+1

I'm not sure how you've ended up with a version with a '-' or bpo has
ended up with a '+'.

I thought the whole point of the bpo numbering was so it sorted before
the next release.

Re: Why is bullseye-backports recommended on bookworm?

[Vincent Lefevre]

On 2023-11-18 09:18:56 -0500, Greg Wooledge wrote:
> The "6.1.0-" part comes from the upstream release series.  All the
> kernel images containing "6.1.0-" in this section should come from the
> same upstream series (6.1.x), and should have basically the same feature
> set, with no major changes.

BTW, since this is for 6.1.x, I've always wondered why Debian uses the
"6.1.0-" prefix instead of "6.1-". The "6.1.0" is a bit confusing.

-- 
Vincent Lefèvre  - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Why is bullseye-backports recommended on bookworm?

[Vincent Lefevre]

On 2023-11-18 00:20:25 -0600, David Wright wrote:
> On Fri 17 Nov 2023 at 13:30:32 (+0100), Vincent Lefevre wrote:
> > On 2023-11-16 14:04:29 -0600, David Wright wrote:
> > > On Thu 16 Nov 2023 at 13:02:28 (+0100), Vincent Lefevre wrote:
> > > > In any case, if a package is renamed (which particularly applies to
> > > > unstable, I don't know about backports), I would expect reportbug
> > > > to also consider the new name for a newer version of the package.
> > > > In short, its search for newer versions should be based on the
> > > > source package rather than the binary package.
> > > 
> > > As I said above, I don't know whether they apply any fuzziness to the
> > > version numbers in view of the multiplicity of linux-image versions
> > > (and sources). As far as a 'rename' is concerned, I don't think that
> > > linux-image has changed name since it was kernel-image in sarge.
> > 
> > The name of the binary package frequently changes. This is why Tixy
> > said "Because it's a different package?".
> 
> Tixy said that because the bookworm-backports packages are
> called "linux-image-6.4.0…" which are all from a different kernel
> source.

He didn't explain. So I thought that he meant the usual rename of
the binary packages from the same kernel source.

> I would call linux-image-x.y.z-386 → linux-image-x.y.z-486
> and suchlike a name change.
> 
> > > > Note that for the Packages files, reportbug just uses the files from
> > > > the /var/lib/apt/lists directory, but I don't have anything matching
> > > > *bullseye* there.
> > > 
> > > I didn't know that, and at least one post in this thread suggests
> > > otherwise.
> > 
> > I'm wondering why you think that.
> 
> Only because Greg wrote ‘What it said was "Hey, I looked on the
> internets and I saw this other kernel that might be newer than the
> one you're running, so maybe you wanna check this other kernel first
> and see if it's still got the same bug, before you report this."’

This does not necessarily mean that it fetches Packages files there.
There are various means to get package information on the web.

-- 
Vincent Lefèvre  - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Why is bullseye-backports recommended on bookworm?

[steve]

Thanks Greg for the precise explanation. I would suggest to put it in the
Debian Wiki for futur reference.

Le 18-11-2023, à 09:18:56 -0500, Greg Wooledge a écrit :


On Sat, Nov 18, 2023 at 12:24:30AM -0600, David Wright wrote:

On Fri 17 Nov 2023 at 14:07:54 (+), Tixy wrote:
> At time of writing, that depended on package in stable is called
> 'linux-image-6.1.0-13-amd64' and the version of that package is
> '6.1.55-1'. This is the kernel installed on my machine.

And AIUI that version is the upstream source version, and a Debian
counter for that source. The counter is rarely used, AFAICT, and can
cause consternation when it is, because it means the kernel gets
upgraded 'in place', making it tricky to revert if you wanted to.
(That shouldn't normally be necessary.) And I'm sure you know all
this, or something like it.


Debian kernel images have a complex naming system, to be sure.  Let's
look at the package name first: linux-image-6.1.0-13-amd64

The "linux-image-" part is obvious.  That's static.

The "6.1.0-" part comes from the upstream release series.  All the
kernel images containing "6.1.0-" in this section should come from the
same upstream series (6.1.x), and should have basically the same feature
set, with no major changes.

The "13" is the ABI (Application Binary Interface) identifier.  This
gets incremented each time the kernel's internal structures change in
a way that would require kernel modules to be recompiled.

And finally, the "-amd64" part is the architecture.

Next, look at the package version string: 6.1.55-1

The "6.1.55" part is the upstream release number.  In this case, this
is the 55th point release in the upstream 6.1.x series.

The "-1" indicates that this is the first Debian package built from
this upstream release, by the Debian kernel image maintainers.

Now, let's say a major bug is found in this kernel, and the maintainers
decide to release a new kernel package built from the same upstream
source, but with a fix.  Depending on the changes they make, one of two
things can happen:

1) If the fix doesn't require an ABI change (old modules can be loaded
  by the new kernel), then they only have to increment the package
  version number.  So they'll release package linux-image-6.1.0-13-amd64
  version 6.1.55-2.  (Or if it were the security team doing it, then
  the version number would be something like 6.1.55-1+deb12u1 instead.)

2) If the fix requires an ABI change, then a new package name has to
  be created.  In this case, they'll release a new package
  linux-image-6.1.0-14-amd64 with version 6.1.55-1 (or something
  like 6.1.55-0+deb12u1 maybe, although the security team is much
  less likely to invoke an ABI change).

In practice, though, new kernel images are most likely to be released
after a whole bunch of upstream point releases have occurred, and
will roll up all of those upstream changes into one gigantic change.
So we would most likely jump from linux-image-6.1.0-13-amd64 version
6.1.55-1 to linux-image-6.1.0-14-amd64 version 6.1.72-1 (or something
along those lines).  Because so many changes get amalgamated together,
it's vanishingly rare for the ABI counter *not* to increment.

Re: Why is bullseye-backports recommended on bookworm?

[Greg Wooledge]

On Sat, Nov 18, 2023 at 12:24:30AM -0600, David Wright wrote:
> On Fri 17 Nov 2023 at 14:07:54 (+), Tixy wrote:
> > At time of writing, that depended on package in stable is called
> > 'linux-image-6.1.0-13-amd64' and the version of that package is
> > '6.1.55-1'. This is the kernel installed on my machine.
> 
> And AIUI that version is the upstream source version, and a Debian
> counter for that source. The counter is rarely used, AFAICT, and can
> cause consternation when it is, because it means the kernel gets
> upgraded 'in place', making it tricky to revert if you wanted to.
> (That shouldn't normally be necessary.) And I'm sure you know all
> this, or something like it.

Debian kernel images have a complex naming system, to be sure.  Let's
look at the package name first: linux-image-6.1.0-13-amd64

The "linux-image-" part is obvious.  That's static.

The "6.1.0-" part comes from the upstream release series.  All the
kernel images containing "6.1.0-" in this section should come from the
same upstream series (6.1.x), and should have basically the same feature
set, with no major changes.

The "13" is the ABI (Application Binary Interface) identifier.  This
gets incremented each time the kernel's internal structures change in
a way that would require kernel modules to be recompiled.

And finally, the "-amd64" part is the architecture.

Next, look at the package version string: 6.1.55-1

The "6.1.55" part is the upstream release number.  In this case, this
is the 55th point release in the upstream 6.1.x series.

The "-1" indicates that this is the first Debian package built from
this upstream release, by the Debian kernel image maintainers.

Now, let's say a major bug is found in this kernel, and the maintainers
decide to release a new kernel package built from the same upstream
source, but with a fix.  Depending on the changes they make, one of two
things can happen:

1) If the fix doesn't require an ABI change (old modules can be loaded
   by the new kernel), then they only have to increment the package
   version number.  So they'll release package linux-image-6.1.0-13-amd64
   version 6.1.55-2.  (Or if it were the security team doing it, then
   the version number would be something like 6.1.55-1+deb12u1 instead.)

2) If the fix requires an ABI change, then a new package name has to
   be created.  In this case, they'll release a new package
   linux-image-6.1.0-14-amd64 with version 6.1.55-1 (or something
   like 6.1.55-0+deb12u1 maybe, although the security team is much
   less likely to invoke an ABI change).

In practice, though, new kernel images are most likely to be released
after a whole bunch of upstream point releases have occurred, and
will roll up all of those upstream changes into one gigantic change.
So we would most likely jump from linux-image-6.1.0-13-amd64 version
6.1.55-1 to linux-image-6.1.0-14-amd64 version 6.1.72-1 (or something
along those lines).  Because so many changes get amalgamated together,
it's vanishingly rare for the ABI counter *not* to increment.

Re: Why is bullseye-backports recommended on bookworm?

[David Wright]

On Fri 17 Nov 2023 at 14:07:54 (+), Tixy wrote:
> On Thu, 2023-11-16 at 14:04 -0600, David Wright wrote:
> > On Thu 16 Nov 2023 at 13:02:28 (+0100), Vincent Lefevre wrote:
> > > On 2023-11-15 13:54:51 -0600, David Wright wrote:
> > > > On Wed 15 Nov 2023 at 20:01:20 (+0100), Vincent Lefevre wrote:
> > > > > On 2023-11-15 18:06:45 +, Tixy wrote:
> > > > > > On Wed, 2023-11-15 at 18:15 +0100, Vincent Lefevre wrote:
> > > > > > > On 2023-11-15 16:39:15 -, Curt wrote:
> > > > > > > > On 2023-11-14, Vincent Lefevre  wrote:
> > > > > > > > > 
> > > > > > > > > The base number is the same, but I would have thought that 
> > > > > > > > > this other
> > > > > > > > > kernel might have additional patches.
> > > > > > > > > 
> > > > > > > > > > That's why I suggested ignoring the message.
> > > > > > > > > 
> > > > > > > > > Then why does reportbug mention the bullseye-backports kernel?
> > > > > > > > 
> > > > > > > > Because it kind of looks newer if you're a not very bright 
> > > > > > > > software
> > > > > > > > construct, he opined.
> > > > > > > 
> > > > > > > But the bookworm-backports kernel is even newer.
> > > > > > > So why not this one?
> > > > > > 
> > > > > > Because it's a different package?
> > > > > 
> > > > > There is no guarantee that a package with the same name in a
> > > > > different distribution has the same meaning (because packages
> > > > > get renamed...).

This is the bit I don't agree with. Perhaps just terminology…

> > > > > So I would say that this is not a good reason.
> > > > Well, it would seem strange to provide a backport for a package
> > > > and call it by a different name. But with kernels, there's always
> > > > the problem of a myriad of slightly different versions, so a
> > > > fuzzy name match might be appropriate.
> > > 
> > > In any case, if a package is renamed (which particularly applies to
> > > unstable, I don't know about backports), I would expect reportbug
> > > to also consider the new name for a newer version of the package.
> > > In short, its search for newer versions should be based on the
> > > source package rather than the binary package.
> > 
> > As I said above, I don't know whether they apply any fuzziness to the
> > version numbers in view of the multiplicity of linux-image versions
> > (and sources). As far as a 'rename' is concerned, I don't think that
> > linux-image has changed name since it was kernel-image in sarge.
> 
> There is no binary package called 'linux-image'.

Of course, the names of Debian packaged binary kernel images are an
agglomeration of 'linux-image', flavour, upstream version, and a
Debian version counter, incremented AIUI when the internal interfaces
change (so that the modules will stay in step).

> My PC has 'linux-
> image-amd64' installed which is a meta package who's description says
> 'This package depends on the latest Linux kernel and modules for use on
> PCs with AMD64'.
> 
> At time of writing, that depended on package in stable is called
> 'linux-image-6.1.0-13-amd64' and the version of that package is
> '6.1.55-1'. This is the kernel installed on my machine.

And AIUI that version is the upstream source version, and a Debian
counter for that source. The counter is rarely used, AFAICT, and can
cause consternation when it is, because it means the kernel gets
upgraded 'in place', making it tricky to revert if you wanted to.
(That shouldn't normally be necessary.) And I'm sure you know all
this, or something like it.

> In the original post that sparked this thread, Vincent showed reportbug
> saying that same binary package (linux-image-6.1.0-13-amd64) had a
> newer version available in backports, with the version number
> 6.1.55+1~bpo11+1.
> 
> Report bug is correct, that is a newer version by Debian versioning
> rules, there is no 'fuzzy' matching involved...
> 
if↘
> $ dpkg --compare-versions 6.1.55+1~bpo11+1 gt 6.1.55-1; then echo True; else 
> echo False; fi
> True
> 
> Now, I admit, looking in a prior releases backports for a newer version
> seems wrong, assuming the machine in question didn't have that release
> in it's source.list.

It is fuzzy in one sense. When APT and dpkg compare versions, they us

Re: Why is bullseye-backports recommended on bookworm?

[David Wright]

On Fri 17 Nov 2023 at 13:30:32 (+0100), Vincent Lefevre wrote:
> On 2023-11-16 14:04:29 -0600, David Wright wrote:
> > On Thu 16 Nov 2023 at 13:02:28 (+0100), Vincent Lefevre wrote:
> > > In any case, if a package is renamed (which particularly applies to
> > > unstable, I don't know about backports), I would expect reportbug
> > > to also consider the new name for a newer version of the package.
> > > In short, its search for newer versions should be based on the
> > > source package rather than the binary package.
> > 
> > As I said above, I don't know whether they apply any fuzziness to the
> > version numbers in view of the multiplicity of linux-image versions
> > (and sources). As far as a 'rename' is concerned, I don't think that
> > linux-image has changed name since it was kernel-image in sarge.
> 
> The name of the binary package frequently changes. This is why Tixy
> said "Because it's a different package?".

Tixy said that because the bookworm-backports packages are
called "linux-image-6.4.0…" which are all from a different kernel
source. I would call linux-image-x.y.z-386 → linux-image-x.y.z-486
and suchlike a name change.

> > > Note that for the Packages files, reportbug just uses the files from
> > > the /var/lib/apt/lists directory, but I don't have anything matching
> > > *bullseye* there.
> > 
> > I didn't know that, and at least one post in this thread suggests
> > otherwise.
> 
> I'm wondering why you think that.

Only because Greg wrote ‘What it said was "Hey, I looked on the
internets and I saw this other kernel that might be newer than the
one you're running, so maybe you wanna check this other kernel first
and see if it's still got the same bug, before you report this."’

Cheers,
David.

Re: Why is bullseye-backports recommended on bookworm?

[Tixy]

On Thu, 2023-11-16 at 14:04 -0600, David Wright wrote:
> On Thu 16 Nov 2023 at 13:02:28 (+0100), Vincent Lefevre wrote:
> > On 2023-11-15 13:54:51 -0600, David Wright wrote:
> > > On Wed 15 Nov 2023 at 20:01:20 (+0100), Vincent Lefevre wrote:
> > > > On 2023-11-15 18:06:45 +, Tixy wrote:
> > > > > On Wed, 2023-11-15 at 18:15 +0100, Vincent Lefevre wrote:
> > > > > > On 2023-11-15 16:39:15 -, Curt wrote:
> > > > > > > On 2023-11-14, Vincent Lefevre  wrote:
> > > > > > > > 
> > > > > > > > The base number is the same, but I would have thought that this 
> > > > > > > > other
> > > > > > > > kernel might have additional patches.
> > > > > > > > 
> > > > > > > > > That's why I suggested ignoring the message.
> > > > > > > > 
> > > > > > > > Then why does reportbug mention the bullseye-backports kernel?
> > > > > > > 
> > > > > > > Because it kind of looks newer if you're a not very bright 
> > > > > > > software
> > > > > > > construct, he opined.
> > > > > > 
> > > > > > But the bookworm-backports kernel is even newer.
> > > > > > So why not this one?
> > > > > 
> > > > > Because it's a different package?
> > > > 
> > > > There is no guarantee that a package with the same name in a
> > > > different distribution has the same meaning (because packages
> > > > get renamed...). So I would say that this is not a good reason.
> > > 
> > > Well, it would seem strange to provide a backport for a package
> > > and call it by a different name. But with kernels, there's always
> > > the problem of a myriad of slightly different versions, so a
> > > fuzzy name match might be appropriate.
> > 
> > In any case, if a package is renamed (which particularly applies to
> > unstable, I don't know about backports), I would expect reportbug
> > to also consider the new name for a newer version of the package.
> > In short, its search for newer versions should be based on the
> > source package rather than the binary package.
> 
> As I said above, I don't know whether they apply any fuzziness to the
> version numbers in view of the multiplicity of linux-image versions
> (and sources). As far as a 'rename' is concerned, I don't think that
> linux-image has changed name since it was kernel-image in sarge.

There is no binary package called 'linux-image'. My PC has 'linux-
image-amd64' installed which is a meta package who's description says
'This package depends on the latest Linux kernel and modules for use on
PCs with AMD64'.

At time of writing, that depended on package in stable is called
'linux-image-6.1.0-13-amd64' and the version of that package is
'6.1.55-1'. This is the kernel installed on my machine.

In the original post that sparked this thread, Vincent showed reportbug
saying that same binary package (linux-image-6.1.0-13-amd64) had a
newer version available in backports, with the version number
6.1.55+1~bpo11+1.

Report bug is correct, that is a newer version by Debian versioning
rules, there is no 'fuzzy' matching involved...

$ dpkg --compare-versions 6.1.55+1~bpo11+1 gt 6.1.55-1; then echo True; else 
echo False; fi
True

Now, I admit, looking in a prior releases backports for a newer version
seems wrong, assuming the machine in question didn't have that release
in it's source.list.

-- 
Tixy

Re: Why is bullseye-backports recommended on bookworm?

[Vincent Lefevre]

On 2023-11-16 14:04:29 -0600, David Wright wrote:
> On Thu 16 Nov 2023 at 13:02:28 (+0100), Vincent Lefevre wrote:
> > In any case, if a package is renamed (which particularly applies to
> > unstable, I don't know about backports), I would expect reportbug
> > to also consider the new name for a newer version of the package.
> > In short, its search for newer versions should be based on the
> > source package rather than the binary package.
> 
> As I said above, I don't know whether they apply any fuzziness to the
> version numbers in view of the multiplicity of linux-image versions
> (and sources). As far as a 'rename' is concerned, I don't think that
> linux-image has changed name since it was kernel-image in sarge.

The name of the binary package frequently changes. This is why Tixy
said "Because it's a different package?".

> > Note that for the Packages files, reportbug just uses the files from
> > the /var/lib/apt/lists directory, but I don't have anything matching
> > *bullseye* there.
> 
> I didn't know that, and at least one post in this thread suggests
> otherwise.

I'm wondering why you think that. Earlier in this thread:


On 2023-11-14 23:54:31 +0700, Max Nikulin wrote:
> On 14/11/2023 19:00, Vincent Lefevre wrote:
> > To my surprise, reportbug asks me to use bullseye-backports
> > (= oldstable-backports) on my bookworm (= stable) machine:
>
> Might it happen that you have bullseye-backports in apt sources.list?

No, and this is actually the complaint of reportbug, which wants
me to add it.


In my bug report (bug 1055931), Nis Martensen found where
6.1.55+1~bpo11+1 came from. As a summary from

  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1055931#34


On 2023-11-16 17:31:13 +0100, Nis Martensen wrote:
> I can find "6.1.55+1~bpo11+1" in https://ftp-master.debian.org/new.822
> so it must come from there.

Thanks. I had first looked at

  https://ftp-master.debian.org/new.html

(as output by reportbug), and it doesn't appear on this page
(I searched for both "linux" and "6.1.55"). Note that clicking
on "Click to toggle all/binary-NEW packages" does not make this
kernel appear either.

FYI, I later looked at https://ftp-master.debian.org/new.822 too
(as I could see it in the strace output), but only searched for
linux-image there, explaining that I didn't find it either (it
actually appears as linux-signed-amd64).

At the bottom of the new.html page:
"You can also look at the RFC822 version."

But why are the contents different? (linux-signed-amd64 appears
only in the RFC822 version.)


-- 
Vincent Lefèvre  - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Why is bullseye-backports recommended on bookworm?

[David Wright]

On Thu 16 Nov 2023 at 13:02:28 (+0100), Vincent Lefevre wrote:
> On 2023-11-15 13:54:51 -0600, David Wright wrote:
> > On Wed 15 Nov 2023 at 20:01:20 (+0100), Vincent Lefevre wrote:
> > > On 2023-11-15 18:06:45 +, Tixy wrote:
> > > > On Wed, 2023-11-15 at 18:15 +0100, Vincent Lefevre wrote:
> > > > > On 2023-11-15 16:39:15 -, Curt wrote:
> > > > > > On 2023-11-14, Vincent Lefevre  wrote:
> > > > > > > 
> > > > > > > The base number is the same, but I would have thought that this 
> > > > > > > other
> > > > > > > kernel might have additional patches.
> > > > > > > 
> > > > > > > > That's why I suggested ignoring the message.
> > > > > > > 
> > > > > > > Then why does reportbug mention the bullseye-backports kernel?
> > > > > > 
> > > > > > Because it kind of looks newer if you're a not very bright software
> > > > > > construct, he opined.
> > > > > 
> > > > > But the bookworm-backports kernel is even newer.
> > > > > So why not this one?
> > > > 
> > > > Because it's a different package?
> > > 
> > > There is no guarantee that a package with the same name in a
> > > different distribution has the same meaning (because packages
> > > get renamed...). So I would say that this is not a good reason.
> > 
> > Well, it would seem strange to provide a backport for a package
> > and call it by a different name. But with kernels, there's always
> > the problem of a myriad of slightly different versions, so a
> > fuzzy name match might be appropriate.
> 
> In any case, if a package is renamed (which particularly applies to
> unstable, I don't know about backports), I would expect reportbug
> to also consider the new name for a newer version of the package.
> In short, its search for newer versions should be based on the
> source package rather than the binary package.

As I said above, I don't know whether they apply any fuzziness to the
version numbers in view of the multiplicity of linux-image versions
(and sources). As far as a 'rename' is concerned, I don't think that
linux-image has changed name since it was kernel-image in sarge.

> > > But I'm still wondering where reportbug gets this particular
> > > version 6.1.55+1~bpo11+1, as it is not in bullseye-backports.
> > 
> > I just downloaded 
> > /debian/dists/bullseye-backports/main/binary-amd64/Packages.xz
> > (2023-11-02 13:59 395K), and it contains:
> [...]
> 
> Note that for the Packages files, reportbug just uses the files from
> the /var/lib/apt/lists directory, but I don't have anything matching
> *bullseye* there.

I didn't know that, and at least one post in this thread suggests otherwise.

> > so there do appear to be 6.1.55-1~bpo11+1 candidates, like
> > linux-image-6.1.0-0.deb11.13-amd64-unsigned.
> 
> Note the difference between 6.1.55-1~bpo11+1 and 6.1.55+1~bpo11+1
> (a "-" has been replaced by "+").

Yes, and I assumed that might be because sources are being looked up
as well as binaries. Looking at the list I posted, the top half has
the Sources (src) specified as plain "linux". The bottom half is more
forthcoming, with versions in parentheses. I can only assume that
these are source versions and that's the reason for their + signs.

But this is way deeper than I have plumbed in version numbering.
I used to compile custom kernels when the hardware was much simpler,
but my versioning was always protected by a nice big Epoch.

Cheers,
David.

Re: Why is bullseye-backports recommended on bookworm?

[Vincent Lefevre]

On 2023-11-15 13:54:51 -0600, David Wright wrote:
> On Wed 15 Nov 2023 at 20:01:20 (+0100), Vincent Lefevre wrote:
> > On 2023-11-15 18:06:45 +, Tixy wrote:
> > > On Wed, 2023-11-15 at 18:15 +0100, Vincent Lefevre wrote:
[...]
> > > > But the bookworm-backports kernel is even newer.
> > > > So why not this one?
> > > 
> > > Because it's a different package?
[...]
> I just downloaded 
> /debian/dists/bullseye-backports/main/binary-amd64/Packages.xz
> (2023-11-02 13:59 395K), and it contains:
[...]
> so there do appear to be 6.1.55-1~bpo11+1 candidates, like
> linux-image-6.1.0-0.deb11.13-amd64-unsigned.

Note that these 6.1.55-1~bpo11+1 candidates do not match my binary
package. So, if these were the reason of the output, then the
answer to "Because it's a different package?" above would be "No".

-- 
Vincent Lefèvre  - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Why is bullseye-backports recommended on bookworm?

[Vincent Lefevre]

On 2023-11-15 13:54:51 -0600, David Wright wrote:
> On Wed 15 Nov 2023 at 20:01:20 (+0100), Vincent Lefevre wrote:
> > On 2023-11-15 18:06:45 +, Tixy wrote:
> > > On Wed, 2023-11-15 at 18:15 +0100, Vincent Lefevre wrote:
> > > > On 2023-11-15 16:39:15 -, Curt wrote:
> > > > > On 2023-11-14, Vincent Lefevre  wrote:
> > > > > > 
> > > > > > The base number is the same, but I would have thought that this 
> > > > > > other
> > > > > > kernel might have additional patches.
> > > > > > 
> > > > > > > That's why I suggested ignoring the message.
> > > > > > 
> > > > > > Then why does reportbug mention the bullseye-backports kernel?
> > > > > 
> > > > > Because it kind of looks newer if you're a not very bright software
> > > > > construct, he opined.
> > > > 
> > > > But the bookworm-backports kernel is even newer.
> > > > So why not this one?
> > > 
> > > Because it's a different package?
> > 
> > There is no guarantee that a package with the same name in a
> > different distribution has the same meaning (because packages
> > get renamed...). So I would say that this is not a good reason.
> 
> Well, it would seem strange to provide a backport for a package
> and call it by a different name. But with kernels, there's always
> the problem of a myriad of slightly different versions, so a
> fuzzy name match might be appropriate.

In any case, if a package is renamed (which particularly applies to
unstable, I don't know about backports), I would expect reportbug
to also consider the new name for a newer version of the package.
In short, its search for newer versions should be based on the
source package rather than the binary package.

> > But I'm still wondering where reportbug gets this particular
> > version 6.1.55+1~bpo11+1, as it is not in bullseye-backports.
> 
> I just downloaded 
> /debian/dists/bullseye-backports/main/binary-amd64/Packages.xz
> (2023-11-02 13:59 395K), and it contains:
[...]

Note that for the Packages files, reportbug just uses the files from
the /var/lib/apt/lists directory, but I don't have anything matching
*bullseye* there.

> so there do appear to be 6.1.55-1~bpo11+1 candidates, like
> linux-image-6.1.0-0.deb11.13-amd64-unsigned.

Note the difference between 6.1.55-1~bpo11+1 and 6.1.55+1~bpo11+1
(a "-" has been replaced by "+").

> I don't know how reportbug operates; nor do I know how to
> drive madison—perhaps it's seeing the third from last line.
> But I'm not sure why you're making such an issue out of
> reportbug's harmless suggestion to check whether you're
> up-to-date.

/usr/lib/python3/dist-packages/reportbug/checkversions.py uses

RMADISON_URL = 'https://qa.debian.org/madison.php?package=%s=on'
NEWQUEUE_URL = 'http://ftp-master.debian.org/new.822'

Now, for the RMADISON_URL URL:

url = RMADISON_URL % package
if dists:
url += '=' + ','.join(dists)
# select only those lines that refers to source pkg
# or to binary packages available on the current arch
url += '=source,all,' + arch
try:
page = open_url(url, http_proxy, timeout)

If package is the binary package, I don't get the expected
version.

If I use "linux" (for the source package), I get in particular

 linux | 6.1.55-1~bpo11+1  | bullseye-backports | source

which looks like what is output, but since the 4 field is "source",
it is ignored:

if a == 'source':
continue

-- 
Vincent Lefèvre  - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Why is bullseye-backports recommended on bookworm?

[David Wright]

On Wed 15 Nov 2023 at 20:01:20 (+0100), Vincent Lefevre wrote:
> On 2023-11-15 18:06:45 +, Tixy wrote:
> > On Wed, 2023-11-15 at 18:15 +0100, Vincent Lefevre wrote:
> > > On 2023-11-15 16:39:15 -, Curt wrote:
> > > > On 2023-11-14, Vincent Lefevre  wrote:
> > > > > 
> > > > > The base number is the same, but I would have thought that this other
> > > > > kernel might have additional patches.
> > > > > 
> > > > > > That's why I suggested ignoring the message.
> > > > > 
> > > > > Then why does reportbug mention the bullseye-backports kernel?
> > > > 
> > > > Because it kind of looks newer if you're a not very bright software
> > > > construct, he opined.
> > > 
> > > But the bookworm-backports kernel is even newer.
> > > So why not this one?
> > 
> > Because it's a different package?
> 
> There is no guarantee that a package with the same name in a
> different distribution has the same meaning (because packages
> get renamed...). So I would say that this is not a good reason.

Well, it would seem strange to provide a backport for a package
and call it by a different name. But with kernels, there's always
the problem of a myriad of slightly different versions, so a
fuzzy name match might be appropriate.

> But I'm still wondering where reportbug gets this particular
> version 6.1.55+1~bpo11+1, as it is not in bullseye-backports.

I just downloaded /debian/dists/bullseye-backports/main/binary-amd64/Packages.xz
(2023-11-02 13:59 395K), and it contains:

$ zgrep -A3 '^Package: linux-image' Packages.xz | paste - - - - - | sed 
's/Package: //;s/\tSource:/ src/;s/\tVersion:/ ver/;s/\tInstalled-Size:/ 
isize/;s/\t--//'
linux-image-6.1.0-0.deb11.11-amd64-dbg src linux ver 6.1.38-4~bpo11+1 isize 
6336657
linux-image-6.1.0-0.deb11.11-amd64-unsigned src linux ver 6.1.38-4~bpo11+1 
isize 499959
linux-image-6.1.0-0.deb11.11-cloud-amd64-dbg src linux ver 6.1.38-4~bpo11+1 
isize 2051897
linux-image-6.1.0-0.deb11.11-cloud-amd64-unsigned src linux ver 
6.1.38-4~bpo11+1 isize 145318
linux-image-6.1.0-0.deb11.11-rt-amd64-dbg src linux ver 6.1.38-4~bpo11+1 isize 
6404909
linux-image-6.1.0-0.deb11.11-rt-amd64-unsigned src linux ver 6.1.38-4~bpo11+1 
isize 518751
linux-image-6.1.0-0.deb11.13-amd64-dbg src linux ver 6.1.55-1~bpo11+1 isize 
6340686
linux-image-6.1.0-0.deb11.13-amd64-unsigned src linux ver 6.1.55-1~bpo11+1 
isize 499954
linux-image-6.1.0-0.deb11.13-cloud-amd64-dbg src linux ver 6.1.55-1~bpo11+1 
isize 2051852
linux-image-6.1.0-0.deb11.13-cloud-amd64-unsigned src linux ver 
6.1.55-1~bpo11+1 isize 145473
linux-image-6.1.0-0.deb11.13-rt-amd64-dbg src linux ver 6.1.55-1~bpo11+1 isize 
6409844
linux-image-6.1.0-0.deb11.13-rt-amd64-unsigned src linux ver 6.1.55-1~bpo11+1 
isize 518558
linux-image-amd64-dbg src linux ver 6.1.55-1~bpo11+1 isize 13
linux-image-amd64-signed-template src linux ver 6.1.55-1~bpo11+1 isize 3884
linux-image-cloud-amd64-dbg src linux ver 6.1.55-1~bpo11+1 isize 13
linux-image-rt-amd64-dbg src linux ver 6.1.55-1~bpo11+1 isize 13
linux-image-6.1.0-0.deb11.11-amd64 src linux-signed-amd64 (6.1.38+4~bpo11+1) 
ver 6.1.38-4~bpo11+1 isize 501754
linux-image-6.1.0-0.deb11.11-cloud-amd64 src linux-signed-amd64 
(6.1.38+4~bpo11+1) ver 6.1.38-4~bpo11+1 isize 145823
linux-image-6.1.0-0.deb11.11-rt-amd64 src linux-signed-amd64 (6.1.38+4~bpo11+1) 
ver 6.1.38-4~bpo11+1 isize 520577
linux-image-6.1.0-0.deb11.9-amd64 src linux-signed-amd64 (6.1.27+1~bpo11+1) ver 
6.1.27-1~bpo11+1 isize 501563
linux-image-6.1.0-0.deb11.9-cloud-amd64 src linux-signed-amd64 
(6.1.27+1~bpo11+1) ver 6.1.27-1~bpo11+1 isize 145610
linux-image-6.1.0-0.deb11.9-rt-amd64 src linux-signed-amd64 (6.1.27+1~bpo11+1) 
ver 6.1.27-1~bpo11+1 isize 520274
linux-image-amd64 src linux-signed-amd64 (6.1.38+4~bpo11+1) ver 
6.1.38-4~bpo11+1 isize 13
linux-image-cloud-amd64 src linux-signed-amd64 (6.1.38+4~bpo11+1) ver 
6.1.38-4~bpo11+1 isize 13
linux-image-rt-amd64 src linux-signed-amd64 (6.1.38+4~bpo11+1) ver 
6.1.38-4~bpo11+1 isize 13
$ 

so there do appear to be 6.1.55-1~bpo11+1 candidates, like
linux-image-6.1.0-0.deb11.13-amd64-unsigned.

I don't know how reportbug operates; nor do I know how to
drive madison—perhaps it's seeing the third from last line.
But I'm not sure why you're making such an issue out of
reportbug's harmless suggestion to check whether you're
up-to-date.

Cheers,
David.

Re: Why is bullseye-backports recommended on bookworm?

[Vincent Lefevre]

On 2023-11-15 18:06:45 +, Tixy wrote:
> On Wed, 2023-11-15 at 18:15 +0100, Vincent Lefevre wrote:
> > On 2023-11-15 16:39:15 -, Curt wrote:
> > > On 2023-11-14, Vincent Lefevre  wrote:
> > > > 
> > > > The base number is the same, but I would have thought that this other
> > > > kernel might have additional patches.
> > > > 
> > > > > That's why I suggested ignoring the message.
> > > > 
> > > > Then why does reportbug mention the bullseye-backports kernel?
> > > 
> > > Because it kind of looks newer if you're a not very bright software
> > > construct, he opined.
> > 
> > But the bookworm-backports kernel is even newer.
> > So why not this one?
> 
> Because it's a different package?

There is no guarantee that a package with the same name in a
different distribution has the same meaning (because packages
get renamed...). So I would say that this is not a good reason.
But I'm still wondering where reportbug gets this particular
version 6.1.55+1~bpo11+1, as it is not in bullseye-backports.

-- 
Vincent Lefèvre  - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Why is bullseye-backports recommended on bookworm?

[Tixy]

On Wed, 2023-11-15 at 18:15 +0100, Vincent Lefevre wrote:
> On 2023-11-15 16:39:15 -, Curt wrote:
> > On 2023-11-14, Vincent Lefevre  wrote:
> > > 
> > > The base number is the same, but I would have thought that this other
> > > kernel might have additional patches.
> > > 
> > > > That's why I suggested ignoring the message.
> > > 
> > > Then why does reportbug mention the bullseye-backports kernel?
> > 
> > Because it kind of looks newer if you're a not very bright software
> > construct, he opined.
> 
> But the bookworm-backports kernel is even newer.
> So why not this one?

Because it's a different package? bookworm-backports has a linux-6.5
package which is not a newer version of the linux-6.1 package it's
totally separate as far as the packaging system is concerned.

-- 
Tixy

Re: Why is bullseye-backports recommended on bookworm?

[Vincent Lefevre]

On 2023-11-15 16:39:15 -, Curt wrote:
> On 2023-11-14, Vincent Lefevre  wrote:
> >
> > The base number is the same, but I would have thought that this other
> > kernel might have additional patches.
> >
> >> That's why I suggested ignoring the message.
> >
> > Then why does reportbug mention the bullseye-backports kernel?
> 
> Because it kind of looks newer if you're a not very bright software
> construct, he opined.

But the bookworm-backports kernel is even newer.
So why not this one?

-- 
Vincent Lefèvre  - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Why is bullseye-backports recommended on bookworm?

[Vincent Lefevre]

On 2023-11-15 08:50:50 +0100, didier gaumet wrote:
> I don't know why particularly a Bullseye-backports kernel is promoted here
> in a mixed stable/unstable context but perhaps (I have not tested it) you
> could set check-available to 0 in /etc/reportbug.conf (1) to avoid to be
> proposed a newer kernel at all.
> 
> (1) https://manpages.debian.org/bookworm/reportbug/reportbug.conf.5.en.html

This would apply to *all* packages. I certainly don't want to do that.

-- 
Vincent Lefèvre  - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Why is bullseye-backports recommended on bookworm?

[Vincent Lefevre]

On 2023-11-15 10:15:35 +0700, Max Nikulin wrote:
> On 15/11/2023 05:01, Vincent Lefevre wrote:
> > > On Tue, Nov 14, 2023 at 10:21:13PM +0100, Vincent Lefevre wrote:
> > > > # $ wget -qO- 
> > > > 'https://qa.debian.org/madison.php?package=emacs=on=oldstable,stable,testing,unstable,experimental=source,all,x86_64'
> 
> The same request without s=... returns versions for all dists and it is
> valid way to call get_newqueue_available. I agree that oldstable-backports
> is confusing, but perhaps it is better to leave decision to common sense of
> users. Too strict filtering might have negative effect in corner cases.

https://qa.debian.org/madison.php?package=linux-image-6.1.0-13-amd64=on=source,all,x86_64
just gives

 linux-image-6.1.0-13-amd64 | 6.1.55-1 | bookworm | amd64

And if you mean the request

  
https://qa.debian.org/madison.php?package=linux-image-amd64=on=source,all,x86_64

then I get

 linux-image-amd64 | 3.16+63+deb8u2  | jessie | amd64, i386
 linux-image-amd64 | 3.16+63+deb8u7  | jessie-security| amd64, i386
 linux-image-amd64 | 4.9+80+deb9u11  | stretch| amd64
 linux-image-amd64 | 4.9+80+deb9u17  | stretch-security   | amd64
 linux-image-amd64 | 4.19+105+deb10u4~bpo9+1 | stretch-backports  | amd64
 linux-image-amd64 | 4.19+105+deb10u16   | buster | amd64
 linux-image-amd64 | 4.19+105+deb10u20   | buster-security| amd64
 linux-image-amd64 | 5.10.127-2~bpo10+1  | buster-backports   | amd64
 linux-image-amd64 | 5.10.191-1  | bullseye-security  | amd64
 linux-image-amd64 | 5.10.197-1  | bullseye   | amd64
 linux-image-amd64 | 6.1.38-4~bpo11+1| bullseye-backports | amd64
 linux-image-amd64 | 6.1.52-1| bookworm-security  | amd64
 linux-image-amd64 | 6.1.55-1| bookworm   | amd64
 linux-image-amd64 | 6.5.3-1~bpo12+1 | bookworm-backports | amd64
 linux-image-amd64 | 6.5.10-1| trixie | amd64
 linux-image-amd64 | 6.5.10-1| sid| amd64

But "reportbug linux-image-6.1.0-13-amd64" still says

The following newer release(s) are available in the Debian archive:
  bullseye-backports (backports-policy): 6.1.55+1~bpo11+1

This doesn't match bullseye-backports in the above list.

In any case, it would have been *much* better to give the
bookworm-backports one.

> > Yes, because the plan is to upgrade the machine to unstable.
> > But I'm trying to solve the touchpad issue first.

Since the bookworm-backports kernel is similar to unstable, perhaps
I should fully upgrade the machine to unstable, and see. In any case,
if the issue still occurs, it would not be specific to unstable.

[Off-topic for this thread...]

> I mostly use mouse, but I have realized that what you described may be
> similar to what I have seen as well. It might be intended behavior:
> 
> https://wayland.freedesktop.org/libinput/doc/latest/clickpad-softbuttons.html#id5
> > Left: moving a finger into the right button area does not trigger a 
> > right-button click.
> 
> After moving cursor, position of finger is likely determined by desired
> cursor position that may be inside the area of an arbitrary clickbutton. On
> the other hand it is too aggressive protection against clicking a wrong
> button.

When the issue occurs, the kernel does not report any click, whatever
the position of the finger. And this can last several minutes. I don't
see any reason and any relation with the libinput behavior.

Moreover, it is said "libinput ignores such button clicks, this
behavior is intentional". So it is libinput that ignores button
clicks. In my case, it is the kernel that does not generate click
events (as seen with evtest).

> Have you tried tools specific to libinput instead of xinput?

I don't use xinput. But what needs to be fixed is on the kernel side.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Why is bullseye-backports recommended on bookworm?

[Curt]

On 2023-11-14, Vincent Lefevre  wrote:
>
> The base number is the same, but I would have thought that this other
> kernel might have additional patches.
>
>> That's why I suggested ignoring the message.
>
> Then why does reportbug mention the bullseye-backports kernel?
>

Because it kind of looks newer if you're a not very bright software
construct, he opined.

 
 

Re: Why is bullseye-backports recommended on bookworm?

[didier gaumet]

Le 14/11/2023 à 23:01, Vincent Lefevre a écrit :
[...]

Then why does reportbug mention the bullseye-backports kernel?

[...]

Hello,

I don't know why particularly a Bullseye-backports kernel is promoted 
here in a mixed stable/unstable context but perhaps (I have not tested 
it) you could set check-available to 0 in /etc/reportbug.conf (1) to 
avoid to be proposed a newer kernel at all.


(1) https://manpages.debian.org/bookworm/reportbug/reportbug.conf.5.en.html

Re: Why is bullseye-backports recommended on bookworm?

[Max Nikulin]

On 15/11/2023 05:01, Vincent Lefevre wrote:

On Tue, Nov 14, 2023 at 10:21:13PM +0100, Vincent Lefevre wrote:

# $ wget -qO- 
'https://qa.debian.org/madison.php?package=emacs=on=oldstable,stable,testing,unstable,experimental=source,all,x86_64'


The same request without s=... returns versions for all dists and it is 
valid way to call get_newqueue_available. I agree that 
oldstable-backports is confusing, but perhaps it is better to leave 
decision to common sense of users. Too strict filtering might have 
negative effect in corner cases.



Yes, because the plan is to upgrade the machine to unstable.
But I'm trying to solve the touchpad issue first.


I mostly use mouse, but I have realized that what you described may be 
similar to what I have seen as well. It might be intended behavior:


https://wayland.freedesktop.org/libinput/doc/latest/clickpad-softbuttons.html#id5

Left: moving a finger into the right button area does not trigger a 
right-button click.


After moving cursor, position of finger is likely determined by desired 
cursor position that may be inside the area of an arbitrary clickbutton. 
On the other hand it is too aggressive protection against clicking a 
wrong button.


Tap-to-click with one, two, or three fingers has no such issue.

Have you tried tools specific to libinput instead of xinput?

Perhaps the that thread on touchpad would be more appropriate for this part.

Re: Why is bullseye-backports recommended on bookworm?

[Vincent Lefevre]

On 2023-11-14 16:34:18 -0500, Greg Wooledge wrote:
> On Tue, Nov 14, 2023 at 10:21:13PM +0100, Vincent Lefevre wrote:
> > On 2023-11-14 23:54:31 +0700, Max Nikulin wrote:
> > > On 14/11/2023 19:00, Vincent Lefevre wrote:
> > > > To my surprise, reportbug asks me to use bullseye-backports
> > > > (= oldstable-backports) on my bookworm (= stable) machine:
> > > 
> > > Might it happen that you have bullseye-backports in apt sources.list?
> > 
> > No, and this is actually the complaint of reportbug, which wants
> > me to add it.
> 
> That is NOT what it said, at all.
> 
> What it said was "Hey, I looked on the internets and I saw this other
> kernel that might be newer than the one you're running, so maybe you
> wanna check this other kernel first and see if it's still got the same
> bug, before you report this."
> 
> But it's NOT actually newer than yours.  It's the same as yours.  It just
> has a bigger, fancier version string because it's a backport.  So it
> kinda looks newer if you are a not very bright software construct.

The base number is the same, but I would have thought that this other
kernel might have additional patches.

> That's why I suggested ignoring the message.

Then why does reportbug mention the bullseye-backports kernel?

> > > apt policy linux-image-amd64
> > 
> > linux-image-amd64:
> >   Installed: 6.1.55-1
> >   Candidate: 6.1.55-1
> >   Version table:
> >  6.5.10-1 500
> > 500 https://ftp.debian.org/debian testing/main amd64 Packages
> > 500 https://ftp.debian.org/debian unstable/main amd64 Packages
> >  *** 6.1.55-1 900
> > 900 https://ftp.debian.org/debian stable/main amd64 Packages
> > 100 /var/lib/dpkg/status
> >  6.1.52-1 900
> > 900 https://security.debian.org/debian-security 
> > stable-security/main amd64 Packages
> 
> You are not running a "bookworm (= stable)" system.  You're running
> unstable.  This doesn't change my analysis, but it's something you should
> be aware of.

Yes, because the plan is to upgrade the machine to unstable.
But I'm trying to solve the touchpad issue first.

-- 
Vincent Lefèvre  - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Why is bullseye-backports recommended on bookworm?

[Greg Wooledge]

On Tue, Nov 14, 2023 at 10:21:13PM +0100, Vincent Lefevre wrote:
> On 2023-11-14 23:54:31 +0700, Max Nikulin wrote:
> > On 14/11/2023 19:00, Vincent Lefevre wrote:
> > > To my surprise, reportbug asks me to use bullseye-backports
> > > (= oldstable-backports) on my bookworm (= stable) machine:
> > 
> > Might it happen that you have bullseye-backports in apt sources.list?
> 
> No, and this is actually the complaint of reportbug, which wants
> me to add it.

That is NOT what it said, at all.

What it said was "Hey, I looked on the internets and I saw this other
kernel that might be newer than the one you're running, so maybe you
wanna check this other kernel first and see if it's still got the same
bug, before you report this."

But it's NOT actually newer than yours.  It's the same as yours.  It just
has a bigger, fancier version string because it's a backport.  So it
kinda looks newer if you are a not very bright software construct.

That's why I suggested ignoring the message.

> > apt policy linux-image-amd64
> 
> linux-image-amd64:
>   Installed: 6.1.55-1
>   Candidate: 6.1.55-1
>   Version table:
>  6.5.10-1 500
> 500 https://ftp.debian.org/debian testing/main amd64 Packages
> 500 https://ftp.debian.org/debian unstable/main amd64 Packages
>  *** 6.1.55-1 900
> 900 https://ftp.debian.org/debian stable/main amd64 Packages
> 100 /var/lib/dpkg/status
>  6.1.52-1 900
> 900 https://security.debian.org/debian-security stable-security/main 
> amd64 Packages

You are not running a "bookworm (= stable)" system.  You're running
unstable.  This doesn't change my analysis, but it's something you should
be aware of.

When you mix binary repositories of unstable, testing and bookworm,
you are by definition running unstable.  "But pinning!"  Nope.  It's an
unstable system, no matter how many hoops you jump through trying to
control the frankendebian.

Re: Why is bullseye-backports recommended on bookworm?

[Vincent Lefevre]

On 2023-11-14 23:54:31 +0700, Max Nikulin wrote:
> On 14/11/2023 19:00, Vincent Lefevre wrote:
> > To my surprise, reportbug asks me to use bullseye-backports
> > (= oldstable-backports) on my bookworm (= stable) machine:
> 
> Might it happen that you have bullseye-backports in apt sources.list?

No, and this is actually the complaint of reportbug, which wants
me to add it.

> apt policy

Package files:
 100 /var/lib/dpkg/status
 release a=now
 500 file:/var/local/apt ./ Packages
 release c=
 900 http://debug.mirrors.debian.org/debian-debug proposed-updates-debug/main 
amd64 Packages
 release 
v=12-updates,o=Debian,a=proposed-updates-debug,n=bookworm-proposed-updates-debug,l=Debian
 debug,c=main,b=amd64
 origin debug.mirrors.debian.org
 900 http://debug.mirrors.debian.org/debian-debug stable-debug/main amd64 
Packages
 release v=12.2,o=Debian,a=stable-debug,n=bookworm-debug,l=Debian 
debug,c=main,b=amd64
 origin debug.mirrors.debian.org
 500 http://debug.mirrors.debian.org/debian-debug unstable-debug/main amd64 
Packages
 release o=Debian,a=unstable-debug,n=sid-debug,l=Debian debug,c=main,b=amd64
 origin debug.mirrors.debian.org
   1 https://ftp.debian.org/debian experimental/main amd64 Packages
 release o=Debian,a=experimental,n=rc-buggy,l=Debian,c=main,b=amd64
 origin ftp.debian.org
 500 https://ftp.debian.org/debian unstable/non-free-firmware amd64 Packages
 release o=Debian,a=unstable,n=sid,l=Debian,c=non-free-firmware,b=amd64
 origin ftp.debian.org
 500 https://ftp.debian.org/debian unstable/non-free amd64 Packages
 release o=Debian,a=unstable,n=sid,l=Debian,c=non-free,b=amd64
 origin ftp.debian.org
 500 https://ftp.debian.org/debian unstable/contrib amd64 Packages
 release o=Debian,a=unstable,n=sid,l=Debian,c=contrib,b=amd64
 origin ftp.debian.org
 500 https://ftp.debian.org/debian unstable/main amd64 Packages
 release o=Debian,a=unstable,n=sid,l=Debian,c=main,b=amd64
 origin ftp.debian.org
 500 https://ftp.debian.org/debian testing/non-free-firmware amd64 Packages
 release o=Debian,a=testing,n=trixie,l=Debian,c=non-free-firmware,b=amd64
 origin ftp.debian.org
 500 https://ftp.debian.org/debian testing/non-free amd64 Packages
 release o=Debian,a=testing,n=trixie,l=Debian,c=non-free,b=amd64
 origin ftp.debian.org
 500 https://ftp.debian.org/debian testing/contrib amd64 Packages
 release o=Debian,a=testing,n=trixie,l=Debian,c=contrib,b=amd64
 origin ftp.debian.org
 500 https://ftp.debian.org/debian testing/main amd64 Packages
 release o=Debian,a=testing,n=trixie,l=Debian,c=main,b=amd64
 origin ftp.debian.org
 900 https://ftp.debian.org/debian stable-updates/main amd64 Packages
 release 
v=12-updates,o=Debian,a=stable-updates,n=bookworm-updates,l=Debian,c=main,b=amd64
 origin ftp.debian.org
 900 https://security.debian.org/debian-security 
stable-security/non-free-firmware amd64 Packages
 release 
v=12,o=Debian,a=stable-security,n=bookworm-security,l=Debian-Security,c=non-free-firmware,b=amd64
 origin security.debian.org
 900 https://security.debian.org/debian-security stable-security/contrib amd64 
Packages
 release 
v=12,o=Debian,a=stable-security,n=bookworm-security,l=Debian-Security,c=contrib,b=amd64
 origin security.debian.org
 900 https://security.debian.org/debian-security stable-security/main amd64 
Packages
 release 
v=12,o=Debian,a=stable-security,n=bookworm-security,l=Debian-Security,c=main,b=amd64
 origin security.debian.org
 900 https://ftp.debian.org/debian stable/non-free-firmware amd64 Packages
 release 
v=12.2,o=Debian,a=stable,n=bookworm,l=Debian,c=non-free-firmware,b=amd64
 origin ftp.debian.org
 900 https://ftp.debian.org/debian stable/non-free amd64 Packages
 release v=12.2,o=Debian,a=stable,n=bookworm,l=Debian,c=non-free,b=amd64
 origin ftp.debian.org
 900 https://ftp.debian.org/debian stable/contrib amd64 Packages
 release v=12.2,o=Debian,a=stable,n=bookworm,l=Debian,c=contrib,b=amd64
 origin ftp.debian.org
 900 https://ftp.debian.org/debian stable/main amd64 Packages
 release v=12.2,o=Debian,a=stable,n=bookworm,l=Debian,c=main,b=amd64
 origin ftp.debian.org
Pinned packages:

> apt policy linux-image-amd64

linux-image-amd64:
  Installed: 6.1.55-1
  Candidate: 6.1.55-1
  Version table:
 6.5.10-1 500
500 https://ftp.debian.org/debian testing/main amd64 Packages
500 https://ftp.debian.org/debian unstable/main amd64 Packages
 *** 6.1.55-1 900
900 https://ftp.debian.org/debian stable/main amd64 Packages
100 /var/lib/dpkg/status
 6.1.52-1 900
900 https://security.debian.org/debian-security stable-security/main 
amd64 Packages

No traces of bullseye in either output.

Note that /usr/lib/python3/dist-packages/reportbug/debbugs.py has
references to oldstable, and oldstable-backports in particular:

oldstable = utils.SUITE2CODENAME['oldstable']
oldstable_pu = 

Re: Why is bullseye-backports recommended on bookworm?

[Max Nikulin]

On 14/11/2023 19:00, Vincent Lefevre wrote:

To my surprise, reportbug asks me to use bullseye-backports
(= oldstable-backports) on my bookworm (= stable) machine:


Might it happen that you have bullseye-backports in apt sources.list?

apt policy
apt policy linux-image-amd64

Re: Why is bullseye-backports recommended on bookworm?

[Greg Wooledge]

On Tue, Nov 14, 2023 at 01:00:47PM +0100, Vincent Lefevre wrote:
> To my surprise, reportbug asks me to use bullseye-backports
> (= oldstable-backports) on my bookworm (= stable) machine:
> 
> Your version (6.1.55-1) of linux-image-6.1.0-13-amd64 appears to be out of 
> date.
> The following newer release(s) are available in the Debian archive:
>   bullseye-backports (backports-policy): 6.1.55+1~bpo11+1
> Please try to verify if the bug you are about to report is already addressed 
> by these releases.  Do you still want to file a report [y|N|q|?]? 

Looks like something that can be ignored.  They're probably both the
same kernel source anyway, just built in different environments.

Why is bullseye-backports recommended on bookworm?

[Vincent Lefevre]

To my surprise, reportbug asks me to use bullseye-backports
(= oldstable-backports) on my bookworm (= stable) machine:

Your version (6.1.55-1) of linux-image-6.1.0-13-amd64 appears to be out of date.
The following newer release(s) are available in the Debian archive:
  bullseye-backports (backports-policy): 6.1.55+1~bpo11+1
Please try to verify if the bug you are about to report is already addressed by 
these releases.  Do you still want to file a report [y|N|q|?]? 

Why?

Shouldn't a more recent kernel be available in bookworm-backports
(= stable-backports)?

-- 
Vincent Lefèvre  - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: can you parse and "tail" at once? (and if you can't why not?)

[Albretch Mueller]

 Thank you very much, Greg!
 Since ".description" is constant (an extension used by youtube) I chose to go:
 ydx=".description"
 ydxL=${#ydx}
 ...
 yut=${yl:-${ydxL}}
...
 where yl is the line read in in the way you suggested.
 lbrtchx

Re: can you parse and "tail" at once? (and if you can't why not?)

[Andy Smith]

Hello,

On Mon, Oct 23, 2023 at 01:32:08AM +, Albretch Mueller wrote:
> On 10/22/23, Andy Smith  wrote:
> > So you might consider telling us what you will do next with the
> > suffix of each line.
> 
>  Now you have gone into mind reading mode.

I've gone into "avoid the need for mind reading" mode. Because I
could see that from your very limited provision of context, you were
going to get a lot of different solutions from different people,
and that you were likely to disregard most of those due to context
you've not supplied. So I've suggested you supply more contact so
that people DON'T have to read your mind.

> On 10/22/23, Andy Smith  wrote:
> > Most of the points Greg makes to you are matters of correctness, not
> > matters of taste.
> 
>  that you called "matters of correctness" may be "visual things" to
> other people.

I think it's important to you understand that often only the
experienced person can tell the difference.

> from where do you take the time to keep talking about any of it?

Okay, message received and understood. I won't be troubling you
again.

Thanks,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting

Re: Shell-script variable name case [WAS: Re: can you parse and "tail" at once? (and if you can't why not?)]

[Eduardo M KALINOWSKI]

On 23/10/2023 10:56, David wrote:

Hi, for your info, this convention is specified by POSIX:
   https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html

Which says:
   Environment variable names used by the utilities in the Shell and
   Utilities volume of POSIX.1-2017 consist solely of uppercase letters,
   digits, and the  ( '_' ) from the characters defined in
   Portable Character Set and do not begin with a digit.

And:
   The name space of environment variable names containing lowercase letters
   is reserved for applications. Applications can define any environment
   variables with names from this name space without modifying the behavior
   of the standard utilities.


That's good to know, and pretty much invalidates everything I said.

--
BOFH excuse #239:

CPU needs bearings repacked

Eduardo M KALINOWSKI
edua...@kalinowski.com.br

Shell-script variable name case [WAS: Re: can you parse and "tail" at once? (and if you can't why not?)]

[David]

On Mon, 23 Oct 2023 at 13:25, Eduardo M KALINOWSKI
 wrote:
> On 22/10/2023 23:13, Greg Wooledge wrote:

> > 2) All-caps variable name IFL.  All-caps variable names are reserved,
> >by convention, for environment variables (e.g. PATH) and special
> >shell variables (e.g. IFS).

> While I don't disagree with the suggestion of using lower case for
> variables (and have even started doing so myself), it seems that this
> "convention" is far from universal.

> I did a quick search on the bash manual page and found no suggestion on
> how to name user variables, or that uppercase names are reserved (but it
> was a very quick search - I might have skipped something).

> Even an internet search shows that people seem to be divided:

Hi, for your info, this convention is specified by POSIX:
  https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html

Which says:
  Environment variable names used by the utilities in the Shell and
  Utilities volume of POSIX.1-2017 consist solely of uppercase letters,
  digits, and the  ( '_' ) from the characters defined in
  Portable Character Set and do not begin with a digit.

And:
  The name space of environment variable names containing lowercase letters
  is reserved for applications. Applications can define any environment
  variables with names from this name space without modifying the behavior
  of the standard utilities.

Re: can you parse and "tail" at once? (and if you can't why not?)

[Eduardo M KALINOWSKI]

On 22/10/2023 23:13, Greg Wooledge wrote:

2) All-caps variable name IFL.  All-caps variable names are reserved, by
convention, for environment variables (e.g. PATH) and special shell
variables (e.g. IFS).


While I don't disagree with the suggestion of using lower case for 
variables (and have even started doing so myself), it seems that this 
"convention" is far from universal.


I did a quick search on the bash manual page and found no suggestion on 
how to name user variables, or that uppercase names are reserved (but it 
was a very quick search - I might have skipped something).


Even an internet search shows that people seem to be divided: this style 
guide[0], for example, suggests lower case variable names (for the same 
reason), but suggests that if the variable is actually a constant, it 
should be upper case. (This seems to be influenced by Java conventions.)


[0] 
https://google.github.io/styleguide/shellguide.html#s7-naming-conventions


This guy[1] even says that all variables should be uppercase.

[1] https://www.linkedin.com/pulse/bash-scripting-conventions-engin-polat

It's certainly very common (even if unfortunate) to use all uppercase 
variables, as you certainly know.


So while I agree that it's a good idea to use lowercase variable names, 
the way you put it seems a bit too strong. I'd call it a preference, or 
even a matter of style, but convention to me implies something that's 
more widely agreed upon or more widespread, which it's not (unfortunately).


--
Every word is like an unnecessary stain on silence and nothingness.
-- Beckett

Eduardo M KALINOWSKI
edua...@kalinowski.com.br

Re: can you parse and "tail" at once? (and if you can't why not?)

[Greg Wooledge]

On Mon, Oct 23, 2023 at 01:21:43PM +0530, Pankaj Jangid wrote:
> Another way to do it is,
> 
> cat "${IFL}"| cut -d "/" -f7-|sed 's/.description//'
> 
> because you already know the prefix, you can count the fields. So "-f7-"
> i.e. 7 onwards. Then use sed to remove the extension.

Whoops!  I completely missed the requirement of removing ".description"
in my original answer.

Here's a corrected version:


#!/bin/bash

inputfile=${1-/whatever}
prefix='[info] 123XYZ: 
/media/user/1234qwer/Algebraic_Geometry04/20231022040239/'

while IFS= read -r line; do
line=${line#"$prefix"}
printf '%s\n' "${line%.description}"
done < <(grep -F -- "$prefix" "$inputfile")


Pankaj's answer is not wrong, but it dropped the "grep" which filters
the lines containing the desired prefix.  I'd also add an $ anchor to
the sed regex, and escape the . with backslash so that it becomes
literal.  This answer also requires counting the fields by hand each
time the prefix changes, which may or may not be a concern.

Re: can you parse and "tail" at once? (and if you can't why not?)

[Pankaj Jangid]

Albretch Mueller  writes:

>  After generating a file with lines of text, I have to:
>  1) parse some of those lines based on a pattern I know (no regex
> necessary, just a FS path)
>  2) once parsed from those lines I need the last n characters only
>  I am trying to use a one liner like:
>  cat "${IFL}" | grep "${DESC_DIR}" | tail -c+$_DESC_DIR
>  but this one liner repeats the output and the tail
>
>  Say the lines of text in IFL are:
> [info] 123XYZ: 
> /media/user/1234qwer/Algebraic_Geometry04/20231022040239/Uppsala_Algebra/45_16.3_45MEB5h5H9Y.description
> [info] 123XYZ: 
> /media/user/1234qwer/Algebraic_Geometry04/20231022040239/Uppsala_Algebra/46_17.1_LE_gl5RcDnY.description
> [info] 123XYZ: 
> /media/user/1234qwer/Algebraic_Geometry04/20231022040239/Göttsche/17_9jKzwoBGFs8.description
> [info] 123XYZ: 
> /media/user/1234qwer/Algebraic_Geometry04/20231022040239/Göttsche/18_plOUIOo91lI.description
> [info] 123XYZ: 
> /media/user/1234qwer/Algebraic_Geometry04/20231022040239/Göttsche/19_TJ7yAiq8gEw.description
> [info] 123XYZ: 
> /media/user/1234qwer/Algebraic_Geometry04/20231022040239/Göttsche/20_UMBEgb14uqw.description
>
>  I know the prefix:
> [info] 123XYZ: /media/user/1234qwer/Algebraic_Geometry04/20231022040239/
>
>  I need as output:
>
> Uppsala_Algebra/45_16.3_45MEB5h5H9Y
> Uppsala_Algebra/46_17.1_LE_gl5RcDnY
> Göttsche/17_9jKzwoBGFs8
> Göttsche/18_plOUIOo91lI
> Göttsche/19_TJ7yAiq8gEw
> Göttsche/20_UMBEgb14uqw
> ~
>  lbrtchx

Another way to do it is,

cat "${IFL}"| cut -d "/" -f7-|sed 's/.description//'

because you already know the prefix, you can count the fields. So "-f7-"
i.e. 7 onwards. Then use sed to remove the extension.

Re: can you parse and "tail" at once? (and if you can't why not?)

[tomas]

On Sun, Oct 22, 2023 at 10:13:26PM -0400, Greg Wooledge wrote:

[...]

Good points all over -- I envy your clarity.

To drive home Greg's point about naming conventions a bit
more: bash (in general, shells) have very little protection
for mis-naming variables (you don' have to declare them,
they have dynamic, not lexical bindings, etc.).

That's why naming conventions are more important there.

In shells, there's more: shell variables and environment
variables (those you "export") are quite different beasts,
so making them visually differentiable is the more important.

See, a programming language is a communication tool. You
talk to other people with it. Observing conventions is
part of a language.

Cheers
-- 
t


signature.asc
Description: PGP signature

Re: can you parse and "tail" at once? (and if you can't why not?)

[Patrick Wiseman]

On Sun, Oct 22, 2023, 10:39 PM Albretch Mueller  wrote:

>  OK, Greg's suggestion once again "made my day".
>  I know at some point I will have to code everything in some
> programming language, but for now I will just get things done as
> quickly as possible.
>  Also, Greg, please, I would like for you to understand that it is not
> my intention to upset you about what you find visually upsetting. We
> have talked about that before.
>  I am sure you could certainly understand that there are reasons why
> some people build certain habits and that not everyone gets upset
> about the same kinds of "secondary accidents" (as Aristotle would put
> it). Some people may have a hard time even noticing what upsets you.
>  lbrtchx
>
> Context? Totally lost. No need to reply, I haven't been following anyway.

Re: can you parse and "tail" at once? (and if you can't why not?)

[Greg Wooledge]

On Mon, Oct 23, 2023 at 01:32:08AM +, Albretch Mueller wrote:
> On 10/22/23, Andy Smith  wrote:
> > Most of the points Greg makes to you are matters of correctness, not
> > matters of taste.
> 
>  that you called "matters of correctness" may be "visual things" to
> other people. this is something I see on a daily basis I don't try to
> "correct" my students ways, but help them learn, see things for
> themselves.

Was I not sufficiently clear in my explanations?  Sorry.

Your original post contained this line of code:

cat "${IFL}" | grep "${DESC_DIR}" | tail -c+$_DESC_DIR

In the interests of learning, let's analyze this line *in full*.  From
left to right:

1) Useless use of cat.  In this particular example, the result is still
   correct.  The code is simply longer and slower than it needs to be.

2) All-caps variable name IFL.  All-caps variable names are reserved, by
   convention, for environment variables (e.g. PATH) and special shell
   variables (e.g. IFS).  I assume your variable IFL stands for "input
   file".  But note how close it is to IFS.  I can easily imagine that
   at some point in the future, if you continue using all-caps variable
   names inappropriately, you might accidentally use one that collides
   with an environment variable or a special shell variable.

   Accidentally overwriting IFS is going to break your scripts faster
   than you can imagine.

3) grep "${DESC_DIR}".  Another all-caps variable name, but I won't repeat
   that point.  $DESC_DIR is being passed to a command that expects a
   regular expression, but in your example, it's a literal string.
   Therefore you should use grep -F.  Or use a regular expression in
   DESC_DIR, but that wouldn't be my preferred choice.

4) The tail command is not correct here.  This is the primary error.
   The tail needs to be replaced to make the script work.  Tail operates
   on whole files, not on each line of input, regardless of what options
   you use.

5) You used the _DESC_DIR variable and the DESC_DIR variable together in
   the same command.  I can only imagine one of them was a typo for the
   other.

6) The _DESC_DIR variable expansion is not quoted.  Assuming that it was
   meant to be DESC_DIR which contains a string with spaces and glob
   characters in it, it *must* be quoted, or else it gets passed as an
   unguessable number of separate arguments.

Those are the actual issues I spotted.  I omitted a few in my previous
post, because I wasn't trying to be fully pedantic.  I am now.

Beyond that, I pointed out (by inference) that the code you wrote was
too *limited*.  If the errors are corrected, it will print the output
that you asked for.  But it can't be extended easily to do anything *more*
than that.

Assuming that you actually wanted to *do* something with each line's
suffix, it makes more sense to include some sort of loop, in which you
can add arbitrary commands to do whatever task is needed.

That's why I replaced the tail with a "while read" loop instead of a
single self-contained command, such as sed, which could have done the
printing task more concisely.

Finally, one might wonder why I used this form:

while IFS= read -r line; do

done < <(grep ...)

instead of the simpler form:

grep ... |
while IFS= read -r line; do
...
done

This was another conscious choice to offer more flexibility for extending
the script.  In the simpler form, the while loop is executed in a subshell
(because of the pipeline).  When the loop terminates, any changes made in
the subshell are discarded.  This makes it impossible to set persistent
variables inside the loop, and then use them afterward.

The form that I used, with the process substition <(...), runs the loop
in the main shell process, not in a subshell.  With this form, it's
possible to set variables within the loop, and refer to them afterward.

If you have additional questions, feel free to ask them.

Re: can you parse and "tail" at once? (and if you can't why not?)

[Albretch Mueller]

On 10/22/23, Andy Smith  wrote:
> So you might consider telling us what you will do next with the
> suffix of each line.

 Now you have gone into mind reading mode.

On 10/22/23, Andy Smith  wrote:
> Most of the points Greg makes to you are matters of correctness, not
> matters of taste.

 that you called "matters of correctness" may be "visual things" to
other people. this is something I see on a daily basis I don't try to
"correct" my students ways, but help them learn, see things for
themselves.

> I don't know how many times you will have the opportunity to
> continue talking with knowledgable people who try to help you, if
> you dismiss their points as them just being "upset" over aesthetics.

 and I don't understand your adhominen cr@p. For one, Greg understood
right away my problem which I explicitly explained, so from where do
you take the time to keep talking about any of it?

On 10/22/23, gene heskett  wrote:
> Very well said Andy.
> "There are four boxes to be used in defense of liberty:
>   soap, ballot, jury, and ammo. Please use in that order."
> -Ed Howdershelt (Author, 1940)
> If we desire respect for the law, we must first make the law respectable.
>   - Louis D. Brandeis

 Nor do I understand what support from "the international community"
(tm) has to do with some Linux bash script.

 lbrtchx

Re: can you parse and "tail" at once? (and if you can't why not?)

[gene heskett]

On 10/22/23 17:11, Andy Smith wrote:

Hello,

On Sun, Oct 22, 2023 at 08:50:55PM +, Albretch Mueller wrote:

Greg, please, I would like for you to understand that it is not my
intention to upset you about what you find visually upsetting. We
have talked about that before.


Most of the points Greg makes to you are matters of correctness, not
matters of taste.

I don't know how many times you will have the opportunity to
continue talking with knowledgable people who try to help you, if
you dismiss their points as them just being "upset" over aesthetics.


  I am sure you could certainly understand that there are reasons why
some people build certain habits and that not everyone gets upset
about the same kinds of "secondary accidents" (as Aristotle would put
it). Some people may have a hard time even noticing what upsets you.


Most of the time when asking for help from people with a lot more
knowledge than me in a given subject, I tend to assume that if I
find something they say to be oddly picky or unnecessarily strict
then it's most likely because of a gap in my own knowledge, rather
than them trying to make me slaves to their own personal style.

Thanks,
Andy


Very well said Andy.


Cheers, Gene Heskett.
--
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author, 1940)
If we desire respect for the law, we must first make the law respectable.
 - Louis D. Brandeis

Re: can you parse and "tail" at once? (and if you can't why not?)

[Andy Smith]

Hello,

On Sun, Oct 22, 2023 at 08:50:55PM +, Albretch Mueller wrote:
> Greg, please, I would like for you to understand that it is not my
> intention to upset you about what you find visually upsetting. We
> have talked about that before.

Most of the points Greg makes to you are matters of correctness, not
matters of taste.

I don't know how many times you will have the opportunity to
continue talking with knowledgable people who try to help you, if
you dismiss their points as them just being "upset" over aesthetics.

>  I am sure you could certainly understand that there are reasons why
> some people build certain habits and that not everyone gets upset
> about the same kinds of "secondary accidents" (as Aristotle would put
> it). Some people may have a hard time even noticing what upsets you.

Most of the time when asking for help from people with a lot more
knowledge than me in a given subject, I tend to assume that if I
find something they say to be oddly picky or unnecessarily strict
then it's most likely because of a gap in my own knowledge, rather
than them trying to make me slaves to their own personal style.

Thanks,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting

Re: can you parse and "tail" at once? (and if you can't why not?)

[Albretch Mueller]

 OK, Greg's suggestion once again "made my day".
 I know at some point I will have to code everything in some
programming language, but for now I will just get things done as
quickly as possible.
 Also, Greg, please, I would like for you to understand that it is not
my intention to upset you about what you find visually upsetting. We
have talked about that before.
 I am sure you could certainly understand that there are reasons why
some people build certain habits and that not everyone gets upset
about the same kinds of "secondary accidents" (as Aristotle would put
it). Some people may have a hard time even noticing what upsets you.
 lbrtchx

Re: can you parse and "tail" at once? (and if you can't why not?)

[Andy Smith]

Hello,

On Sun, Oct 22, 2023 at 04:33:12PM +, Albretch Mueller wrote:
>  After generating a file with lines of text, I have to:
>  1) parse some of those lines based on a pattern I know (no regex
> necessary, just a FS path)
>  2) once parsed from those lines I need the last n characters only

There's a large number of ways to achieve what you want, but it's
hard to believe that simply getting the suffix is all that you want.
I think it's likely that you'll reject most of the otherwise correct
answers because they don't suit what you're about to do next.

So you might consider telling us what you will do next with the
suffix of each line.

Thanks,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting

Re: can you parse and "tail" at once? (and if you can't why not?)

[Roberto C . Sánchez]

On Sun, Oct 22, 2023 at 04:33:12PM +, Albretch Mueller wrote:
>  After generating a file with lines of text, I have to:
>  1) parse some of those lines based on a pattern I know (no regex
> necessary, just a FS path)
>  2) once parsed from those lines I need the last n characters only
>  I am trying to use a one liner like:
>  cat "${IFL}" | grep "${DESC_DIR}" | tail -c+$_DESC_DIR
>  but this one liner repeats the output and the tail
> 
The way that -c operates for tail is based on 'each file' provided as
input:

   -c, --bytes=[+]NUM
  output the last NUM bytes; or use -c +NUM to output starting with 
byte NUM of
  each file

However, when piping input in the way you are above, the entire output
of grep is taken as one file and fed into the input of tail. You could
instead loop over the lines coming out of grep, but that feels somewhat
ugly.

Essentially, your pipeline probably needs to be something more like
this:

sed -r -e "s+.*${DESC_DIR}++" "${IFL}" | tail

Or drop the tail entirely if what you really want is the last part of
each line, regardless of how many there are.

Regards,

-Roberto

-- 
Roberto C. Sánchez

Re: can you parse and "tail" at once? (and if you can't why not?)

[Greg Wooledge]

On Sun, Oct 22, 2023 at 04:33:12PM +, Albretch Mueller wrote:
>  I am trying to use a one liner like:
>  cat "${IFL}" | grep "${DESC_DIR}" | tail -c+$_DESC_DIR
>  but this one liner repeats the output and the tail

Brevity is not the measure of goodness.  When you write a shell script,
it's *OK* to use more than one line.

Also, please stop using ALL_CAPS variable names for internal script
variables.  ALL_CAPS names are by convention reserved for the environment.
Names with a leading underscore are allowed, but should be used only
when you need something special.  There's nothing special in the code
above to merit a variable name like _DESC_DIR.

Also also, useless use of cat .

Finally, mixing up _DESC_DIR and DESC_DIR is probably not helping you.

>  Say the lines of text in IFL are:
> [info] 123XYZ: 
> /media/user/1234qwer/Algebraic_Geometry04/20231022040239/Uppsala_Algebra/45_16.3_45MEB5h5H9Y.description
> [info] 123XYZ: 
> /media/user/1234qwer/Algebraic_Geometry04/20231022040239/Uppsala_Algebra/46_17.1_LE_gl5RcDnY.description
> [info] 123XYZ: 
> /media/user/1234qwer/Algebraic_Geometry04/20231022040239/Göttsche/17_9jKzwoBGFs8.description
> [info] 123XYZ: 
> /media/user/1234qwer/Algebraic_Geometry04/20231022040239/Göttsche/18_plOUIOo91lI.description
> [info] 123XYZ: 
> /media/user/1234qwer/Algebraic_Geometry04/20231022040239/Göttsche/19_TJ7yAiq8gEw.description
> [info] 123XYZ: 
> /media/user/1234qwer/Algebraic_Geometry04/20231022040239/Göttsche/20_UMBEgb14uqw.description
> 
>  I know the prefix:
> [info] 123XYZ: /media/user/1234qwer/Algebraic_Geometry04/20231022040239/
> 
>  I need as output:
> 
> Uppsala_Algebra/45_16.3_45MEB5h5H9Y
> Uppsala_Algebra/46_17.1_LE_gl5RcDnY
> Göttsche/17_9jKzwoBGFs8
> Göttsche/18_plOUIOo91lI
> Göttsche/19_TJ7yAiq8gEw
> Göttsche/20_UMBEgb14uqw

Yay!  Sample input!  That makes it nice and easy.


#!/bin/bash

inputfile=${1-/whatever}
prefix='[info] 123XYZ: 
/media/user/1234qwer/Algebraic_Geometry04/20231022040239/'

while IFS= read -r line; do
printf '%s\n' "${line#"$prefix"}"
done < <(grep -F -- "$prefix" "$inputfile")


Yes, I could make it shorter.  I could even use that functional approach
where you pipe grep and sed together, which some people think is the only
way to write shell code.  It could actually be done in a single sed
command (but you'd have to convert the prefix string to a regular
expression first).

I chose not to do any of those, for two reasons:

1) To make it easier to read and understand.
2) To make it possible to do more than simply *printing* the suffixes.

If your script wants to do something with those suffixes, more than
simply printing them on the screen, then the script I wrote is a much
better starting point.  You can stick additional commands inside the
loop body to do whatever you need.


On Sun, Oct 22, 2023 at 11:07:46AM -0600, Charles Curley wrote:
> Bash has built in operators to extract this sort of thing. Greg
> Wooledge will know what they are called. Something like:
> 
> fspec="/exp/home1/abc.txt" 
> filename="${fspec##*/}"  # get filename
> dirname="${fspec%/*}" # get directory/path name
> 
> That doesn't do exactly what you want, but comes close.

Those are called Parameter Expansions.  For a beginner-friendly intro,
please see .

Re: can you parse and "tail" at once? (and if you can't why not?)

[Charles Curley]

On Sun, 22 Oct 2023 16:33:12 +
Albretch Mueller  wrote:

> After generating a file with lines of text, I have to:
>  1) parse some of those lines based on a pattern I know (no regex
> necessary, just a FS path)
>  2) once parsed from those lines I need the last n characters only
>  I am trying to use a one liner like:
>  cat "${IFL}" | grep "${DESC_DIR}" | tail -c+$_DESC_DIR
>  but this one liner repeats the output and the tail

Bash has built in operators to extract this sort of thing. Greg
Wooledge will know what they are called. Something like:

fspec="/exp/home1/abc.txt" 
filename="${fspec##*/}"  # get filename
dirname="${fspec%/*}" # get directory/path name

That doesn't do exactly what you want, but comes close.

-- 
Does anybody read signatures any more?

https://charlescurley.com
https://charlescurley.com/blog/

can you parse and "tail" at once? (and if you can't why not?)

[Albretch Mueller]

 After generating a file with lines of text, I have to:
 1) parse some of those lines based on a pattern I know (no regex
necessary, just a FS path)
 2) once parsed from those lines I need the last n characters only
 I am trying to use a one liner like:
 cat "${IFL}" | grep "${DESC_DIR}" | tail -c+$_DESC_DIR
 but this one liner repeats the output and the tail

 Say the lines of text in IFL are:
[info] 123XYZ: 
/media/user/1234qwer/Algebraic_Geometry04/20231022040239/Uppsala_Algebra/45_16.3_45MEB5h5H9Y.description
[info] 123XYZ: 
/media/user/1234qwer/Algebraic_Geometry04/20231022040239/Uppsala_Algebra/46_17.1_LE_gl5RcDnY.description
[info] 123XYZ: 
/media/user/1234qwer/Algebraic_Geometry04/20231022040239/Göttsche/17_9jKzwoBGFs8.description
[info] 123XYZ: 
/media/user/1234qwer/Algebraic_Geometry04/20231022040239/Göttsche/18_plOUIOo91lI.description
[info] 123XYZ: 
/media/user/1234qwer/Algebraic_Geometry04/20231022040239/Göttsche/19_TJ7yAiq8gEw.description
[info] 123XYZ: 
/media/user/1234qwer/Algebraic_Geometry04/20231022040239/Göttsche/20_UMBEgb14uqw.description

 I know the prefix:
[info] 123XYZ: /media/user/1234qwer/Algebraic_Geometry04/20231022040239/

 I need as output:

Uppsala_Algebra/45_16.3_45MEB5h5H9Y
Uppsala_Algebra/46_17.1_LE_gl5RcDnY
Göttsche/17_9jKzwoBGFs8
Göttsche/18_plOUIOo91lI
Göttsche/19_TJ7yAiq8gEw
Göttsche/20_UMBEgb14uqw
~
 lbrtchx

Re: why would ping and traceroute give you different IP addresses?

[Geert Stappers]

On Tue, Aug 15, 2023 at 05:02:49AM +, Albretch Mueller wrote:
> site="download.gluonhq.com"
> date
> time ping "${site}" -c 4
> time traceroute "${site}"
> 
> $ site="download.gluonhq.com"
> date
> time ping "${site}" -c 4
> time traceroute "${site}"
> Mon 14 Aug 2023 11:54:19 PM UTC
> PING s3-website.us-east-1.amazonaws.com (54.231.134.85) 56(84) bytes of data.
> 64 bytes from s3-website-us-east-1.amazonaws.com (54.231.134.85): icmp_seq=1 
> ttl=242 time=47.8 ms
> 64 bytes from s3-website-us-east-1.amazonaws.com (54.231.134.85): icmp_seq=2 
> ttl=242 time=54.9 ms
> 64 bytes from s3-website-us-east-1.amazonaws.com (54.231.134.85): icmp_seq=3 
> ttl=242 time=46.2 ms
> 64 bytes from s3-website-us-east-1.amazonaws.com (54.231.134.85): icmp_seq=4 
> ttl=242 time=48.1 ms
> 
> --- s3-website.us-east-1.amazonaws.com ping statistics ---
> 4 packets transmitted, 4 received, 0% packet loss, time 3004ms
> rtt min/avg/max/mdev = 46.246/49.253/54.875/3.320 ms
> 
> real  0m3.186s
> user  0m0.005s
> sys   0m0.000s
> 
> traceroute to download.gluonhq.com (52.216.228.138), 30 hops max, 60
> byte packets
>  1  _gateway (192.168.68.1)  9.267 ms  9.223 ms  8.035 ms
>  2  dsldevice.attlocal.net (192.168.1.254)  10.349 ms  10.333 ms  10.319 ms
>  3  99-178-252-1.lightspeed.stlsmo.sbcglobal.net (99.178.252.1) 33.434 ms  
> 33.420 ms  33.406 ms
>  4  71.154.70.113 (71.154.70.113)  34.515 ms  33.379 ms  38.092 ms 5  * * *
>  6  32.130.88.131 (32.130.88.131)  50.165 ms  42.200 ms  35.492 ms
>  7  * * *
   
> 22  * * *
> 23  s3-website-us-east-1.amazonaws.com (52.216.228.138)  50.960 ms
> 50.902 ms  53.215 ms
> 
> real  0m15.213s
> user  0m0.009s
> sys   0m0.000s
> $
> 

$ host download.gluonhq.com
download.gluonhq.com is an alias for 
download.gluonhq.com.s3-website-us-east-1.amazonaws.com.
download.gluonhq.com.s3-website-us-east-1.amazonaws.com is an alias for 
s3-website.us-east-1.amazonaws.com.
s3-website.us-east-1.amazonaws.com has address 52.217.163.181
s3-website.us-east-1.amazonaws.com has address 52.217.83.203
s3-website.us-east-1.amazonaws.com has address 52.217.206.181
s3-website.us-east-1.amazonaws.com has address 52.217.98.51
s3-website.us-east-1.amazonaws.com has address 16.182.72.93
s3-website.us-east-1.amazonaws.com has address 52.216.227.162
s3-website.us-east-1.amazonaws.com has address 52.216.212.237
s3-website.us-east-1.amazonaws.com has address 52.217.83.179
$ host download.gluonhq.com
download.gluonhq.com is an alias for 
download.gluonhq.com.s3-website-us-east-1.amazonaws.com.
download.gluonhq.com.s3-website-us-east-1.amazonaws.com is an alias for 
s3-website.us-east-1.amazonaws.com.
s3-website.us-east-1.amazonaws.com has address 52.217.163.181
s3-website.us-east-1.amazonaws.com has address 52.217.83.203
s3-website.us-east-1.amazonaws.com has address 52.217.206.181
s3-website.us-east-1.amazonaws.com has address 52.217.98.51
s3-website.us-east-1.amazonaws.com has address 16.182.72.93
s3-website.us-east-1.amazonaws.com has address 52.216.227.162
s3-website.us-east-1.amazonaws.com has address 52.216.212.237
s3-website.us-east-1.amazonaws.com has address 52.217.83.179
$
 

Groeten
Geert Stappers
-- 
Silence is hard to parse

why would ping and traceroute give you different IP addresses?

[Albretch Mueller]

site="download.gluonhq.com"
date
time ping "${site}" -c 4
time traceroute "${site}"

$ site="download.gluonhq.com"
date
time ping "${site}" -c 4
time traceroute "${site}"
Mon 14 Aug 2023 11:54:19 PM UTC
PING s3-website.us-east-1.amazonaws.com (54.231.134.85) 56(84) bytes of data.
64 bytes from s3-website-us-east-1.amazonaws.com (54.231.134.85):
icmp_seq=1 ttl=242 time=47.8 ms
64 bytes from s3-website-us-east-1.amazonaws.com (54.231.134.85):
icmp_seq=2 ttl=242 time=54.9 ms
64 bytes from s3-website-us-east-1.amazonaws.com (54.231.134.85):
icmp_seq=3 ttl=242 time=46.2 ms
64 bytes from s3-website-us-east-1.amazonaws.com (54.231.134.85):
icmp_seq=4 ttl=242 time=48.1 ms

--- s3-website.us-east-1.amazonaws.com ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3004ms
rtt min/avg/max/mdev = 46.246/49.253/54.875/3.320 ms

real0m3.186s
user0m0.005s
sys 0m0.000s

traceroute to download.gluonhq.com (52.216.228.138), 30 hops max, 60
byte packets
 1  _gateway (192.168.68.1)  9.267 ms  9.223 ms  8.035 ms
 2  dsldevice.attlocal.net (192.168.1.254)  10.349 ms  10.333 ms  10.319 ms
 3  99-178-252-1.lightspeed.stlsmo.sbcglobal.net (99.178.252.1)
33.434 ms  33.420 ms  33.406 ms
 4  71.154.70.113 (71.154.70.113)  34.515 ms  33.379 ms  38.092 ms
 5  * * *
 6  32.130.88.131 (32.130.88.131)  50.165 ms  42.200 ms  35.492 ms
 7  * * *
 8  * * *
 9  * * *
10  * * *
11  * * *
12  * * *
13  * * *
14  * * *
15  * * *
16  * * *
17  * * *
18  * * *
19  * * *
20  * * *
21  * * *
22  * * *
23  s3-website-us-east-1.amazonaws.com (52.216.228.138)  50.960 ms
50.902 ms  53.215 ms

real0m15.213s
user0m0.009s
sys 0m0.000s
$

Re: Why or why not back up "/lost+found"

[Default User]

On Fri, 2023-08-11 at 14:45 +0200, to...@tuxteam.de wrote:
> On Fri, Aug 11, 2023 at 01:30:13PM +0100,
> debian-u...@howorth.org.uk wrote:
> 
> [...]
> 
> > One other consideration that I haven't seen mentioned elsewhere in
> > this
> > thread is what happens if you back up filesystems to filesystems?
> 
> I never use the backup's medium top-level dir as a target. In my
> current
> usage, that's how my medium looks like:
> 
>   tomas@trotzki:~$ ls -l /media/backup
>   total 20
>   drwx-- 2 root root 16384 Aug 27  2022 lost+found
>   drwxr-xr-x 4 root root  4096 Aug 27  2022 trotzki
> 
> ...so there's a top-level dir named after the "client" host. My
> backup
> scripts gets a positive list of things to back up -- and I sprinkle
> files named .backup-filter for more control (I tell rsync about
> that).
> 
> Simple, effective. Putting tons of "--exclude" and things into the
> command line will drive you crazy after a while :)
> 
> Cheers


Hi! 

Just a clarification, FWIW. 

I earlier stated that I back up to:
/media/[user]/MSD1

I actually back up to:
/media/[user]/MSD1/rsnapshot_backups_of_[host]
which is of course a sub-directory of /media/[user]/MSD1. 

Anyway, from now on, I am going to explicitly exclude all "lost+found"
directories from being backed up. 

Thank you to all who weighed in on this!

Re: Why or why not back up "/lost+found"

[tomas]

On Fri, Aug 11, 2023 at 01:30:13PM +0100, debian-u...@howorth.org.uk wrote:

[...]

> One other consideration that I haven't seen mentioned elsewhere in this
> thread is what happens if you back up filesystems to filesystems?

I never use the backup's medium top-level dir as a target. In my current
usage, that's how my medium looks like:

  tomas@trotzki:~$ ls -l /media/backup
  total 20
  drwx-- 2 root root 16384 Aug 27  2022 lost+found
  drwxr-xr-x 4 root root  4096 Aug 27  2022 trotzki

...so there's a top-level dir named after the "client" host. My backup
scripts gets a positive list of things to back up -- and I sprinkle
files named .backup-filter for more control (I tell rsync about that).

Simple, effective. Putting tons of "--exclude" and things into the
command line will drive you crazy after a while :)

Cheers
-- 
t


signature.asc
Description: PGP signature

Re: Why or why not back up "/lost+found"

[debian-user]

 wrote:
> On Thu, Aug 10, 2023 at 03:42:01PM -0400, Default User wrote:
> > Hi!
> > 
> > When backing up my system I have been using this exclusions list: 
> > 
> > /dev/*
> > /proc/*
> > /sys/*
> > /tmp/*
> > /run/*
> > /mnt/*
> > /media/*
> > /lost+found
> > 
> > There are many sources online that suggest that "/lost+found"
> > should be excluded from backups, but I can't seem to find a good
> > explanation for why.  
> 
> Squinting the other way (which doesn't contradict what others have
> said in this thread): if you see anything in lost+found, the idea
> is that you do something about it (rescue it, throw it away). Thus,
> the content of lost+found is (or should be) as temporary as, say,
> /tmp. And you don't back up that.
> 
> So I'd tend to exclude the special "lost+found" dirs (perhaps not
> any dir called like that). But then, it's not that important.
> 
> Cheers

One other consideration that I haven't seen mentioned elsewhere in this
thread is what happens if you back up filesystems to filesystems? That
is, if the backup for /home is an ext-? filesystem it may well already
have a /lost+found directory. That directory is hopefully empty but if
not then it contains data related to errors on the backup system, so
you do not want to overwrite it! I suspect that may be why it is
conventionally excluded from backups. It's normally empty so there's no
point in copying it. If there's anything in it on the original or the
backup, then that's a more urgent problem than a backup.

FWIW, I just don't use ext-? filesystems so I don't have to worry.

Re: Why or why not back up "/lost+found"

[tomas]

On Thu, Aug 10, 2023 at 03:42:01PM -0400, Default User wrote:
> Hi!
> 
> When backing up my system I have been using this exclusions list: 
> 
> /dev/*
> /proc/*
> /sys/*
> /tmp/*
> /run/*
> /mnt/*
> /media/*
> /lost+found
> 
> There are many sources online that suggest that "/lost+found" should be
> excluded from backups, but I can't seem to find a good explanation for
> why.

Squinting the other way (which doesn't contradict what others have
said in this thread): if you see anything in lost+found, the idea
is that you do something about it (rescue it, throw it away). Thus,
the content of lost+found is (or should be) as temporary as, say,
/tmp. And you don't back up that.

So I'd tend to exclude the special "lost+found" dirs (perhaps not
any dir called like that). But then, it's not that important.

Cheers
-- 
t


signature.asc
Description: PGP signature

Re: Why or why not back up "/lost+found"

[Greg Wooledge]

On Thu, Aug 10, 2023 at 09:15:31PM -0400, Default User wrote:
> Fortunately, since my lost+found  directories are all empty, I have no
> "raw material" to practice extraction on.

There's no "extraction".  If a file is there, it will be a file.  It
will have a meaningful owner, group and permissions.  It will have a
meaningless name.  You will have to guess what it is based on the
owner, group, permissions, timestamps, and contents.

> Now at this point, it seems that I could:
> 
> 1)  Continue to back up /home/lost+found and /var/lost+found, but NOT 
> back up /lost+found.
> 
> 2)  Start backing up /home/lost+found AND /var/lost+found AND
> /lost+found. 
> 
> 3)  Start NOT backing up /home/lost+found OR /var/lost+found OR
> /lost+found.
> 
> What is everyone else doing about backing up, or not backing up, the
> various lost+found directories on their systems? 

You are asking the wrong question.

If you find a file in lost+found, SOMETHING BAD HAS HAPPENED.  You
will want to figure out how bad the damage is, and decide what to do
about it.

If the damage is extremely small, then you might be able to figure
out where the files go, and move them.

If the damage is large, you may need to replace the disk and restore
your last working backup.  Which will not have anything in lost+found.

Backing up lost+found is pretty meaningless, because if you're backing
up a lost+found with files in it, it means you're backing up a damaged
file system.

What you should be asking instead is whether your backup should ALERT
you if it finds stuff in lost+found.  Having it quietly back up the
files is not as helpful as you might be imagining.

Re: Why or why not back up "/lost+found"

[Default User]

On Thu, 2023-08-10 at 22:03 +, Minecraftchest1 wrote:
> This post on the U Stack Exchange sitr summs it up fairly well.
> https://unix.stackexchange.com/a/18157.
> 
> In short, `lost+found` is a place for fscheck to link filesystem
> entries that don't have an entry anywhere on the file system, during
> a filesystem check. Chances are, the files there were bring deleted,
> but weren't fully removed from the file system due to an open handle
> on 
> the file. If you aren't specifically looking for a file that got lost
> due to a system crash, the files there likely aren't relavent
> and can be safely ignored.
> 
> 
> On August 10, 2023 9:46:27 PM UTC, Default User
>  wrote:
> > On Thu, 2023-08-10 at 21:45 +0200, Nicolas George wrote:
> > > Default User (12023-08-10):
> > > > And, if /lost+found should be excluded, then shouldn't
> > > > "lost+found"
> > > > in
> > > > any other directories be excluded from backups as well? Why/why
> > > > not? 
> > > > 
> > > 
> > > Before anybody answers all your question, there is something that
> > > needs
> > > checking:
> > > 
> > > Do you know what lost+found is for?
> > > 
> > > If not, please read some documentation about it. The explanation
> > > of
> > > its
> > > function contains the answer to the question I left quoted above.
> > > 
> > > Regards,
> > > 
> > > 
> > 
> > 
> > Hi, Nicholas.
> > 
> > Well, I did read some documentation, and I have at least a
> > rudimentary
> > understanding of fsck and lost+found. 
> > 
> > Unfortunately, I regret to say that I did not find that the answer
> > to
> > the question(s) about lost+found in the original post were
> > contained in
> > the explanation(s) of its function - at least in what I have read
> > so
> > far. 
> > 
> > Or, maybe I'm just stupid. 
> > 
> > I'm sorry if I have bothered you. 
> > 


Three of my mount points, each of which is on a separate partition,
have lost+found directories:
/lost+found
/home/lost+found
/var/lost+found

All of these appear to be empty. 

Of these, I have rsnapshot configured to exclude:
/lost+found

but the other two:
/home/lost+found
/var/lost+found

ARE currently being backed up.

The backups are to an external usb hard drive, mounted as:
/media/[user]/MSD1
which has its own lost+found:
/media/[user]/MSD1/lost+found

which also appears to be empty. 

I have read a number of things online about fsck and lost+found,
including the previously mentioned:
https://unix.stackexchange.com/a/18157. 

I think I understand the basic concept of how fsck puts stray files and
file parts in lost+found, and (in theory) how to manually extract whole
and partial files from lost+found. 

Fortunately, since my lost+found  directories are all empty, I have no
"raw material" to practice extraction on.

Now at this point, it seems that I could:

1)  Continue to back up /home/lost+found and /var/lost+found, but NOT 
back up /lost+found.

2)  Start backing up /home/lost+found AND /var/lost+found AND
/lost+found. 

3)  Start NOT backing up /home/lost+found OR /var/lost+found OR
/lost+found.

What is everyone else doing about backing up, or not backing up, the
various lost+found directories on their systems? 

Bonus points for why you do it that way!

:)

Re: Why or why not back up "/lost+found"

[David Wright]

On Thu 10 Aug 2023 at 15:52:02 (-0700), Bob McGowan wrote:
> On 8/10/23 03:03 PM, Nicolas George wrote:
> > Default User (12023-08-10):
> > > > > And, if /lost+found should be excluded, then shouldn't "lost+found"
> > > > > in any other directories be excluded from backups as well? Why/why
> > > > > not?
> > > Unfortunately, I regret to say that I did not find that the answer to
> > > the question(s) about lost+found in the original post were contained in
> > > the explanation(s) of its function - at least in what I have read so
> > > far.
> > The lost+found directory at the root of the filesystem is special, it is
> > created when the filesystem is created and set up to receive orphaned
> > files. A lost+found directory elsewhere… is just a directory with a
> > wacky name.
> > 
> Almost but not quite completely true.
> 
> If you have more than one filesystem on your host, they may also have
> lost+found directories which will show up in their mount points.
> 
> For example, with your home on a separate disk, mounted on /home,
> there may be a /home/lost+found.
> 
> The same thinking applies to them as to /lost+found.

But /home/lost+found is still called /lost+found on the filesystem
it belongs to; it's only now called /home/lost+found because you
mounted that filesystem on /home. That's why Nicolas wrote "at the
root of the filesystem"; that's any filesystem (that uses the concept).

The OP should also note that you musn't use mkdir to create such
directories: mklost+found should be used instead. Typically it
pre-allocates space so that fsck doesn't have to disturb the rest
of the filesystem when it runs.

Cheers,
David.

Re: Why or why not back up "/lost+found"

[Bob McGowan]

On 8/10/23 03:03 PM, Nicolas George wrote:

Default User (12023-08-10):

And, if /lost+found should be excluded, then shouldn't "lost+found"
in any other directories be excluded from backups as well? Why/why
not?

Unfortunately, I regret to say that I did not find that the answer to
the question(s) about lost+found in the original post were contained in
the explanation(s) of its function - at least in what I have read so
far.

The lost+found directory at the root of the filesystem is special, it is
created when the filesystem is created and set up to receive orphaned
files. A lost+found directory elsewhere… is just a directory with a
wacky name.

Regards,


Almost but not quite completely true.

If you have more than one filesystem on your host, they may also have 
lost+found directories which will show up in their mount points.


For example, with your home on a separate disk, mounted on /home, there 
may be a /home/lost+found.


The same thinking applies to them as to /lost+found.

Any other location, then it is a user creating a directory or file with 
a wacky name, as Nicolas suggests.


Bob

Re: Why or why not back up "/lost+found"

[Greg Wooledge]

On Fri, Aug 11, 2023 at 12:03:26AM +0200, Nicolas George wrote:
> Default User (12023-08-10):
> > > > And, if /lost+found should be excluded, then shouldn't "lost+found"
> > > > in any other directories be excluded from backups as well? Why/why
> > > > not? 
> 
> > Unfortunately, I regret to say that I did not find that the answer to
> > the question(s) about lost+found in the original post were contained in
> > the explanation(s) of its function - at least in what I have read so
> > far. 
> 
> The lost+found directory at the root of the filesystem is special, it is
> created when the filesystem is created and set up to receive orphaned
> files. A lost+found directory elsewhere… is just a directory with a
> wacky name.

Expanding on this: the only time there are ever going to be files in
the lost+found directory is if your file system has suffered some
damage/corruption, and fsck has found some files that are not
referenced by any directory.  Fsck puts those files in lost+found with
a procedurally generated name, since fsck doesn't have any idea what
the file's original names were.

If you ever see files in lost+found, then you know that this has
occurred.  Furthermore, it is on *you* to try to figure out what each
file is, where it came from, and what it should be called.  This will
be a 100% manual process.