sudo+use_pty+urxvt+pipe [WAS: apt-get aborts in subshell with redirections on Debian 12]

[John Crawley]

On 26/09/2023 11:52, John Crawley wrote:

On 25/09/2023 20:21, Greg Wooledge wrote:

On Mon, Sep 25, 2023 at 11:14:24AM +0200, Michael wrote:

so i looked into /etc/sudoers and all /etc/sudoers.d/* and found two
suspicous flags:

/etc/sudoers:
Defaults   use_pty

/etc/sudoers.d/0pwfeedback:
Defaults pwfeedback

then consulting the sudo manpage convinced me, it was the 'use_pty' flag (in
section SUDOERS OPTIONS). after removing that flag everything works as
'expected':


Well, that is quite the find. 


Indeed! Many thanks Michael.

8< snip >8

Anyway, for my personal purposes, this new behaviour looks too unpredictable 
and I plan to fall back on capturing stderr in a tempfile. Kludgy but maybe 
more robust than multiple redirections?

:~$ temp=$(mktemp)
:~$ sudo apt-get install nopkg 2>"$temp"
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
:~$ errors=$(<"$temp")
:~$ echo "$errors"
E: Unable to locate package nopkg
:~$ rm "$temp"

It would be nice to unpick the rest of the mystery though...



Revision:
apt commands where STDERR was captured in a variable were (still are) causing the apt 
prompt to be pre-empted to "No", eg:

(replace mirage with any package which is not installed, and needs to pull in 
some dependencies, so will call a prompt to confirm)

john@bookworm-tmp:~$ errors=$(sudo apt-get install mirage 2>&1 1>/dev/tty)
[sudo] password for john:
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  gir1.2-gexiv2-0.10 libexiv2-27 libgexiv2-2
Suggested packages:
  exiv2 gimp
The following NEW packages will be installed:
  gir1.2-gexiv2-0.10 libexiv2-27 libgexiv2-2 mirage
0 upgraded, 4 newly installed, 0 to remove and 12 not upgraded.
Need to get 0 B/1,735 kB of archives.
After this operation, 8,373 kB of additional disk space will be used.
Do you want to continue? [Y/n] Abort.

That was because of the new sudo default use_pty, and there seemed to be no 
workaround, so I switched to sending apt's STDERR to a temporary file instead. 
Life could go on.

Background:
The function which ran apt was part of a long script which logged all its 
output to a file for later possible bugfixing, using this line near the 
beginning:

exec > >( tee -ia "$logfile" ) 2>&1

Once the apt functions no longer used the previous subshell capturing then 
everything worked fine, until recently, when the aborts reappeared, but only 
under very specific circumstances, all these:

*) use_pty is set in /etc/sudoers (now default and desirable for security)
*) the apt command is run in a new terminal launched with  -e bash -c 
''
*) the terminal is urxvt (I tested xterm, gnome-terminal, lxterminal and 
terminator - none aborted)
*) the apt command output is either piped (apt install mirage | cat) or 
redirected to a process substitution (as in the above exec).

Redirecting STDERR to a file (sudo apt-get install mirage 2>"$tempfile") is 
fine,
and even redirecting STDOUT to a file works (typing blind!).
Redirecting to a command triggers the Abort.

Examples of commands which will cause apt to abort:
urxvt -e bash -c "sudo apt-get install mirage >  >( tee -ia \"$logfile\" )"
urxvt -e bash -c 'sudo apt-get install mirage | cat'
urxvt -e bash -c 'sudo apt-get install mirage | tee /dev/null'
urxvt -e bash -c 'sudo apt-get install mirage >  >( cat )'
urxvt -e bash -c 'sudo apt-get install mirage 2>  >( cat )'

Examples where the Abort is NOT triggered:
urxvt -e bash -c "sudo apt-get install mirage >$filename"
urxvt -e bash -c "sudo apt-get install mirage 2>$filename"
urxvt -e bash -c 'sudo apt-get install mirage 3>  >( cat )'
urxvt -e bash -c "sudo apt-get install mirage 3>  >( tee -ia \"$logfile\" )"
any command where the terminal is not urxvt
any command run directly in the terminal, not in a new one with -e

In fact, to see what's happening in the terminal before it closes, you should 
append a 'read' command:
urxvt -e bash -c "sudo apt-get install mirage >  >( tee -ia \"$logfile\" ); echo 
'Press any key to continue'; read -srn1"

So, is this a urxvt bug, or is it doing the Right Thing here when in a 
pseudo-terminal and all the others are not?

More practically, is there now any way to record a script's STDOUT to a 
logfile, while also showing it to the user, while urxvt is set as 
x-terminal-emulator, if that script is going to run apt commands with sudo?

--
John

Re: "sudo apt-get install android-tools-adb" ... (then no device listed)

[Anssi Saari]

David Wright  writes:

> You'd have to specify a set of criteria to test. I just treat
> /media/samsungd like any other filesystem, copying files in the
> usual manner.

Well, when I last tried MTP in Linux I got maybe half of a directory
listing and then it hung there. Concluded it doesn't work but to be sure
it's been a minute.

Re: "sudo apt-get install android-tools-adb" ... (then no device listed)

[David Wright]

On Mon 25 Sep 2023 at 21:08:34 (+0300), Anssi Saari wrote:
> David Wright  writes:
> > On Sun 24 Sep 2023 at 22:13:20 (+), Albretch Mueller wrote:
> >> On 9/24/23, Marco M.  wrote:
> >> > On most Android phones, you need to explicit allow data transfers.
> >> 
> >>  What do you functionally mean? I need for you to talk to me like
> >> this:  a) go "Settings"; b) ...
> >
> > On bullseye I have android-file-transfer installed. I connect the
> > phone to the PC with USB, and run this function:
> >
> >   samsungd () 
> >   { 
> >   sudo mkdir -p /media/samsungd || true;
> >   sudo chown "$USER" /media/samsungd;
> >   aft-mtp-mount /media/samsungd
> >   }
> 
> Does this android-file-transfer provide a working and reliable MTP
> implementation for Linux then? I think I'll stick with rsync and adb but
> would be good to know if MTP is working on Linux these days.

You'd have to specify a set of criteria to test. I just treat
/media/samsungd like any other filesystem, copying files in the
usual manner. As mentioned in:

  https://en.wikipedia.org/wiki/Media_Transfer_Protocol

the phone screens are operable while connected (though I haven't
tested making calls), file modification timestamps are only preserved
phone→PC, not the other way, and you don't have to worry about pulling
the plug before unmounting.

In contrast, older phones that use MSC are inoperable as the screen is
locked, with a warning not to physically disconnect until a button has
been tapped.

Cheers,
David.

Re: "sudo apt-get install android-tools-adb" ... (then no device listed)

[David Wright]

On Mon 25 Sep 2023 at 17:41:13 (+), Albretch Mueller wrote:
> On 9/24/23, Michel Verdier  wrote:
> > If you use USB you need a cable allowing data, some allow only power.
> 
>  The  USB cable I have been using to charge the battery of that phone
> visually seems to be the same exact one being advertised as doubling
> as a data cable, but running:
>  $ sudo lsusb
>  Before and after plugging in the phone doesn’t show any difference.
>  Is there a way to test for sure that cable is the right one?

You should see lines appear in the kern.log (or wherever you have
configured your logs) like:

  kernel: usb 2-2: new high-speed USB device number 15 using xhci_hcd
  kernel: usb 2-2: New USB device found, idVendor=04e8, idProduct=6860, 
bcdDevice= 4.04
  kernel: usb 2-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
  kernel: usb 2-2: Product: SAMSUNG_Android
  kernel: usb 2-2: Manufacturer: SAMSUNG
  kernel: usb 2-2: SerialNumber: ABCDEF123GH
  kernel: cdc_acm 2-2:1.1: ttyACM0: USB ACM device

when you connect it and

  kernel: usb 2-2: USB disconnect, device number 15

when you disconnect.

On Mon 25 Sep 2023 at 19:21:53 (+), Albretch Mueller wrote:
> On 9/25/23, Albretch Mueller  wrote:
> >  Most probably there is a setting in that phone I haven’t been able to
> > find.

That's possible. I tried to think back to when I first connected up
the phone and, searching diligently, I recalled that when the phone
is connected (data cable, obviously), swiping down the Notifications
screen shows an entry "USB for file transfer". Tapping that changes
it to:

  Android System
  USB for file transfer
  Tap for other USB options

Tapping the last gives another screen with "radio" options to:

  Control USB by This/Connected device
  Use USB for FileTransfer/Tethering/MIDI/ImageTransfer/ChargingOnly

plus a "slider" to transcode exported video. I selected both the
first options. You might have a phone still set to charging only.
On this phone, the selected options stay as selected, though I have
to change the second set when I tether (only done that in the UK).

>  Android awakens when I unplug the cable from the computer; so,
> something is being somehow detected.

Typically that would only indicate that the phone was switched on
and being charged. It could be a data cable, or a power-only one,
like those ones with a bunch of different connectors on the end.

On Mon 25 Sep 2023 at 17:41:13 (+), Albretch Mueller wrote:
> On 9/25/23, Marco  wrote:
> > According to the Google documentation:
> 
>  which I found:
> 
> // __ Transfer files between your computer and Android device
> 
>  https://support.google.com/android/answer/9064445?hl=en-GB
> ~
>  doesn’t really explain what to do. They apparently want for you to
> transfer your files to google drive and all that non sense.

That's not the way I read Option 2: Move files with a USB cable.
My method is very similar to option 2's Mac computer:

  1 Download and install Android File Transfer on your computer.
  4 With a USB cable, connect your device to your computer.

Just the same.

  5 On your device, tap the 'Charging this device via USB' notification.
  6 Under 'Use USB for', select File transfer.

As outlined above.

  2 Open Android File Transfer. The next time that you connect your device, it 
opens automatically.

That's when I run my samsungd command.

  3 Unlock your device.

That's when I tap "Allow". (Using the term "unlock" is confusing to me.)

  7 An Android File Transfer window will open on your computer. Use it to drag 
files.

Perhaps linux DEs do that too, IDK. I just use mc, cp, or anything
else you'd use for a mounted filesystem. (And tools like find,
du and df etc.)

Before their final step, I unmount (with fusermount -u) the
filesystem, but that step is a convenience for the PC side,
taking listings, and removing the mount point.

  8 When you've finished, unplug the USB cable.

This is safe to do at any time.

I don't know anything about clouds etc, and have only used Google
on the phone (AFAIK) for regular searches, maps, and navigation.
I archive my photos, along with those from all the other cameras
of mine, on spinning rust.

>  I also tried mtp fs utility, but I am getting the error message:
> detect failed: no MTP devices found
> 
>  go-mtpfs -android  ""
>  2023/09/25 12:10:10 detect failed: no MTP devices found
> ~
>  I also tried jmtpfs:
> 
> $ which jmtpfs
> /usr/bin/jmtpfs
> 
> $ jmtpfs --version
> jmtpfs version: 0.5
> FUSE library version: 2.9.9
> fusermount version: 2.9.9
> using FUSE kernel interface version 7.19
> 
> $ sudo jmtpfs
> No mtp devices found.

I've no experience with using them, and don't know the pros and cons,
not having had the need. They're mentioned in:

  https://wiki.archlinux.org/title/Media_Transfer_Protocol

Cheers,
David.

Re: apt-get aborts in subshell with redirections on Debian 12

[John Crawley]

On 26/09/2023 12:06, Greg Wooledge wrote:

On Tue, Sep 26, 2023 at 11:52:09AM +0900, John Crawley wrote:

On 25/09/2023 20:21, Greg Wooledge wrote:

Given the presence of an /etc/sudoers.dpkg-dist file on my system,
which does in fact contain this:

# This fixes CVE-2005-4890 and possibly breaks some versions of kdesu
# (#1011624, https://bugs.kde.org/show_bug.cgi?id=452532)
Defaultsuse_pty

it looks like you've solved *most* of the mystery.


But @Greg, if you have "Defaultsuse_pty" why haven't you been getting 
the same instant aborts that I was?


Because it's NOT in /etc/sudoers.  It's in /etc/sudoers.dpkg-dist which
is what WOULD have been installed if I had chosen to overwrite my
existing conffile with the new one.  But I didn't.



Ah OK, I saw  /etc/sudoers.d and carelessly thought of the directory.

--
John

Re: apt-get aborts in subshell with redirections on Debian 12

[Greg Wooledge]

On Tue, Sep 26, 2023 at 11:52:09AM +0900, John Crawley wrote:
> On 25/09/2023 20:21, Greg Wooledge wrote:
> > Given the presence of an /etc/sudoers.dpkg-dist file on my system,
> > which does in fact contain this:
> > 
> > # This fixes CVE-2005-4890 and possibly breaks some versions of kdesu
> > # (#1011624, https://bugs.kde.org/show_bug.cgi?id=452532)
> > Defaultsuse_pty
> > 
> > it looks like you've solved *most* of the mystery.
> 
> But @Greg, if you have "Defaultsuse_pty" why haven't you been getting 
> the same instant aborts that I was?

Because it's NOT in /etc/sudoers.  It's in /etc/sudoers.dpkg-dist which
is what WOULD have been installed if I had chosen to overwrite my
existing conffile with the new one.  But I didn't.

Re: apt-get aborts in subshell with redirections on Debian 12

[John Crawley]

On 25/09/2023 20:21, Greg Wooledge wrote:

On Mon, Sep 25, 2023 at 11:14:24AM +0200, Michael wrote:

so i looked into /etc/sudoers and all /etc/sudoers.d/* and found two
suspicous flags:

/etc/sudoers:
Defaults   use_pty

/etc/sudoers.d/0pwfeedback:
Defaults pwfeedback

then consulting the sudo manpage convinced me, it was the 'use_pty' flag (in
section SUDOERS OPTIONS). after removing that flag everything works as
'expected':


Well, that is quite the find.  


Indeed! Many thanks Michael.

For background, I followed up some links to get:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=657784
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1011624
https://bugs.kde.org/show_bug.cgi?id=452532#c28
https://salsa.debian.org/sudo-team/sudo/-/commit/59db341d46aa4c26b54c1270e69f2562e7f3d751


Given the presence of an /etc/sudoers.dpkg-dist file on my system,
which does in fact contain this:

# This fixes CVE-2005-4890 and possibly breaks some versions of kdesu
# (#1011624, https://bugs.kde.org/show_bug.cgi?id=452532)
Defaultsuse_pty

it looks like you've solved *most* of the mystery.  


But @Greg, if you have "Defaultsuse_pty" why haven't you been getting 
the same instant aborts that I was?


I still don't get
why it matters whether the command was executed from bash or dash --


...or ksh or zsh?? Only bash seemed to suffer from having stdin cut off.

Anyway, for my personal purposes, this new behaviour looks too unpredictable 
and I plan to fall back on capturing stderr in a tempfile. Kludgy but maybe 
more robust than multiple redirections?

:~$ temp=$(mktemp)
:~$ sudo apt-get install nopkg 2>"$temp"
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
:~$ errors=$(<"$temp")
:~$ echo "$errors"
E: Unable to locate package nopkg
:~$ rm "$temp"

It would be nice to unpick the rest of the mystery though...

(Also, if some day bash had a way of making a variable look like a file for 
writing.)

--
John

Re: "sudo apt-get install android-tools-adb" ... (then no device listed)

[Jeffrey Walton]

On Mon, Sep 25, 2023 at 10:13 PM Jeffrey Walton  wrote:
>
> On Mon, Sep 25, 2023 at 7:05 PM Albretch Mueller  wrote:
> >
> > On 9/24/23, Michel Verdier  wrote:
> > > If you use USB you need a cable allowing data, some allow only power.
> >
> >  The  USB cable I have been using to charge the battery of that phone
> > visually seems to be the same exact one being advertised as doubling
> > as a data cable, but running:
> >  $ sudo lsusb
> >  Before and after plugging in the phone doesn’t show any difference.
> >  Is there a way to test for sure that cable is the right one?
>
> It sounds like you are using a charging cable, or a cheap data cable.
> I would try a quality data cable before going down the rabbit hole.
>
> You should also state which USB protocol your phone uses. Different
> USB standards have different requirements. For example, USB 1.0 and
> 2.0 only need a 4-wire data cable (PWR, GND, D+ and D-). USB-C needs a
> 24-pin data cable and must implement the USB-PD protocol.
>
> Also see .

And one other thing I've found, but I have no explanation for...

Sometimes you cannot use an intermediate USB hub. I have a Dell XPS
8930 and Pixel 4a. I also bought a USB 3.0 Hub with power ports for
charging from my desktop,
.

When I want to use adb, I have to plug the phone directly into the
computer. When the phone is directly plugged into the computer, 'adb
install ' works as expected. I can also transfer pictures.

When plugged into the hub, I can only install small apk's. If the apk
is too large, then I get an error when running 'adb install '.
You may experience the same or a similar problem with large file
transfers.

Like I said, I have no explanation for it. I just know it happens.
When I experience it, I plug the phone directly into the computer.

Jeff

Re: apt-get aborts in subshell with redirections on Debian 12

[John Crawley]

Many thanks to Michael for finding the change in sudo behaviour!

For historical accuracy:
On 25/09/2023 20:24, Greg Wooledge wrote:

On Mon, Sep 25, 2023 at 01:35:38PM +0900, John Crawley wrote:

4) In a bash shell as root (e.g. "su" or "sudo -s"), do:

 errors=$(apt-get install mirage 2>&1 1>/dev/tty)


-bash: syntax error near unexpected token `2'

So bash as root has a problem with the redirection.


At this point I'm guessing you made a typo.  Since you didn't show us
the command you typed, that's my official guess.  Any alternative is
too far beyond belief.


It must have been a typo.
I should have copy-pasted the command, which was at that point possible, but 
anyway that syntax error no longer appears.


But this worked OK in the same root shell:

exec 3>1
errors=$( apt-get install mirage 2>&1 1>&3 )
exec 3>&-

If 'mirage' is replaced with 'mirag' then $errors holds the error message.

So at least something is working!


Note your typo in the first exec command.  You've got FD 3 redirected
to a file named '1', rather than dup-ing stdout.


Oops, sorry. With corrected commands it still works, even with default 
"use_pty" in sudoers.

So running in a root shell, the new setting does not interfere. Clearly sudo 
only.

--
John

Re: "sudo apt-get install android-tools-adb" ... (then no device listed)

[Jeffrey Walton]

On Mon, Sep 25, 2023 at 7:05 PM Albretch Mueller  wrote:
>
> On 9/24/23, Michel Verdier  wrote:
> > If you use USB you need a cable allowing data, some allow only power.
>
>  The  USB cable I have been using to charge the battery of that phone
> visually seems to be the same exact one being advertised as doubling
> as a data cable, but running:
>  $ sudo lsusb
>  Before and after plugging in the phone doesn’t show any difference.
>  Is there a way to test for sure that cable is the right one?

It sounds like you are using a charging cable, or a cheap data cable.
I would try a quality data cable before going down the rabbit hole.

You should also state which USB protocol your phone uses. Different
USB standards have different requirements. For example, USB 1.0 and
2.0 only need a 4-wire data cable (PWR, GND, D+ and D-). USB-C needs a
24-pin data cable and must implement the USB-PD protocol.

Also see .

Jeff

Re: "sudo apt-get install android-tools-adb" ... (then no device listed)

[Michel Verdier]

On 2023-09-25, Albretch Mueller wrote:

>  Is there a way to test for sure that cable is the right one?

Usually the original cable furnished with the phone is a data cable. My
only test was to successfully use adb then change cable and see that I
have some power only cables.

Re: "sudo apt-get install android-tools-adb" ... (then no device listed)

[Michel Verdier]

On 2023-09-25, Albretch Mueller wrote:

>  Android awakens when I unplug the cable from the computer; so,
> something is being somehow detected.

android also awakens on power on/off

Re: "sudo apt-get install android-tools-adb" ... (then no device listed)

[Nicolas George]

David Wright (12023-09-25):
> On bullseye I have android-file-transfer installed. I connect the
> phone to the PC with USB, and run this function:

If we are sharing how we do file transfer to and from an Android phone:

My favorite solution is tu install Termux and run sshd in it. Then I can
use rsync or anything I want to do the transfer. Add Tinc to get a
stable reachable IP.

Regards,

-- 
  Nicolas George


signature.asc
Description: PGP signature

Re: "sudo apt-get install android-tools-adb" ... (then no device listed)

[Albretch Mueller]

On 9/25/23, Albretch Mueller  wrote:
>  Most probably there is a setting in that phone I haven’t been able to
> find.

 Android awakens when I unplug the cable from the computer; so,
something is being somehow detected.

 lbrtchx

Re: "sudo apt-get install android-tools-adb" ... (then no device listed)

[Anssi Saari]

David Wright  writes:

> On Sun 24 Sep 2023 at 22:13:20 (+), Albretch Mueller wrote:
>> On 9/24/23, Marco M.  wrote:
>> > On most Android phones, you need to explicit allow data transfers.
>> 
>>  What do you functionally mean? I need for you to talk to me like
>> this:  a) go "Settings"; b) ...
>
> On bullseye I have android-file-transfer installed. I connect the
> phone to the PC with USB, and run this function:
>
>   samsungd () 
>   { 
>   sudo mkdir -p /media/samsungd || true;
>   sudo chown "$USER" /media/samsungd;
>   aft-mtp-mount /media/samsungd
>   }

Does this android-file-transfer provide a working and reliable MTP
implementation for Linux then? I think I'll stick with rsync and adb but
would be good to know if MTP is working on Linux these days.

Re: "sudo apt-get install android-tools-adb" ... (then no device listed)

[Albretch Mueller]

On 9/24/23, Michel Verdier  wrote:
> If you use USB you need a cable allowing data, some allow only power.

 The  USB cable I have been using to charge the battery of that phone
visually seems to be the same exact one being advertised as doubling
as a data cable, but running:
 $ sudo lsusb
 Before and after plugging in the phone doesn’t show any difference.
 Is there a way to test for sure that cable is the right one?
~
On 9/25/23, Marco  wrote:
> According to the Google documentation:

 which I found:

// __ Transfer files between your computer and Android device

 https://support.google.com/android/answer/9064445?hl=en-GB
~
 doesn’t really explain what to do. They apparently want for you to
transfer your files to google drive and all that non sense.

 I also tried mtp fs utility, but I am getting the error message:
detect failed: no MTP devices found

 go-mtpfs -android  ""
 2023/09/25 12:10:10 detect failed: no MTP devices found
~
 I also tried jmtpfs:

$ which jmtpfs
/usr/bin/jmtpfs

$ jmtpfs --version
jmtpfs version: 0.5
FUSE library version: 2.9.9
fusermount version: 2.9.9
using FUSE kernel interface version 7.19

$ sudo jmtpfs
No mtp devices found.
~
 Most probably there is a setting in that phone I haven’t been able to find.

 lbrtchx

Re: "sudo apt-get install android-tools-adb" ... (then no device listed)

[David Wright]

On Sun 24 Sep 2023 at 22:13:20 (+), Albretch Mueller wrote:
> On 9/24/23, Marco M.  wrote:
> > On most Android phones, you need to explicit allow data transfers.
> 
>  What do you functionally mean? I need for you to talk to me like
> this:  a) go "Settings"; b) ...

On bullseye I have android-file-transfer installed. I connect the
phone to the PC with USB, and run this function:

  samsungd () 
  { 
  sudo mkdir -p /media/samsungd || true;
  sudo chown "$USER" /media/samsungd;
  aft-mtp-mount /media/samsungd
  }

whereupon the phone will ask:

  Allow access to phone data?
  … …
 [Deny]   [Allow]

Tapping Allow makes the contents of the phone appear under
/media/samsungd. I use mc to transfer files in each direction.
When I've finished, I run:

  unsamsungd () 
  { 
  ###;
  fusermount -u /media/samsungd;
  sudo rmdir /media/samsungd || true
  }

and disconnect the cable (the one that came in the box with the phone).

Actually, the ### is really a call to a function that runs a flavour
of updatedb, and generates a compressed ls -lAR (for mc) and a
compressed custom listing of all the files (date/size/name).
BTW the d distinguishes my own phone from our other one when they're
both connected (there's no daemon involved).

As for (android-tools-)adb, I read that unless
android-sdk-platform-tools-common is installed, you need to run adb
as root. I don't know what extra things adb buys you, compared with
android-file-transfer. I'm mainly interested in pictures, movies,
audio, and stuff like that, rather than screwing around the phone's
internals.

Cheers,
David.

Re: apt-get aborts in subshell with redirections on Debian 12

[Greg Wooledge]

On Mon, Sep 25, 2023 at 01:35:38PM +0900, John Crawley wrote:
> > 4) In a bash shell as root (e.g. "su" or "sudo -s"), do:
> > 
> > errors=$(apt-get install mirage 2>&1 1>/dev/tty)
> 
> -bash: syntax error near unexpected token `2'
> 
> So bash as root has a problem with the redirection.

At this point I'm guessing you made a typo.  Since you didn't show us
the command you typed, that's my official guess.  Any alternative is
too far beyond belief.

> But this worked OK in the same root shell:
> 
> exec 3>1
> errors=$( apt-get install mirage 2>&1 1>&3 )
> exec 3>&-
> 
> If 'mirage' is replaced with 'mirag' then $errors holds the error message.
> 
> So at least something is working!

Note your typo in the first exec command.  You've got FD 3 redirected
to a file named '1', rather than dup-ing stdout.

> Maybe it is an issue with sudo?

According to Michael's discovery, yes.

Re: apt-get aborts in subshell with redirections on Debian 12

[Greg Wooledge]

On Mon, Sep 25, 2023 at 11:14:24AM +0200, Michael wrote:
> so i looked into /etc/sudoers and all /etc/sudoers.d/* and found two
> suspicous flags:
> 
> /etc/sudoers:
> Defaults   use_pty
> 
> /etc/sudoers.d/0pwfeedback:
> Defaults pwfeedback
> 
> then consulting the sudo manpage convinced me, it was the 'use_pty' flag (in
> section SUDOERS OPTIONS). after removing that flag everything works as
> 'expected':

Well, that is quite the find.  My systems are not freshly installed
Debian 12, but rather upgraded from previous releases.

Given the presence of an /etc/sudoers.dpkg-dist file on my system,
which does in fact contain this:

# This fixes CVE-2005-4890 and possibly breaks some versions of kdesu
# (#1011624, https://bugs.kde.org/show_bug.cgi?id=452532)
Defaultsuse_pty

it looks like you've solved *most* of the mystery.  I still don't get
why it matters whether the command was executed from bash or dash --
maybe bash sets up the file descriptors differently when performing
a command substitution, in a way that has a subtle interaction surprise
with sudo's use_pty option.  I don't know.

Re: "sudo apt-get install android-tools-adb" ... (then no device listed)

[Roland Mueller]

As far as I remember adb requires debugging to be enabled on the Android
device.

For newer androids following - slighly obscure - process has to be used:

Enabling USB Debugging on an Android Device
- On the device, go to Settings > About .
- Tap the Build number seven times to make Settings > Developer options
available.
- Then enable the USB Debugging option.

ma 25. syysk. 2023 klo 7.52 Marco (m...@dorfdsl.de) kirjoitti:

> Am 24.09.2023 22:13 schrieb Albretch Mueller:
>
> >  What do you functionally mean? I need for you to talk to me like
> > this:  a) go "Settings"; b) ...
>
> According to the Google documentation:
> With a USB cable, connect your device to your computer.
> On your device, tap the "Charging this device via USB" notification.
> Under "Use USB for," select File Transfer.
>
>

Re: apt-get aborts in subshell with redirections on Debian 12

[Michael]

hey,

i was curious, so i tried it on the only machine i have sudo installed on, 
which is my Debian based Ubuntu based LinuxMint Laptop.


 micha@HP-Laptop: ~
 > errors=$(sudo apt-get install mirage 2>&1 1>/dev/tty)
 [sudo] password for micha:
 Reading package lists... Done

 Building dependency tree... Done
 Reading state information... Done
 The following additional packages will be installed:
   gir1.2-gexiv2-0.10
 The following NEW packages will be installed:
   gir1.2-gexiv2-0.10 mirage
 0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
 Need to get 111 kB of archives.
 After this operation, 648 kB of additional disk space will be used.
 Do you want to continue? [Y/n] Abort.

then i switched to a shell, opened i with 'sudo tmux'

root@HP-Laptop: ~
# errors=$(apt-get install mirage 2>&1 1>/dev/tty)
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
 gir1.2-gexiv2-0.10
The following NEW packages will be installed:
 gir1.2-gexiv2-0.10 mirage
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 111 kB of archives.
After this operation, 648 kB of additional disk space will be used.
Do you want to continue? [Y/n] n
Abort.

so i looked into /etc/sudoers and all /etc/sudoers.d/* and found two 
suspicous flags:


/etc/sudoers:
Defaults   use_pty

/etc/sudoers.d/0pwfeedback:
Defaults pwfeedback

then consulting the sudo manpage convinced me, it was the 'use_pty' flag 
(in section SUDOERS OPTIONS). after removing that flag everything works as 
'expected':


 micha@HP-Laptop: ~
 > errors=$(sudo apt-get install mirage 2>&1 1>/dev/tty)
 [sudo] password for micha:
 Reading package lists... Done

 Building dependency tree... Done
 Reading state information... Done
 The following additional packages will be installed:
   gir1.2-gexiv2-0.10
 The following NEW packages will be installed:
   gir1.2-gexiv2-0.10 mirage
 0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
 Need to get 111 kB of archives.
 After this operation, 648 kB of additional disk space will be used.
 Do you want to continue? [Y/n] n
 Abort.

hth.

greetings...

Re: "sudo apt-get install android-tools-adb" ... (then no device listed)

[Marco]

Am 24.09.2023 22:13 schrieb Albretch Mueller:

>  What do you functionally mean? I need for you to talk to me like
> this:  a) go "Settings"; b) ...

According to the Google documentation:
With a USB cable, connect your device to your computer.
On your device, tap the "Charging this device via USB" notification.
Under "Use USB for," select File Transfer.

Re: apt-get aborts in subshell with redirections on Debian 12

[John Crawley]

On 25/09/2023 12:42, Greg Wooledge wrote:

On Mon, Sep 25, 2023 at 11:58:13AM +0900, John Crawley wrote:

adduser tmp
adduser tmp sudo

Log in to tmp (no graphical session set up), and the results are the same: 
behaviour in a bash shell is wrong, everything else works.


I simply can't reproduce your results at all.


This is so weird. There's probably some simple explanation somewhere...

I have a freshly installed Bookworm netinstall CLI system among my VMs. Only 
"standard system utilities" added during the install process. Restored the 
just-after-install snapshot, did an apt update/upgrade, and:

$ id
uid=1000(john) gid=1000(john) 
groups=1000(john),24(cdrom),25(floppy),27(sudo),29(audio),30(dip),44(video),46(plugdev),100(users),106(netdev)

Is there a hint here? Those are the default groups I was put in by the Debian 
Installer, but it's a much longer list than yours.

Anyway 'errors=$(sudo apt-get install mirage 2>&1 1>/dev/tty)' wrote a long 
list of dependencies (of course) but then aborted as usual.


This leaves me very confused.  There's something different about your
systems compared to mine (and the other person who also could not
reproduce your results), but damned if I can guess *what*.

The only other experiment I can think of at this time would be:

4) In a bash shell as root (e.g. "su" or "sudo -s"), do:

errors=$(apt-get install mirage 2>&1 1>/dev/tty)


-bash: syntax error near unexpected token `2'

So bash as root has a problem with the redirection.

But this worked OK in the same root shell:

exec 3>1
errors=$( apt-get install mirage 2>&1 1>&3 )
exec 3>&-

If 'mirage' is replaced with 'mirag' then $errors holds the error message.

So at least something is working!

Maybe it is an issue with sudo?


I can generate the *message* that you get by redirecting stdin:

But there's nothing you've shown that's redirecting stdin, so I don't
know whether this is what's causing your problem or not.  If it *is*
what's causing your problem, I still don't have a clue what's *doing*
it.

Is there any unusual configuration of sudo on your system?  Sudo
configuration is utterly baffling, and about 2 orders of magnitude more
stupidly complex than it should be, so I don't even know what kind of
atrocities are *possible* in sudoers, let alone which of them might be
capable of doing something insane like closing stdin on commands run
via sudo.  But if you've changed anything in sudoers or sudoers.d you
might want to report on those changes.


No. Those last tests were done on a freshly installed Bookworm netinstall VM, 
no changes apart from apt updating.


(But why would this only happen when sudo's parent is bash?  None of
this makes *any* sense.)


I just hope I haven't been making an awful stupid mistake all along...

--
John

Re: apt-get aborts in subshell with redirections on Debian 12

[Greg Wooledge]

On Mon, Sep 25, 2023 at 11:58:13AM +0900, John Crawley wrote:
> adduser tmp
> adduser tmp sudo
> 
> Log in to tmp (no graphical session set up), and the results are the same: 
> behaviour in a bash shell is wrong, everything else works.

I simply can't reproduce your results at all.

unicorn:~$ sudo adduser test2
[...]
unicorn:~$ sudo adduser test2 sudo
[...]
unicorn:~$ su - test2
Password: 
test2@unicorn:~$ id
uid=1001(test2) gid=1001(test2) groups=1001(test2),27(sudo),100(users)
test2@unicorn:~$ errors=$(sudo apt-get install mirage 2>&1 1>/dev/tty)
[sudo] password for test2: 
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  gir1.2-gexiv2-0.10 gir1.2-gtk-3.0 libexiv2-27 libgexiv2-2 python3-cairo
  python3-gi python3-gi-cairo
Suggested packages:
  exiv2 gimp
The following NEW packages will be installed:
  gir1.2-gexiv2-0.10 gir1.2-gtk-3.0 libexiv2-27 libgexiv2-2 mirage
  python3-cairo python3-gi python3-gi-cairo
0 upgraded, 8 newly installed, 0 to remove and 0 not upgraded.
Need to get 2,249 kB of archives.
After this operation, 10.5 MB of additional disk space will be used.
Do you want to continue? [Y/n] n
Abort.
test2@unicorn:~$ 

This leaves me very confused.  There's something different about your
systems compared to mine (and the other person who also could not
reproduce your results), but damned if I can guess *what*.

The only other experiment I can think of at this time would be:

4) In a bash shell as root (e.g. "su" or "sudo -s"), do:

   errors=$(apt-get install mirage 2>&1 1>/dev/tty)

This removes sudo from the picture.  But I don't know what the results
will tell us.

I can generate the *message* that you get by redirecting stdin:

unicorn:~$ sudo apt-get install mirage 0

Re: apt-get aborts in subshell with redirections on Debian 12

[John Crawley]

On 25/09/2023 11:58, John Crawley wrote:


So the 32bit system is different??



Doesn't semm to be that.
amd64 Bookworm VM behaves the same way.

--
John

Re: apt-get aborts in subshell with redirections on Debian 12

[John Crawley]

Thanks for the ideas!

On 25/09/2023 09:36, Greg Wooledge wrote:

On Mon, Sep 25, 2023 at 09:10:28AM +0900, John Crawley wrote:

I just tried, and yes it runs OK when the commands are in a script.
But type directly into the terminal:

errors=$(sudo apt-get install mirage 2>&1 1>/dev/tty)

To see the immediate abort.
(But not on Debian 11, or after invoking 'sh'.)


I actually tested in a terminal, not with a script.  But the fact that
*you* are getting different results with a script vs. a terminal is a
clue.  It doesn't give us a complete answer, but it's definitely a
significant observation.

The first thing you'll want to check is "type sudo", in your shell.

unicorn:~$ type sudo
sudo is hashed (/usr/bin/sudo)


sudo is hashed (/usr/bin/sudo)


If that doesn't give any further clues, you might look for unusual
configuration in your interactive shell.  Things like a DEBUG trap,
or the PROMPT_COMMAND or PS0 variables, which might trigger surprises
when running commands, would be good to know about.


I don't think I've added anything like that - this is a fairly new test install.
I tried logging into a Ctrl+Alt+F2 tty and got the same results.
For a moment I thought it might be because it's a Qemu VM, so dragged out my 
old test laptop and the results are the same.

The only thing I can think of right now is that this (VM and laptop both) is an 
i386 system. Could it be bash is missing some 32 bit update?

Anyway, I tried these:


1) Run "bash --noprofile --norc" to get a vanilla shell, without any
aliases, functions, weird shopts, etc.  See if you still get the
undesired results.


Same undesired result.


2) Try in dash, ksh, zsh or whatever other shells you have.  See if you
get the undesired results in any shell other than bash.  (It looks
like you already tried in dash, assuming sh -> dash.)


Dash is OK. I installed and tried ksh and zsh and there was no problem there 
either.


3) Create a new user account with only the default dotfiles, and see
if you get the undesired results there.  (You may have to add it to
the sudo group.)


adduser tmp
adduser tmp sudo

Log in to tmp (no graphical session set up), and the results are the same: 
behaviour in a bash shell is wrong, everything else works.

So the 32bit system is different??

--
John

Re: apt-get aborts in subshell with redirections on Debian 12

[Greg Wooledge]

On Mon, Sep 25, 2023 at 09:10:28AM +0900, John Crawley wrote:
> On 24/09/2023 21:36, Greg Wooledge wrote:
> > On Sun, Sep 24, 2023 at 12:03:12PM +0200, Michel Verdier wrote:
> > > #!/bin/bash
> > > errors=$(sudo apt-get install unknown 2>&1 1>/dev/tty)
> > > echo "output: $errors"
> > > errors=$(sudo apt-get install mirage 2>&1 1>/dev/tty)
> > > echo "output: $errors"
> > > 
> > > [...]
> > > It waits until I type 'n'
> > 
> > Same here.
> > 
> Thanks for testing.
> 
> I just tried, and yes it runs OK when the commands are in a script.
> But type directly into the terminal:
> 
> errors=$(sudo apt-get install mirage 2>&1 1>/dev/tty)
> 
> To see the immediate abort.
> (But not on Debian 11, or after invoking 'sh'.)

I actually tested in a terminal, not with a script.  But the fact that
*you* are getting different results with a script vs. a terminal is a
clue.  It doesn't give us a complete answer, but it's definitely a
significant observation.

The first thing you'll want to check is "type sudo", in your shell.

unicorn:~$ type sudo
sudo is hashed (/usr/bin/sudo)

If yours is an alias, or a function, or comes from some other directory,
that might explain the difference you see when running in a terminal.

If that doesn't give any further clues, you might look for unusual
configuration in your interactive shell.  Things like a DEBUG trap,
or the PROMPT_COMMAND or PS0 variables, which might trigger surprises
when running commands, would be good to know about.

You could also try a few experiments:

1) Run "bash --noprofile --norc" to get a vanilla shell, without any
   aliases, functions, weird shopts, etc.  See if you still get the
   undesired results.

2) Try in dash, ksh, zsh or whatever other shells you have.  See if you
   get the undesired results in any shell other than bash.  (It looks
   like you already tried in dash, assuming sh -> dash.)

3) Create a new user account with only the default dotfiles, and see
   if you get the undesired results there.  (You may have to add it to
   the sudo group.)

Re: apt-get aborts in subshell with redirections on Debian 12

[John Crawley]

On 24/09/2023 21:36, Greg Wooledge wrote:

On Sun, Sep 24, 2023 at 12:03:12PM +0200, Michel Verdier wrote:

#!/bin/bash
errors=$(sudo apt-get install unknown 2>&1 1>/dev/tty)
echo "output: $errors"
errors=$(sudo apt-get install mirage 2>&1 1>/dev/tty)
echo "output: $errors"

[...]
It waits until I type 'n'


Same here.


Thanks for testing.

I just tried, and yes it runs OK when the commands are in a script.
But type directly into the terminal:

errors=$(sudo apt-get install mirage 2>&1 1>/dev/tty)

To see the immediate abort.
(But not on Debian 11, or after invoking 'sh'.)

--
John

Re: "sudo apt-get install android-tools-adb" ... (then no device listed)

[Albretch Mueller]

On 9/24/23, Marco M.  wrote:
> On most Android phones, you need to explicit allow data transfers.

 What do you functionally mean? I need for you to talk to me like
this:  a) go "Settings"; b) ...

 Thank you,
 lbrtchx

On 9/24/23, Marco M.  wrote:
> Am 24.09.2023 um 19:45:11 Uhr schrieb Albretch Mueller:
>
>>  How can you troubleshoot that problem or, do you know about any other
>> way to transfer your data to a drive off your phone?
>
> On most Android phones, you need to explicit allow data transfers.
> Did you do?
>
>

Re: "sudo apt-get install android-tools-adb" ... (then no device listed)

[Michel Verdier]

On 2023-09-24, Albretch Mueller wrote:

>  Basically, I need to transfer selected data (whatsapp, messages,
> phone calls, ...) off my phone to my computer's hdd.
>
>  How can you troubleshoot that problem or, do you know about any other
> way to transfer your data to a drive off your phone?

If you use USB you need a cable allowing data, some allow only power.
Then you have to enable USB debug in developpement options (which you
have to enable using some magic specific to your model, search internet
for it). And USB must be accepted in MTP mode on your phone (usually by
default). If all this is done your device should be listed.

There is also some applications that can do transfer, like syncopoli
using rsync without USB. Or syncthing. Or termux on which you can use
subversion or git or other unix tools.

Re: "sudo apt-get install android-tools-adb" ... (then no device listed)

[Marco M.]

Am 24.09.2023 um 19:45:11 Uhr schrieb Albretch Mueller:

>  How can you troubleshoot that problem or, do you know about any other
> way to transfer your data to a drive off your phone?

On most Android phones, you need to explicit allow data transfers.
Did you do?

"sudo apt-get install android-tools-adb" ... (then no device listed)

[Albretch Mueller]

$ uname -a
Linux debian 5.10.0-18-amd64 #1 SMP Debian 5.10.140-1 (2022-09-02)
x86_64 GNU/Linux

$ sudo apt-get update
...

$ date; sudo apt-get install android-tools-adb
Sun 24 Sep 2023 02:07:24 PM UTC
...

$ which adb
/usr/bin/adb

$ adb --version
Android Debug Bridge version 1.0.41
Version 28.0.2-debian
Installed as /usr/lib/android-sdk/platform-tools/adb

$ adb devices
* daemon not running; starting now at tcp:5037
* daemon started successfully
List of devices attached

$

 As usual (being a TI makes your life so enjoyable!) I found similar
posts online and ways to solve such problems (some of them quite
voodoo-like):

 
https://stackoverflow.com/questions/21170392/my-android-device-does-not-appear-in-the-list-of-adb-devices
~
 Basically, I need to transfer selected data (whatsapp, messages,
phone calls, ...) off my phone to my computer's hdd.

 How can you troubleshoot that problem or, do you know about any other
way to transfer your data to a drive off your phone?

 lbrtchx

Re: apt-get aborts in subshell with redirections on Debian 12

[Greg Wooledge]

On Sun, Sep 24, 2023 at 12:03:12PM +0200, Michel Verdier wrote:
> #!/bin/bash
> errors=$(sudo apt-get install unknown 2>&1 1>/dev/tty)
> echo "output: $errors"
> errors=$(sudo apt-get install mirage 2>&1 1>/dev/tty)
> echo "output: $errors"
> 
> [...]
> It waits until I type 'n'

Same here.

Re: apt-get aborts in subshell with redirections on Debian 12

[Michel Verdier]

On 2023-09-24, John Crawley wrote:

> errors=$( sudo apt-get install mirage 2>&1 1>/dev/tty )
> echo "$errors"

#!/bin/bash
errors=$(sudo apt-get install unknown 2>&1 1>/dev/tty)
echo "output: $errors"
errors=$(sudo apt-get install mirage 2>&1 1>/dev/tty)
echo "output: $errors"

debian version 12.1
ii  apt 2.6.1amd64 
ii  bash5.2.15-2+b2  amd64 
ii  sudo   1.9.13p3-1+deb12u1 amd64

> Going straight to "Abort" which is what happens if apt has no stdin. But 
> surely it does?
>
> Running the same command on Debian 11 works as expected, waiting for USER 
> response at the prompt.

# ./test 
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
output: E: Unable to locate package unknown
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  gir1.2-gexiv2-0.10
The following NEW packages will be installed:
  gir1.2-gexiv2-0.10 mirage
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 120 kB of archives.
After this operation, 649 kB of additional disk space will be used.
Do you want to continue? [Y/n] n
Abort.
output: 

It waits until I type 'n'

Perhaps you have another command disturbing tty ?
Or check sudo ?

apt-get aborts in subshell with redirections on Debian 12

[John Crawley]

Hi folks,

I've just about run out of ideas here, so appealing for suggestions.
I don't know if this is a change in the behaviour of apt or bash, or both, 
between Debian 11 and 12.

I have a script which is supposed to smooth out some apt actions, while still 
keeping user interactions, but with the upgrade to Debian Bookworm some apt 
prompts are aborting immediately. Any apt errors are stored in a variable for 
checking over. I'll paste the full function below for those who are curious - 
but this simple command will illustrate:

errors=$( sudo apt-get install mirage 2>&1 1>/dev/tty )
echo "$errors"

(Replace 'mirage' with 'mirag' to see some content in $errors, and anyway 
choose a package you don't already have installed and which will bring in some 
dependencies, so apt will prompt to continue.)

On Debian 12, I get:

Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  gir1.2-gexiv2-0.10 libexiv2-27 libgexiv2-2
Suggested packages:
  exiv2 gimp imagemagick
The following NEW packages will be installed:
  gir1.2-gexiv2-0.10 libexiv2-27 libgexiv2-2 mirage
0 upgraded, 4 newly installed, 0 to remove and 0 not upgraded.
Need to get 1,761 kB of archives.
After this operation, 8,250 kB of additional disk space will be used.
Do you want to continue? [Y/n] Abort.

Going straight to "Abort" which is what happens if apt has no stdin. But surely 
it does?

Running the same command on Debian 11 works as expected, waiting for USER 
response at the prompt.

Also, even on Debian 12, the command does not abort if run on dash instead of 
bash.

This works OK too:

( sudo apt-get install mirage 2>&1 1>/dev/tty )

It's only when stderr is captured that the abort is triggered.

If there's no nice workaround I'll switch to making a temp file and put stderr 
there instead. Not elegant, but this works:

tmp=$(mktemp)
sudo apt-get install mirage 2> $tmp
errors=$(<$tmp)

FWIW this is the original bash function (maybe no longer needed anyway) which 
used to work on Bullseye but not on Bookworm:

# Usage safeInstall [--apt-get-option] package [package...]
safeInstall() {
local ignore_string_pw='Sorry, try again.' # message if password mistyped
local ignore_string1='Extracting templates from packages: 100%' # apt sends 
this to stderr!
local ignore_string2='Retrieving bug reports... Done' # apt-listbugs sends 
this to stderr!
local ignore_string3='Parsing Found/Fixed information... Done' # 
apt-listbugs sends this to stderr!
local apt_error
    if exec 3>&1; apt_error=$(LC_MESSAGES=C sudo apt-get install "${@}" 2>&1 1>&3) 
&& {
[[ -z "$apt_error" ]] || ! grep -iqEv 
"(${ignore_string_pw:-$^}|${ignore_string1:-$^}|${ignore_string2:-$^}|${ignore_string3:-$^}|^$)" 
<<<"$apt_error"
}
then
exec 3>&-
say 'Installation finished sucessfully.' 1
return 0
else
exec 3>&-
errorExit "There were problems installing ${*}" "${apt_error:-Install 
Aborted}"
return 1
fi
}

--
John

Re: For syslog : apt-get install rsyslog

[tomas]

On Thu, Jul 06, 2023 at 08:21:18PM -0600, Charles Curley wrote:
> On Fri, 7 Jul 2023 10:53:10 +1200
> "C.T.F. Jansen"  wrote:
> 
> > The current default logging is unsatisfactory and needs to include a 
> > text log.
> 
> You can get much of the effect of /var/log/syslog with journalctl.

The OP seems to prefer "all of the effect", and not just "much of",
which I understand. So thanks them for the hint (I run systemd-less,
so I just know enough journalctl to help other poor souls ;-)

Cheers
-- 
t


signature.asc
Description: PGP signature

Re: For syslog : apt-get install rsyslog

[Charles Curley]

On Fri, 7 Jul 2023 10:53:10 +1200
"C.T.F. Jansen"  wrote:

> The current default logging is unsatisfactory and needs to include a 
> text log.

You can get much of the effect of /var/log/syslog with journalctl. "man
journactl" for the gory details. Of particular use are the -f and -u
options.

-- 
Does anybody read signatures any more?

https://charlescurley.com
https://charlescurley.com/blog/

Re: For syslog : apt-get install rsyslog

[jeremy ardley]

On 7/7/23 07:59, Greg Wooledge wrote:

As with many of the unpopular changes that Debian embraces, this is a
decision they made, and no matter how utterly daft and ridiculous it is,
no amount of griping by users will change their minds about it.



As I have taken the first step of changing one of my systems to V12, the 
OP revelation that I can get back my syslogs is most welcome.


On the topic of gripes, 12 still has the obnoxious NetworkManager as 
immutable default on new install. Luckily this is easily undone. It 
takes moments replace it with my now preferred systemd-networkd which is 
highly competent , predictable, and unbuggy - unlike NetworkManager



Jeremy

Re: For syslog : apt-get install rsyslog

[Greg Wooledge]

On Fri, Jul 07, 2023 at 10:53:10AM +1200, C.T.F. Jansen wrote:
> To restore a readable and accessible syslog in Debian 12.0 enter
> 
>            apt-get  install rsyslog
> 
> One doesn't need to do anything with the journal suite.
> I installed rsyslog-doc as well. It is in: /usr/share/doc/rsyslog-doc/html

... is there a question in here?

> The current default logging is unsatisfactory and needs to include a text
> log.

Oh, it's a gripe, not a question.

You may dislike the decision all you wish, and we may even all agree
with you, but griping here will not change anything.

This was an intentional change made by Debian, and they are proud of it.
They even documented it, and told you how you can achieve this
no-text-log-files state after an upgrade from a previous release:

https://www.debian.org/releases/bookworm/amd64/release-notes/ch-information.en.html#changes-to-system-logging

As with many of the unpopular changes that Debian embraces, this is a
decision they made, and no matter how utterly daft and ridiculous it is,
no amount of griping by users will change their minds about it.

For syslog : apt-get install rsyslog

[C.T.F. Jansen]

To restore a readable and accessible syslog in Debian 12.0 enter

   apt-get  install rsyslog

One doesn't need to do anything with the journal suite.
I installed rsyslog-doc as well. It is in: 
/usr/share/doc/rsyslog-doc/html


The logs appear to be set up to rotate etc already in: 
/etc/logrotate.conf and   /etc/logrotate.d/


This seems to restore other logs as well.

To implement and maintain a system there should be a text log which is 
immediately accessible.
The current default logging is unsatisfactory and needs to include a 
text log.

One can easily stop it later. This can be done with rsyslog etc already...

Cheers

CTF Jansen,  frank.jan...@actrix.gen.nz, ZL2TTS

Re: apt-get under Kali Linux reports wrong data rate.

[Jeffrey Walton]

On Sat, Dec 24, 2022 at 2:27 PM Mark <196...@protonmail.com> wrote:
>
> I have no idea who to report this issue to.

For Kali issues, see https://www.kali.org/community/

Jeff

Re: apt-get under Kali Linux reports wrong data rate.

[Greg Wooledge]

On Sat, Dec 24, 2022 at 07:08:20PM +, Mark wrote:
> I have no idea who to report this issue to.
> 
> Under latest build of Kali Linux i have noticed the download speed with 
> apt-get commands, it shows K Bits/s and should in fact be K bits/s
> The difference between Bytes & Bits is important.
> 
> N.B small b not capital B!
> KIDS!

What evidence do you have to support this claim?

apt-get shows "B/s" for me, which I'm assuming is Bytes per second.  At
no point does it ever say "Bits", nor would I expect a user-facing
application that works with files to report bits per second.

ii  apt2.2.4amd64commandline package manager

If your derivative hacker-centric OS is doing something different, then
you should report it as a bug to YOUR operating system's support team.
Not to the Debian end user mailing list.

apt-get under Kali Linux reports wrong data rate.

[Mark]

Dear People

I have no idea who to report this issue to.

Under latest build of Kali Linux i have noticed the download speed with apt-get 
commands, it shows K Bits/s and should in fact be K bits/s
The difference between Bytes & Bits is important.

N.B small b not capital B!
KIDS!

Please pass on to Debian & Kali developers this major error in download speeds.

Regards

​Mark Edgar

Sent with [Proton Mail](https://proton.me/) secure email.

Re: debugging apt-get

[David Wright]

On Wed 12 Oct 2022 at 16:42:10 (-0700), Stefan Seefeld wrote:
> On Wednesday, 12 October 2022 at 01:10:06 UTC-4, tomas wrote:
> 
> > Try doing "apt-cache policy Y", that might shed light on this.
> 
> This reports
> ```
> Y:
>  Installed: (none)
>  Candidate: 1.0.3
>  Version table:
> 1.0.3 500
>500  focal/main amd64 Packages
> 1.0.1 500
>500  focal/main amd64 Packages
> ```
> 
> > > (I'm running on Ubuntu 20.04, using apt-get=2.0.6)
> > (Isn't there a Ubuntu mailing list, btw? They might be doing funny stuff 
> > with their packaging which perhaps change the problem space) 
> 
> The repo is actually our own in-house repo (managed via artifactory).
> Are there any clear semantic rules spelled out in some document that govern 
> what should happen ? I'm surprised that if my package "X" has specific and 
> unambiguous dependencies on an existing package "Y", and that version exists 
> in the repo, it should be installed. Am I missing something ?
> 
> Is this an `apt-get` bug ? Is there any way to debug this further ?
> (For a specific case I can work around the issue by injecting an explicit 
> `apt-get install` command, but this is of course not a scalable solution for 
> the general case.)

Can we check your package's control file, by your typing:

$ ar --output=/tmp/ -x your-X-package.deb control.tar.xz ; zcat 
/tmp/control.tar.xz

to see what X instructs apt to do.

Cheers,
David.

Re: debugging apt-get

[Tim Woodall]

On Wed, 12 Oct 2022, Stefan Seefeld wrote:


On Wednesday, 12 October 2022 at 01:10:06 UTC-4, tomas wrote:


Try doing "apt-cache policy Y", that might shed light on this.


This reports
```
Y:
Installed: (none)
Candidate: 1.0.3
Version table:
   1.0.3 500
  500  focal/main amd64 Packages
   1.0.1 500
  500  focal/main amd64 Packages
```


(I'm running on Ubuntu 20.04, using apt-get=2.0.6)

(Isn't there a Ubuntu mailing list, btw? They might be doing funny stuff
with their packaging which perhaps change the problem space)


The repo is actually our own in-house repo (managed via artifactory).
Are there any clear semantic rules spelled out in some document that govern what should happen ? 
I'm surprised that if my package "X" has specific and unambiguous dependencies on an 
existing package "Y", and that version exists in the repo, it should be installed. Am I 
missing something ?



In your original command you had X as a file on the commandline. Does it
work when X is in a repo?

I have my own repo and I've had no issues like this.

$ apt-mark showauto | grep dump
dump
$ apt-mark showmanual | grep backup
backup
$

Is there something strange about what release or architecture X thinks
it is in? Perhaps that is why apt won't autoinstall Y but Y can satisfy
X's dependencies.

This file has all the debugging options for apt:

/usr/share/doc/apt/examples/configure-index

-o Debug::pkgProblemResolver=yes might show something. (that's a guess,
I've not used this option)

Re: debugging apt-get

[Stefan Seefeld]

On Wednesday, 12 October 2022 at 01:10:06 UTC-4, tomas wrote:

> Try doing "apt-cache policy Y", that might shed light on this.

This reports
```
Y:
 Installed: (none)
 Candidate: 1.0.3
 Version table:
1.0.3 500
   500  focal/main amd64 Packages
1.0.1 500
   500  focal/main amd64 Packages
```

> > (I'm running on Ubuntu 20.04, using apt-get=2.0.6)
> (Isn't there a Ubuntu mailing list, btw? They might be doing funny stuff 
> with their packaging which perhaps change the problem space) 

The repo is actually our own in-house repo (managed via artifactory).
Are there any clear semantic rules spelled out in some document that govern 
what should happen ? I'm surprised that if my package "X" has specific and 
unambiguous dependencies on an existing package "Y", and that version exists in 
the repo, it should be installed. Am I missing something ?

Is this an `apt-get` bug ? Is there any way to debug this further ?
(For a specific case I can work around the issue by injecting an explicit 
`apt-get install` command, but this is of course not a scalable solution for 
the general case.)

Would there be another place (perhaps an apt-get - specific mailing list or bug 
tracker) where I can follow up to try to resolve this ?

Thanks,

Re: debugging apt-get

[tomas]

On Tue, Oct 11, 2022 at 10:35:46PM -0400, Stefan Seefeld wrote:
> I'm trying to build a docker image containing a debian package I just
> created locally.
> 
> I'm thus running the command `apt-get update && apt-get install -y ./X.deb`
> from my dockerfile. This command fails with the message
> 
> ...
> 
> The following packages have unmet dependencies:
>  X : Depends: Y (= 1.0.3) but it is not going to be installed
> 
> ...
> 
> However, if I run `apt-get update && apt-get install Y=1.0.3` first, the
> build completes successfully. What could cause this behaviour ? Why would
> `apt-get` not be able to install a prerequisite package, when installing it
> explicitly works ?

Most probably because there is another version of Y with a higher priority.

Try doing "apt-cache policy Y", that might shed light on this.

> (I'm running on Ubuntu 20.04, using apt-get=2.0.6)

(Isn't there a Ubuntu mailing list, btw? They might be doing funny stuff
with their packaging which perhaps change the problem space)

Cheers
-- 
t
> 
> Thanks,


signature.asc
Description: PGP signature

debugging apt-get

[Stefan Seefeld]

I'm trying to build a docker image containing a debian package I just 
created locally.


I'm thus running the command `apt-get update && apt-get install -y 
./X.deb` from my dockerfile. This command fails with the message


...

The following packages have unmet dependencies:
 X : Depends: Y (= 1.0.3) but it is not going to be installed

...

However, if I run `apt-get update && apt-get install Y=1.0.3` first, the 
build completes successfully. What could cause this behaviour ? Why 
would `apt-get` not be able to install a prerequisite package, when 
installing it explicitly works ?


Is there a way to debug this, i.e. have `apt-get` produce more verbose 
output that indicates *why* the prerequisite package is not going to be 
installed ?


(I'm running on Ubuntu 20.04, using apt-get=2.0.6)

Thanks,

Re: Re: Re: apt-get dist-upgrade keeping back some packets

[Gionatan Danti]

On Fri, 23 Sep 2022 07:51:15 +0200 Gionatan Danti wrote:

Using libsystemd0 as an example, apt-cache policy shown the installed 
packages
with score 100, and an available update with score 500. Still, the 
update was

not installed until I manually specified the package on the
apt-get dist-upgrade command line.


Ok, I went to the bottom: it was due to phased updates not selecting my 
machine for applying the update. Disabling it via -o 
APT::Get::Always-Include-Phased-Updates=1 force apt-get dist-upgrade to 
update anything.


I hope this can be useful for others.
Thanks.

--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.da...@assyoma.it - i...@assyoma.it
GPG public key ID: FF5F32A8

Re: Re: apt-get dist-upgrade keeping back some packets

[Gionatan Danti]

On Thu, 22 Sep 2022 19:33:20 -0400 Greg Wooledge wrote:

You've either got a Frankendebian system, or a pin.  Or both.



Review your sources.list and sources.list.d/* and see if you've mixed
different branches, or different operating systems.


Or pick a package from the "kept back" list, and do an "apt-cache 
policy

pkgname" on it.  See whether it's pinned, or has a version that's ahead
of stable, and then try to remember what you did to achieve that state.


It was my thinking as well, but no source.list changes on this system, 
no pinning, no apt-mark. Nothing.


Using libsystemd0 as an example, apt-cache policy shown the installed 
packages with score 100, and an available update with score 500. Still, 
the update was not installed until I manually specified the package on 
the apt-get dist-upgrade command line.


Full disclosure: this specific system is an Ubuntu installation. I wrote 
here because the issue seems with how apt-get identifies protected 
packages, rather than distro-related.


On Fri, 23 Sep 2022 01:48:38 +0100, Peter Hillier-Brook

I've had a similar problem in the recent past with .
I resolved it by running Synaptic, which solved the issue with no 
reported errors.

I'm running pure Bullseye with no manually pinned packages.


Yep, the issue seems confined to apt-get: issuing aptitude full-upgrade 
worked flawlessy.


Regards.

--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.da...@assyoma.it - i...@assyoma.it
GPG public key ID: FF5F32A8

Re: apt-get dist-upgrade keeping back some packets

[Peter Hillier-Brook]

On 23/09/2022 00:33, Greg Wooledge wrote:

On Fri, Sep 23, 2022 at 01:04:26AM +0200, Gionatan Danti wrote:

root@localhost:/var/log/apt# apt-get dist-upgrade
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Calculating upgrade... Done
The following packages have been kept back:
   grub-efi-amd64 grub-efi-amd64-bin grub-efi-amd64-signed libnss-mymachines
libnss-systemd libpam-systemd libpython3-dev libpython3-stdlib libpython3.10
libpython3.10-dev libpython3.10-minimal libpython3.10-stdlib libsystemd0
libudev1
   python3 python3-dev python3-distutils python3-gdbm python3-lib2to3
python3-minimal python3-tk python3.10 python3.10-dev python3.10-minimal
systemd systemd-container systemd-sysv systemd-timesyncd udev
The following packages will be upgraded:
   bind9-dnsutils bind9-host bind9-libs libpcre2-8-0 python3-oauthlib


You've either got a Frankendebian system, or a pin.  Or both.

Review your sources.list and sources.list.d/* and see if you've mixed
different branches, or different operating systems.

Or pick a package from the "kept back" list, and do an "apt-cache policy
pkgname" on it.  See whether it's pinned, or has a version that's ahead
of stable, and then try to remember what you did to achieve that state.


I've had a similar problem in the recent past with . I 
resolved it by running Synaptic, which solved the issue with no reported 
errors.


I'm running pure Bullseye with no manually pinned packages.

Peter HB

Re: apt-get dist-upgrade keeping back some packets

[Greg Wooledge]

On Fri, Sep 23, 2022 at 01:04:26AM +0200, Gionatan Danti wrote:
> root@localhost:/var/log/apt# apt-get dist-upgrade
> Reading package lists... Done
> Building dependency tree... Done
> Reading state information... Done
> Calculating upgrade... Done
> The following packages have been kept back:
>   grub-efi-amd64 grub-efi-amd64-bin grub-efi-amd64-signed libnss-mymachines
> libnss-systemd libpam-systemd libpython3-dev libpython3-stdlib libpython3.10
> libpython3.10-dev libpython3.10-minimal libpython3.10-stdlib libsystemd0
> libudev1
>   python3 python3-dev python3-distutils python3-gdbm python3-lib2to3
> python3-minimal python3-tk python3.10 python3.10-dev python3.10-minimal
> systemd systemd-container systemd-sysv systemd-timesyncd udev
> The following packages will be upgraded:
>   bind9-dnsutils bind9-host bind9-libs libpcre2-8-0 python3-oauthlib

You've either got a Frankendebian system, or a pin.  Or both.

Review your sources.list and sources.list.d/* and see if you've mixed
different branches, or different operating systems.

Or pick a package from the "kept back" list, and do an "apt-cache policy
pkgname" on it.  See whether it's pinned, or has a version that's ahead
of stable, and then try to remember what you did to achieve that state.

apt-get dist-upgrade keeping back some packets

[Gionatan Danti]

Hi all,
I have a question about apt-get dist-upgrade refusing to update some 
packages:


root@localhost:/var/log/apt# apt-get dist-upgrade
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Calculating upgrade... Done
The following packages have been kept back:
  grub-efi-amd64 grub-efi-amd64-bin grub-efi-amd64-signed 
libnss-mymachines libnss-systemd libpam-systemd libpython3-dev 
libpython3-stdlib libpython3.10 libpython3.10-dev libpython3.10-minimal 
libpython3.10-stdlib libsystemd0 libudev1
  python3 python3-dev python3-distutils python3-gdbm python3-lib2to3 
python3-minimal python3-tk python3.10 python3.10-dev python3.10-minimal 
systemd systemd-container systemd-sysv systemd-timesyncd udev

The following packages will be upgraded:
  bind9-dnsutils bind9-host bind9-libs libpcre2-8-0 python3-oauthlib
5 upgraded, 0 newly installed, 0 to remove and 29 not upgraded.
5 standard security updates
Need to get 1,719 kB of archives.
After this operation, 3,072 B of additional disk space will be used.
Do you want to continue? [Y/n] n
Abort.

To update, I had to specify all the packages on the command line after 
the dist-upgrade command (or use aptitude full-upgrade). Enabling debug 
via resulted in something similar to that:


root@localhost:/etc/apt# apt-get -oDebug::pkgProblemResolver=1 
-oDebug::BuildDep=1 -oDebug::pkgDepCache::AutoInstall=1 
-oDebug::pkgDepCache::Marker=1 -oDebug::pkgOrderList=1 
-oDebug::pkgProblemResolver::ShowScores=1 -oDebug::pkgDpkgPm=1 
-opkgDPkgPm::Item=1 dist-upgrade 2>&1 | less

Calculating upgrade...Starting pkgProblemResolver with broken count: 0
Settings used to calculate pkgProblemResolver::Scores::
  Required => 3
  Important => 2
  Standard => 1
  Optional => -1
  Extra => -2
  Essentials => 100
  InstalledAndNotObsolete => 1
  Pre-Depends => 1
  Depends => 1
  Recommends => 1
  Suggests => 0
  Conflicts => -1
  Breaks => -1
  Replaces => 0
  Obsoletes => 0
  Enhances => 0
  AddProtected => 1
  AddEssential => 5000
Show Scores
10539 libsystemd0:amd64 < 249.11-0ubuntu3.6 @ii pmK >
10419 python3:amd64 < 3.10.6-1~22.04 @ii pmK >
10320 libudev1:amd64 < 249.11-0ubuntu3.6 @ii pmK >
10206 python3-minimal:amd64 < 3.10.6-1~22.04 @ii pmK >
10206 libpython3-stdlib:amd64 < 3.10.6-1~22.04 @ii pmK >
10150 systemd-sysv:amd64 < 249.11-0ubuntu3.6 @ii pmK >
...

It seems to me that some package is marked as Protected. However, 
running dpkg on one of these packages show no 
protected/essential/important flag:


root@localhost:/var/cache# dpkg-query -Wf 'package:${Package} 
arch:${Architecture} bugs:${Bugs} ess:${Essential} pri:${Priority} 
pro:${Protected} imp:${Important}\n' libsystemd0

package:libsystemd0 arch:amd64 bugs: ess:no pri:optional pro:no imp:

So, why apt-get dist-upgrade refuses to upgrade? How it collect the list 
of protected packages?

Thanks.

--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.da...@assyoma.it - i...@assyoma.it
GPG public key ID: FF5F32A8

Re: apt-get -qq install

[Victor Sudakov]

Curt wrote:
> On 2022-08-03, Victor Sudakov  wrote:
> >
> >
> > Curt wrote:
> >> >
> >> >
> >> > I'm trying to quiet apt's output by using `apt-get -qqy` in a CI/CD
> >> > pipeline, however I still see ugly stuff like this in my CI/CD log:
> 
> >> Quiet level 2 implies -y.
> >
> > An extra -y won't do any harm, especially if some day someone decides
> > to remove one -q to debug the pipeline for example.
> 
> So what's acceptable in the input is intolerable in the output.

That's correct. In fact, the apt command with its arguments is not
even visible in the CI/CD log, only its output is.

-- 
Victor Sudakov VAS4-RIPE
http://vas.tomsk.ru/
2:5005/49@fidonet


signature.asc
Description: PGP signature

Re: apt-get -qq install

[Curt]

On 2022-08-03, Victor Sudakov  wrote:
>
>
> Curt wrote:
>> >
>> >
>> > I'm trying to quiet apt's output by using `apt-get -qqy` in a CI/CD
>> > pipeline, however I still see ugly stuff like this in my CI/CD log:

>> Quiet level 2 implies -y.
>
> An extra -y won't do any harm, especially if some day someone decides
> to remove one -q to debug the pipeline for example.

So what's acceptable in the input is intolerable in the output.

Re: apt-get -qq install

[Victor Sudakov]

Curt wrote:
> >
> >
> > I'm trying to quiet apt's output by using `apt-get -qqy` in a CI/CD
> > pipeline, however I still see ugly stuff like this in my CI/CD log:
> 
> Quiet level 2 implies -y.

An extra -y won't do any harm, especially if some day someone decides
to remove one -q to debug the pipeline for example.

> 
> > Selecting previously unselected package php-common.
> 
> That's dpkg ouput, isn't it? So maybe add '-o=Dpkg::Use-Pty=0' to your 
> command.

I have already written in  that I've tried 
setting `sudo DEBIAN_FRONTEND=noninteractive apt-get -qqy 
--no-install-recommends install ...` 
and `sudo apt-get -qqy -o Dpkg::Use-Pty=0 ...` but the rubbish is still there.

-- 
Victor Sudakov VAS4-RIPE
http://vas.tomsk.ru/
2:5005/49@fidonet


signature.asc
Description: PGP signature

Re: apt-get -qq install

[Victor Sudakov]

David Christensen wrote:
> >>>
> >>> (Un)fortunately this is a CI/CD pipeline, the VM and its data will be
> >>> gone forever after the build. Unless I care to keep apt output as an
> >>> artifact somewhere which is IMHO an overkill. I just want an concise
> >>> CI/CD log without interactive bells and whistles like progress
> >>> indicators.
> >>
> >>
> >> Without seeing your CI/CD pipeine -- create a patterns file containing
> >> regular expressions that match the lines you want removed, then add a
> >> grep(1) filter into the pipeline:
> > 
> > Of course I can think of many workarounds. The original question was
> > however why "-qq" was not working and what I was missing in "apt"
> > usage.
> 
> 
> You're right -- I made no attempt to figure out why apt(8) apparently 
> does not implement proper "quiet", "real-quiet", etc., options.
> 
> 
> Call it pragmatism.  I've been down plenty of rabbit holes trying to 
> trouble-shoot software; and I expect apt(8) is non-trivial.  How many 
> hours have you and others spent on this issue?  

The link Davidson has provided shows that many people have spent some
time on it since 2009.

> Have you found the 
> "correct" solution?  Using grep(1) in a command pipeline to filter out 
> unwanted lines is a known technique.  It took me a few minutes to write 
> the example, and it works.

Yes, I think I'll have to resort to a workaround once it's now clear
that this is a bug. I just don't like the idea of creating a
workaround when there is an official way to do something, but this is
clearly not the case.

-- 
Victor Sudakov VAS4-RIPE
http://vas.tomsk.ru/
2:5005/49@fidonet


signature.asc
Description: PGP signature

Re: apt-get -qq install

[Victor Sudakov]

davidson wrote:
> > Thank you, how do you activate this option? I've just tried
> > `apt-get -o quiet::NoProgress=true -qqy ...`
> 
> I would do it that way too.
> 
> > but the "Reading database ... 5%" stuff is still there.
> 
> Yeah, after some unsatisfying experimentation I've had no luck either.
> 
> Maybe this bug report will interest you:
> 
>   #539617 - dpkg: Add option to suppress progress display while reading 
> database
>   https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=539617

So, it's an old problem since 2009, and still unresolved? Thank you for the 
link.

On the one hand, it's sad. On the other hand, I'm glad I'm not the only one who 
is annoyed.

-- 
Victor Sudakov VAS4-RIPE
http://vas.tomsk.ru/
2:5005/49@fidonet


signature.asc
Description: PGP signature

Re: apt-get -qq install

[David Christensen]

On 8/2/22 17:17, Victor Sudakov wrote:

David Christensen wrote:

On 8/2/22 15:53, Victor Sudakov wrote:


(Un)fortunately this is a CI/CD pipeline, the VM and its data will be
gone forever after the build. Unless I care to keep apt output as an
artifact somewhere which is IMHO an overkill. I just want an concise
CI/CD log without interactive bells and whistles like progress
indicators.



Without seeing your CI/CD pipeine -- create a patterns file containing
regular expressions that match the lines you want removed, then add a
grep(1) filter into the pipeline:


Of course I can think of many workarounds. The original question was
however why "-qq" was not working and what I was missing in "apt"
usage.



You're right -- I made no attempt to figure out why apt(8) apparently 
does not implement proper "quiet", "real-quiet", etc., options.



Call it pragmatism.  I've been down plenty of rabbit holes trying to 
trouble-shoot software; and I expect apt(8) is non-trivial.  How many 
hours have you and others spent on this issue?  Have you found the 
"correct" solution?  Using grep(1) in a command pipeline to filter out 
unwanted lines is a known technique.  It took me a few minutes to write 
the example, and it works.



David

Re: apt-get -qq install

[Curt]

On 2022-08-02, Victor Sudakov  wrote:
>
> Dear Colleagues,
>
> I'm trying to quiet apt's output by using `apt-get -qqy` in a CI/CD
> pipeline, however I still see ugly stuff like this in my CI/CD log:

Quiet level 2 implies -y.

> Selecting previously unselected package php-common.

That's dpkg ouput, isn't it? So maybe add '-o=Dpkg::Use-Pty=0' to your command.




-- 

Re: apt-get -qq install

[Tim Woodall]

On Tue, 2 Aug 2022, Victor Sudakov wrote:


David Wright wrote:

On Tue 02 Aug 2022 at 18:27:22 (+), Victor Sudakov wrote:

I'm trying to quiet apt's output by using `apt-get -qqy` in a CI/CD
pipeline, however I still see ugly stuff like this in my CI/CD log:

Selecting previously unselected package php-common.
(Reading database ...
(Reading database ... 5%

[ ? ]


and this:

Scanning processes... []
Scanning processes... [=   ]

[ ? ]


Don't you think the `-qq` modifier should spare me these things?

I don't actually like the idea of redirecting apt-get's output to
/dev/null because I want to see the list of packages installed, but without
these pseudographics. Do you think it's possible? Any ideas?


The packages installed and the terminal output are logged in
/var/log/apt/{history,term}.log respectively (which I never rotate).


Which will be lost in a CI/CD environment because the VM image is ephemeral.

BTW I've tried setting `sudo DEBIAN_FRONTEND=noninteractive apt-get -qqy 
--no-install-recommends install ...` and
`sudo apt-get -qqy -o Dpkg::Use-Pty=0 ...` to no avail, the rubbish is still 
there.



something strange - I see this in my console log:

$ less essential-phase1-sid-i386.log
/bin/sh: 0: can't access tty; job control turned off
# # # Get:1 http://ftp.uk.debian.org/debian sid InRelease [192 kB]
Get:2 http://ftp.uk.debian.org/debian sid/main Sources [9857 kB]
Get:3 http://ftp.uk.debian.org/debian sid/main i386 Packages [9185 kB]
Get:4 http://ftp.uk.debian.org/debian sid/main Translation-en [6861 kB]
Fetched 26.1 MB in 20s (1311 kB/s)
Reading package lists...
# # Reading package lists...
Building dependency tree...
Reading state information...
Calculating upgrade...
The following packages will be upgraded:
  apt base-files libc-bin util-linux
4 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 3690 kB of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 http://ftp.uk.debian.org/debian sid/main i386 base-files i386 12.2 [70.1 
kB]
Get:2 http://ftp.uk.debian.org/debian sid/main i386 util-linux i386 2.38-6 
[1258 kB]
Get:3 http://ftp.uk.debian.org/debian sid/main i386 apt i386 2.5.2 [1522 kB]
Get:4 http://ftp.uk.debian.org/debian sid/main i386 libc-bin i386 2.33-8 [840 
kB]
debconf: delaying package configuration, since apt-utils is not installed
Fetched 3690 kB in 2s (1981 kB/s)
E: Can not write log (Is /dev/pts mounted?) - posix_openpt (2: No such file or 
directory)
(Reading database ... 6645 files and directories currently installed.)
Preparing to unpack .../base-files_12.2_i386.deb ...
Unpacking base-files (12.2) over (12.2~tjw) ...
Setting up base-files (12.2) ...
(Reading database ... 6645 files and directories currently installed.)
...


Which is coming from:
chroot ${workdir} <

Re: apt-get -qq install

[davidson]

On Tue, 2 Aug 2022 Victor Sudakov wrote:

davidson wrote:


I'm trying to quiet apt's output by using `apt-get -qqy` in a CI/CD
pipeline, however I still see ugly stuff like this in my CI/CD log:

Selecting previously unselected package php-common.
(Reading database ...
(Reading database ... 5%
(Reading database ... 10%

[snip]


and this:

Scanning processes... []
Scanning processes... []
Scanning processes... []
Scanning processes... [=   ]
Scanning processes... [=   ]
Scanning processes... [==  ]
Scanning processes... [==  ]

[snip]


Don't you think the `-qq` modifier should spare me these things?


I would expect so too.


I don't actually like the idea of redirecting apt-get's output to
/dev/null because I want to see the list of packages installed, but without
these pseudographics. Do you think it's possible? Any ideas?


In the file

  /usr/share/doc/apt/examples/configure-index.gz

I notice there is a stanza that begins like so:

  quiet "" {
NoUpdate ""; // never update progress information - included in -q=1
NoProgress ""; // disables the 0% -> 100% progress on cache 
generation and stuff

I don't know whether this will help you, but it does look suggestive.


Thank you, how do you activate this option? I've just tried
`apt-get -o quiet::NoProgress=true -qqy ...`


I would do it that way too.


but the "Reading database ... 5%" stuff is still there.


Yeah, after some unsatisfying experimentation I've had no luck either.

Maybe this bug report will interest you:

 #539617 - dpkg: Add option to suppress progress display while reading database
 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=539617


--
Ce qui est important est rarement urgent
et ce qui est urgent est rarement important
-- Dwight David Eisenhower

Re: apt-get -qq install

[Victor Sudakov]

David Christensen wrote:
> On 8/2/22 15:53, Victor Sudakov wrote:
> > 
> > (Un)fortunately this is a CI/CD pipeline, the VM and its data will be
> > gone forever after the build. Unless I care to keep apt output as an
> > artifact somewhere which is IMHO an overkill. I just want an concise
> > CI/CD log without interactive bells and whistles like progress
> > indicators.
> 
> 
> Without seeing your CI/CD pipeine -- create a patterns file containing 
> regular expressions that match the lines you want removed, then add a 
> grep(1) filter into the pipeline:

Of course I can think of many workarounds. The original question was
however why "-qq" was not working and what I was missing in "apt"
usage.

-- 
Victor Sudakov VAS4-RIPE
http://vas.tomsk.ru/
2:5005/49@fidonet


signature.asc
Description: PGP signature

Re: apt-get -qq install

[David Christensen]

On 8/2/22 15:53, Victor Sudakov wrote:

David Christensen wrote:


I don't actually like the idea of redirecting apt-get's output to
/dev/null because I want to see the list of packages installed, but without
these pseudographics. Do you think it's possible? Any ideas?



I prefer the idea of collecting all of the data and then writing queries
and reports to print the interesting bits.  If you keep all of the data,
you can apply future queries and reports to past data.  But if you
filter the data up front and save one report format, your future options
are limited.


(Un)fortunately this is a CI/CD pipeline, the VM and its data will be
gone forever after the build. Unless I care to keep apt output as an
artifact somewhere which is IMHO an overkill. I just want an concise
CI/CD log without interactive bells and whistles like progress
indicators.



Without seeing your CI/CD pipeine -- create a patterns file containing 
regular expressions that match the lines you want removed, then add a 
grep(1) filter into the pipeline:


2022-08-02 16:58:46 dpchrist@laalaa ~
$ cat remove-patterns.txt
bar
quux

2022-08-02 16:58:50 dpchrist@laalaa ~
$ echo -e "foo\nbar\nbaz\nquux" | egrep -v -f remove-patterns.txt
foo
baz


David

Re: apt-get -qq install

[Victor Sudakov]

David Christensen wrote:
> > 
> > I don't actually like the idea of redirecting apt-get's output to
> > /dev/null because I want to see the list of packages installed, but without
> > these pseudographics. Do you think it's possible? Any ideas?
> 
> 
> I prefer the idea of collecting all of the data and then writing queries 
> and reports to print the interesting bits.  If you keep all of the data, 
> you can apply future queries and reports to past data.  But if you 
> filter the data up front and save one report format, your future options 
> are limited.

(Un)fortunately this is a CI/CD pipeline, the VM and its data will be
gone forever after the build. Unless I care to keep apt output as an
artifact somewhere which is IMHO an overkill. I just want an concise
CI/CD log without interactive bells and whistles like progress
indicators.

-- 
Victor Sudakov VAS4-RIPE
http://vas.tomsk.ru/
2:5005/49@fidonet


signature.asc
Description: PGP signature

Re: apt-get -qq install

[Victor Sudakov]

davidson wrote:
> >
> > I'm trying to quiet apt's output by using `apt-get -qqy` in a CI/CD
> > pipeline, however I still see ugly stuff like this in my CI/CD log:
> >
> > Selecting previously unselected package php-common.
> > (Reading database ...
> > (Reading database ... 5%
> > (Reading database ... 10%
> [snip]
> >
> > and this:
> >
> > Scanning processes... [ 
> >]
> > Scanning processes... [ 
> >]
> > Scanning processes... [ 
> >]
> > Scanning processes... [=
> >]
> > Scanning processes... [=
> >]
> > Scanning processes... [==   
> >]
> > Scanning processes... [==   
> >]
> [snip]
> 
> > Don't you think the `-qq` modifier should spare me these things?
> 
> I would expect so too.
> 
> > I don't actually like the idea of redirecting apt-get's output to
> > /dev/null because I want to see the list of packages installed, but without
> > these pseudographics. Do you think it's possible? Any ideas?
> 
> In the file
> 
>   /usr/share/doc/apt/examples/configure-index.gz
> 
> I notice there is a stanza that begins like so:
> 
>   quiet "" {
> NoUpdate ""; // never update progress information - included in -q=1
> NoProgress ""; // disables the 0% -> 100% progress on cache 
> generation and stuff
> 
> I don't know whether this will help you, but it does look suggestive.

Thank you, how do you activate this option? I've just tried 
`apt-get -o quiet::NoProgress=true -qqy ...` but the "Reading database ... 5%"
stuff is still there.


-- 
Victor Sudakov VAS4-RIPE
http://vas.tomsk.ru/
2:5005/49@fidonet


signature.asc
Description: PGP signature

Re: apt-get -qq install

[David Christensen]

On 8/2/22 11:27, Victor Sudakov wrote:

Dear Colleagues,

I'm trying to quiet apt's output by using `apt-get -qqy` in a CI/CD
pipeline, however I still see ugly stuff like this in my CI/CD log:

Selecting previously unselected package php-common.
(Reading database ...
(Reading database ... 5%
(Reading database ... 10%
(Reading database ... 15%
(Reading database ... 20%
(Reading database ... 25%
(Reading database ... 30%
(Reading database ... 35%
(Reading database ... 40%
(Reading database ... 45%
(Reading database ... 50%
(Reading database ... 55%
(Reading database ... 60%
(Reading database ... 65%
(Reading database ... 70%

and this:

Scanning processes... []
Scanning processes... []
Scanning processes... []
Scanning processes... [=   ]
Scanning processes... [=   ]
Scanning processes... [==  ]
Scanning processes... [==  ]
Scanning processes... [=== ]
Scanning processes... [=== ]
Scanning processes... []
Scanning processes... []
Scanning processes... [=   ]
Scanning processes... [=   ]
Scanning processes... [==  ]
Scanning processes... [==  ]
Scanning processes... [=== ]
Scanning processes... [=== ]
Scanning processes... []
Scanning processes... []
Scanning processes... [=   ]
Scanning processes... [=   ]

Don't you think the `-qq` modifier should spare me these things?

I don't actually like the idea of redirecting apt-get's output to
/dev/null because I want to see the list of packages installed, but without
these pseudographics. Do you think it's possible? Any ideas?



I prefer the idea of collecting all of the data and then writing queries 
and reports to print the interesting bits.  If you keep all of the data, 
you can apply future queries and reports to past data.  But if you 
filter the data up front and save one report format, your future options 
are limited.



David

Re: apt-get -qq install

[davidson]

On Tue, 2 Aug 2022 Victor Sudakov wrote:

Dear Colleagues,

I'm trying to quiet apt's output by using `apt-get -qqy` in a CI/CD
pipeline, however I still see ugly stuff like this in my CI/CD log:

Selecting previously unselected package php-common.
(Reading database ...
(Reading database ... 5%
(Reading database ... 10%

[snip]


and this:

Scanning processes... []
Scanning processes... []
Scanning processes... []
Scanning processes... [=   ]
Scanning processes... [=   ]
Scanning processes... [==  ]
Scanning processes... [==  ]

[snip]


Don't you think the `-qq` modifier should spare me these things?


I would expect so too.


I don't actually like the idea of redirecting apt-get's output to
/dev/null because I want to see the list of packages installed, but without
these pseudographics. Do you think it's possible? Any ideas?


In the file

 /usr/share/doc/apt/examples/configure-index.gz

I notice there is a stanza that begins like so:

 quiet "" {
   NoUpdate ""; // never update progress information - included in -q=1
   NoProgress ""; // disables the 0% -> 100% progress on cache generation 
and stuff

I don't know whether this will help you, but it does look suggestive.

--
Ce qui est important est rarement urgent
et ce qui est urgent est rarement important
-- Dwight David Eisenhower

Re: apt-get -qq install

[Victor Sudakov]

David Wright wrote:
> On Tue 02 Aug 2022 at 18:27:22 (+), Victor Sudakov wrote:
> > I'm trying to quiet apt's output by using `apt-get -qqy` in a CI/CD
> > pipeline, however I still see ugly stuff like this in my CI/CD log:
> > 
> > Selecting previously unselected package php-common.
> > (Reading database ... 
> > (Reading database ... 5%
> [ … ]
> > 
> > and this:
> > 
> > Scanning processes... [ 
> >]
> > Scanning processes... [=
> >]
> [ … ]
> > 
> > Don't you think the `-qq` modifier should spare me these things? 
> > 
> > I don't actually like the idea of redirecting apt-get's output to
> > /dev/null because I want to see the list of packages installed, but without
> > these pseudographics. Do you think it's possible? Any ideas?
> 
> The packages installed and the terminal output are logged in
> /var/log/apt/{history,term}.log respectively (which I never rotate).

Which will be lost in a CI/CD environment because the VM image is ephemeral.

BTW I've tried setting `sudo DEBIAN_FRONTEND=noninteractive apt-get -qqy 
--no-install-recommends install ...` and 
`sudo apt-get -qqy -o Dpkg::Use-Pty=0 ...` to no avail, the rubbish is still 
there.

-- 
Victor Sudakov VAS4-RIPE
http://vas.tomsk.ru/
2:5005/49@fidonet


signature.asc
Description: PGP signature

Re: apt-get -qq install

[David Wright]

On Tue 02 Aug 2022 at 18:27:22 (+), Victor Sudakov wrote:
> I'm trying to quiet apt's output by using `apt-get -qqy` in a CI/CD
> pipeline, however I still see ugly stuff like this in my CI/CD log:
> 
> Selecting previously unselected package php-common.
> (Reading database ... 
> (Reading database ... 5%
[ … ]
> 
> and this:
> 
> Scanning processes... [   
>  ]
> Scanning processes... [=  
>  ]
[ … ]
> 
> Don't you think the `-qq` modifier should spare me these things? 
> 
> I don't actually like the idea of redirecting apt-get's output to
> /dev/null because I want to see the list of packages installed, but without
> these pseudographics. Do you think it's possible? Any ideas?

The packages installed and the terminal output are logged in
/var/log/apt/{history,term}.log respectively (which I never rotate).

Cheers,
David.

apt-get -qq install

[Victor Sudakov]

Dear Colleagues,

I'm trying to quiet apt's output by using `apt-get -qqy` in a CI/CD
pipeline, however I still see ugly stuff like this in my CI/CD log:

Selecting previously unselected package php-common.
(Reading database ... 
(Reading database ... 5%
(Reading database ... 10%
(Reading database ... 15%
(Reading database ... 20%
(Reading database ... 25%
(Reading database ... 30%
(Reading database ... 35%
(Reading database ... 40%
(Reading database ... 45%
(Reading database ... 50%
(Reading database ... 55%
(Reading database ... 60%
(Reading database ... 65%
(Reading database ... 70%

and this:

Scanning processes... []
Scanning processes... []
Scanning processes... []
Scanning processes... [=   ]
Scanning processes... [=   ]
Scanning processes... [==  ]
Scanning processes... [==  ]
Scanning processes... [=== ]
Scanning processes... [=== ]
Scanning processes... []
Scanning processes... []
Scanning processes... [=   ]
Scanning processes... [=   ]
Scanning processes... [==  ]
Scanning processes... [==  ]
Scanning processes... [=== ]
Scanning processes... [=== ]
Scanning processes... []
Scanning processes... []
Scanning processes... [=   ]
Scanning processes... [=   ]

Don't you think the `-qq` modifier should spare me these things? 

I don't actually like the idea of redirecting apt-get's output to
/dev/null because I want to see the list of packages installed, but without
these pseudographics. Do you think it's possible? Any ideas?

-- 
Victor Sudakov VAS4-RIPE
http://vas.tomsk.ru/
2:5005/49@fidonet


signature.asc
Description: PGP signature

Re: How can I find out why apt-get is keeping a package back?

[Cindy Sue Causey]

On 7/10/22, David Wright  wrote:
> On Sun 10 Jul 2022 at 15:24:11 (-0700), L L wrote:
>> How can I find out why apt-get is keeping a package back?
>
> I usually look at the output of  apt-cache show .

I accidentally stumbled upon that I can "apt-get upgrade "
and see what is likely the trigger for held packages. For me, it
always hesitates and waits for approval. Most often, packages I've
seen are being held back because of an upgrade to the next versioning
plateau or because completely brand new packages are being installed
along with.

Being able to "-s" simulate with apt-get is something I always forget
exists. I just test drove it, and it worked for upgrade in general:

apt-get upgrade -s

There wasn't anything held back this time so I can only assume that
simulate would additionally work for:

apt-get upgrade  -s

That can be run just like that, as regular User instead of root.

Simulation doesn't give the 100% full picture. It might not generate
enough focus on the held package to answer why it's being held. May be
why I don't remember to use it. I played with simulation a couple
times, and it's a cute trick. At the end of the day, though, I just go
straight for the real upgrade. For me so far, that route has been
informative and has worked safely #1 because it doesn't continue until
I hit the ENTER key again. :)

Cindy :)
-- 
Talking Rock, Pickens County, Georgia, USA
* runs with birdseed *

Re: How can I find out why apt-get is keeping a package back?

[David Wright]

On Sun 10 Jul 2022 at 15:24:11 (-0700), L L wrote:
> How can I find out why apt-get is keeping a package back?

I usually look at the output of  apt-cache show .
For example, after running apt-get update, I currently have
linux-image-amd64 held back. Thus:

$ apt-cache show linux-image-amd64
Package: linux-image-amd64
Source: linux-signed-amd64 (5.10.127+1)
Version: 5.10.127-1
Provides: linux-image-generic, linux-latest-modules-5.10.0-16-amd64, 
virtualbox-guest-modules, wireguard-modules (= 1.0.0)
Depends: linux-image-5.10.0-16-amd64 (= 5.10.127-1)
Description-en: Linux for 64-bit PCs (meta-package)
 This package depends on the latest Linux kernel and modules for use on PCs
 with AMD64, Intel 64 or VIA Nano processors.
Built-Using: linux (= 5.10.127-1)
Filename: pool/main/l/linux-signed-amd64/linux-image-amd64_5.10.127-1_amd64.deb

Package: linux-image-amd64
Source: linux-signed-amd64 (5.10.120+1)
Version: 5.10.120-1
Provides: linux-image-generic, linux-latest-modules-5.10.0-15-amd64, 
virtualbox-guest-modules, wireguard-modules (= 1.0.0)
Depends: linux-image-5.10.0-15-amd64 (= 5.10.120-1)
Description-en: Linux for 64-bit PCs (meta-package)
 This package depends on the latest Linux kernel and modules for use on PCs
 with AMD64, Intel 64 or VIA Nano processors.
Built-Using: linux (= 5.10.120-1)
Filename: 
pool/updates/main/l/linux-signed-amd64/linux-image-amd64_5.10.120-1_amd64.deb

$ 

(with many uninteresting lines snipped).

Its Depends is currently satisfied by linux-image-5.10.0-15-amd64
(installed), but it could be upgraded by installing the new package,
linux-image-5.10.0-16-amd64, which would require running
# apt-get dist-upgrade.

Cheers,
David.

How can I find out why apt-get is keeping a package back?

[L L]

How can I find out why apt-get is keeping a package back?

Re: Suggestions for rm [WAS: Re: Feature request: install package by passing URL to apt-get]

[Jonathan Dowland]

On Fri, Jun 24, 2022 at 07:02:35AM +, Andrew M.A. Cater wrote:

Train your brain and your fingers to move the rf to the end of the command so
that you _have_ to check what filename you are typing as you type it.


I set a shell alias

alias rm='echo use trash instead'

This was enough to train me out of typing "rm" in most situations and
instead use 'trash':

  https://tracker.debian.org/pkg/trash-cli

Another advantage of trash-cli, as well as the safety measure, is
'trashing' large trees is much faster than deleting them.


--
Please do not CC me for listmail.

  Jonathan Dowland
✎j...@debian.org
   https://jmtd.net

Re: Suggestions for rm [WAS: Re: Feature request: install package by passing URL to apt-get]

[Joel Roth]

On Sat, Jun 25, 2022 at 09:45:58AM -0400, Greg Wooledge wrote:
> On Fri, Jun 24, 2022 at 07:42:25PM -1000, Joel Roth wrote:
> > I list the files first:
> > 
> > ls some-pattern
> > 
> > then add a pipe to rm:
> > 
> > ls some-pattern | rm 
> > 
> > or
> > 
> > ls some-pattern | rm -rf
> 
> Those commands do not work.  rm does not read a list of files from stdin.
> 
> Even if you were to add xargs, those commands still would not work in
> all cases.  They would only work in the simplest cases, where none of
> the filenames contain whitespace, single-quote characters, or double-quote
> characters.
> 
> That's because xargs does not split its input on newlines.  It splits
> its input on "quoted words".  Like this:
> 
> unicorn:~$ echo 'a list of "quoted words"' | xargs printf '<%s>\n'
> 
> 
> 
> 
> 
> Any unbalanced single or double quote will cause an error:
> 
> unicorn:~$ echo "I don't know.mp3" | xargs printf '<%s>\n'
> xargs: unmatched single quote; by default quotes are special to xargs unless 
> you use the -0 option
> 
> 
> And even if xargs *did* have some option to split its input only on
> newlines, a filename that *contains* a newline would still break it.
> 
> Finally, ls does not always reproduce filenames exactly.  There are
> some systems (I'm not sure about all versions of Debian, but definitely
> some Unix systems) where certain characters will be printed as
> question marks by ls.  That means any file containing one of those
> characters would make the ls|xargs rm construct fail.
> 
> If you want to use ls as a preview for what will be removed, that's
> perfectly fine.  You just can't use "up arrow | xargs rm" as your
> follow-up command.  Instead, use "up arrow" and then use command editing
> to replace the ls with rm.

I was actually using 'xargs rm' in an alias. Thanks for
pointing out the limitations. 

-- 
Joel Roth

Re: Suggestions for rm [WAS: Re: Feature request: install package by passing URL to apt-get]

[Greg Wooledge]

On Fri, Jun 24, 2022 at 07:42:25PM -1000, Joel Roth wrote:
> I list the files first:
> 
> ls some-pattern
> 
> then add a pipe to rm:
> 
> ls some-pattern | rm 
> 
> or
> 
> ls some-pattern | rm -rf

Those commands do not work.  rm does not read a list of files from stdin.

Even if you were to add xargs, those commands still would not work in
all cases.  They would only work in the simplest cases, where none of
the filenames contain whitespace, single-quote characters, or double-quote
characters.

That's because xargs does not split its input on newlines.  It splits
its input on "quoted words".  Like this:

unicorn:~$ echo 'a list of "quoted words"' | xargs printf '<%s>\n'





Any unbalanced single or double quote will cause an error:

unicorn:~$ echo "I don't know.mp3" | xargs printf '<%s>\n'
xargs: unmatched single quote; by default quotes are special to xargs unless 
you use the -0 option


And even if xargs *did* have some option to split its input only on
newlines, a filename that *contains* a newline would still break it.

Finally, ls does not always reproduce filenames exactly.  There are
some systems (I'm not sure about all versions of Debian, but definitely
some Unix systems) where certain characters will be printed as
question marks by ls.  That means any file containing one of those
characters would make the ls|xargs rm construct fail.

If you want to use ls as a preview for what will be removed, that's
perfectly fine.  You just can't use "up arrow | xargs rm" as your
follow-up command.  Instead, use "up arrow" and then use command editing
to replace the ls with rm.

Re: Suggestions for rm [WAS: Re: Feature request: install package by passing URL to apt-get]

[Joel Roth]

On Fri, Jun 24, 2022 at 07:02:35AM +, Andrew M.A. Cater wrote:
> There are a couple of useful habits to get into when removing things:
> 
> There's an 
> 
>  rm -i
 
> Use the pwd  command to check where you are in the
> filesystem. (It may be short for "print working
> directory").
 
> If you are deleting one file - change to the directory it is in, check that
> it exists there first with the
> 
>  ls -al [filename]
> 
> command. Since the file is in the current directory,you can use the
> 
>  rm ./[filename]
> 
> [That's a period and a forward slash - limiting you to a file in the current
> directory]
> 
> Try and avoid using rm -rf and forced removal. The one exception is that
> you have to remove a non-empty directory with -rf. If you first try -r
> and it fails, that's a clue that you are actually about to delete a 
> directory.
> 
> Again, if it's a single directory, change directories to the directory
> that it is in and use the ./ The last suggestion, and it's the simplest:
> 
>  rm [filename] -rf
... 
> If you need to be doubly sure rm [filename] -irf will put in the interactive
> prompt again.
> 
> Some of this is learnt the hard way from administering one machine that
> other people relied on :)

All good suggestions, along with making backups. 

I list the files first:

ls some-pattern

then add a pipe to rm:

ls some-pattern | rm 

or

ls some-pattern | rm -rf

I know the OP said they weren't asking advice, but I can't
help putting in my two bits :-)

cheers,






> With every good wish, as ever,
> 

-- 
Joel Roth

Re: Feature request: install package by passing URL to apt-get

[Bijan Soleymani]

On 6/24/2022 2:24 PM, David Wright wrote:

This strikes me as a nightmare waiting to happen.

There was probably a similar feeling when .deb and dpkg and apt rolled out.

I think you made this up. I'm sure there were some people who were
happy enough with Slackware when dpkg was evolving, but I never read
of anyone using Debian who would have preferred dpkg not to exist.
As for APT, well, using dselect was getting gradually less able to
cope as the number of packages increased into the thousands.
Effectively, the apt method in dselect came to its rescue. The idea
of dist-upgrading with dselect is "Aaargh".


Thanks!

Sorry, my memory of the state of things around Potato when I started 
with Debian is very hazy. I remember dselect and a lot of fumbling.


Bijan

Re: Feature request: install package by passing URL to apt-get

[David Wright]

On Wed 22 Jun 2022 at 18:21:10 (-0400), Bijan Soleymani wrote:
> On 6/22/2022 5:22 PM, Charles Curley wrote:
> > On Wed, 22 Jun 2022 16:25:02 -0400 Bijan Soleymani wrote:
> > 
> > > Anyways it would be nice to do:
> > > 
> > > apt install http://domain/filename.deb

> > This strikes me as a nightmare waiting to happen.
> 
> There was probably a similar feeling when .deb and dpkg and apt rolled out.

I think you made this up. I'm sure there were some people who were
happy enough with Slackware when dpkg was evolving, but I never read
of anyone using Debian who would have preferred dpkg not to exist.
As for APT, well, using dselect was getting gradually less able to
cope as the number of packages increased into the thousands.
Effectively, the apt method in dselect came to its rescue. The idea
of dist-upgrading with dselect is "Aaargh".

Cheers,
David.

Re: Suggestions for rm [WAS: Re: Feature request: install package by passing URL to apt-get]

[David Wright]

On Fri 24 Jun 2022 at 08:48:05 (-0400), rhkra...@gmail.com wrote:
> On Friday, June 24, 2022 03:02:35 AM Andrew M.A. Cater wrote:
> > The last suggestion, and it's the simplest:
> > 
> >  rm [filename] -rf
> > 
> > Train your brain and your fingers to move the rf to the end of the command
> > so that you _have_ to check what filename you are typing as you type it.
> > 
> > If you need to be doubly sure rm [filename] -irf will put in the
> > interactive prompt again.
> 
> I like all these suggestions, as well as Thomas' addition (although I'm not 
> sure I'm ambitious enough to go back and edit the comand buffer).

Ambitious? Line editing and command history are two of the most useful
things about working at the command line.

And I recommend that you train your fingers to type rm -i (similarly
for cp and mv) rather than relying on the system to add -i for you.
You can thereby avoid a lot of harm -i to your system (oops).

For mass deletions, I would:

Use rm -i … and check that the filenames being reflected are from the
tree you expected.

Recall the command, then first add a space at the start of the line,
and next, remove the -i. The space prevents the potent version of the
command (without the -i) being added to your history when you press
Return.¹

> I do something slightly different than what Andrew suggests, I put the 
> filespec 
> on the blank command line, and then go back and insert the rm -rf before the 
> filespec.

Slightly different, but I use this with other potent commands, like dd:

Start the command line with # space, followed by the command and its
arguments. Study what you have written, then delete the # (but leave
the space: you don't want such potent commands left in your history).

¹ You don't want to be able to later recall this command accidentally,
  in a different context. So put   export HISTCONTROL=ignoreboth
  into your .bashrc file, preventing lines starting with a space from
  being added to your command history.

Cheers,
David.

Re: Feature request: install package by passing URL to apt-get

[Bijan Soleymani]

On 6/24/2022 10:03 AM, Bijan Soleymani wrote:


To me anytime you need to use su or sudo to do something, you should 
not think it is harmless, no matter how simple or easy. 


Although all that being said, I remember a long time ago getting into a 
situation with apt or dpkg (removing libc or installing an incompatible 
libc), where I got a message, along the lines of:


Warning you're probably about to destroy your system. Please literally type:

Yes, do what I say!

to proceed.

:)

Bijan

Re: Feature request: install package by passing URL to apt-get

[Bijan Soleymani]

Hi Didier,

On 6/24/2022 4:36 AM, didier gaumet wrote:

Le jeudi 23 juin 2022 à 16:27 -0400, Bijan Soleymani a écrit :

rm wrongfile

how do I undelete?

better put deleted files in the "recycling box" and prompt users on
every deletion by default

rm -rf /

oops!

sigh...

Bijan

Hello,

Please open a new thread when you have a question, do not answer to another 
one's post to ask it :-)

I have never used it but basically just use commands from the trash-cli package 
instead of rm.


I definitely appreciate all the helpful replies I received, but my message was 
mainly a comment and not a request for help.

My point was that Unix/Linux has a lot of sharp edges where the user can hurt 
themselves if not careful.

And when doing system administration tasks it is routine to run commands under 
sudo (and sudo caches the password).

And then if you do:
sudo rm -rf $x/$y and x and y are not defined...

Or you mean to format a bootable usb and use: dd but end up with the wrong 
output device...

Anyways it was meant as a response to op, who said:
On 6/23/2022 12:03 PM, Person the human wrote:


The easier something is to do, the more harmless people will think it 
is, so you're right. Thanks.


To me anytime you need to use su or sudo to do something, you should not think 
it is harmless, no matter how simple or easy.

Bijan

Re: Feature request: install package by passing URL to apt-get

[Greg Wooledge]

On Fri, Jun 24, 2022 at 09:17:40AM -0400, Bijan Soleymani wrote:
> On 6/24/2022 4:48 AM, Jonathan Dowland wrote:
> > On Wed, Jun 22, 2022 at 03:22:30PM -0600, Charles Curley wrote:
> > > How do you handle dependencies where there is a version of the
> > > dependency on the server, and another version on a repo in the user's
> > > sources.list? Multiple nested dependencies?
> 
> Apparently some packages just add their apt sources to /etc/apt/ during the
> install of their .deb
> 
> (I think google chrome does that)

Indeed it does.

unicorn:~$ cat /etc/apt/sources.list.d/google-chrome.list 
### THIS FILE IS AUTOMATICALLY CONFIGURED ###
# You may comment out this entry, but any other modifications may be lost.
deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main

Re: Feature request: install package by passing URL to apt-get

[Bijan Soleymani]

On 6/24/2022 4:01 AM, Darac Marjal wrote:


On 22/06/2022 18:04, Person the human wrote:
I just want to get everyone's opinion on this before I request it 
from the developers or possibly try to add it myself.


Would it be nice if it was possible to pass a URL to 'apt install' so 
that a package could be installed without first downloading its .deb 
file? I think it's good because it can save time and prevent 
unneeded damage to SSDs. Even if you don't have any info to add, 
please let me know what you think.


Our cousins Ubuntu already have this capability since 7.10: 
https://wiki.ubuntu.com/AptUrl


The fact that this functionality has been available for ~15 years but 
hasn't made it upstream into Debian suggests that Debian is not 
interested in such functionality.


From playing with that it looks like that triggers:

apt install package

(install from the apt sources the user already has, there's syntax to 
trigger a refresh, but I don't think you can add random apt sources or URLs)


I wondered why if Ubuntu has it, practically no 3rd party software 
distributors/developers were using it.


Bijan

Re: Feature request: install package by passing URL to apt-get

[Bijan Soleymani]

On 6/24/2022 4:48 AM, Jonathan Dowland wrote:

On Wed, Jun 22, 2022 at 03:22:30PM -0600, Charles Curley wrote:

How do you handle dependencies where there is a version of the
dependency on the server, and another version on a repo in the user's
sources.list? Multiple nested dependencies?


You don't: just like "apt install ./foo.deb", apt takes its existing
package database, integrates knowledge of foo.deb, and nothing else.
If there are adjacent packages to https://example.com/foo.deb, apt
shouldn't know anything about them, IMHO.


Apparently some packages just add their apt sources to /etc/apt/ during 
the install of their .deb


(I think google chrome does that)

Re: Suggestions for rm [WAS: Re: Feature request: install package by passing URL to apt-get]

[rhkramer]

On Friday, June 24, 2022 03:02:35 AM Andrew M.A. Cater wrote:
> The last suggestion, and it's the simplest:
> 
>  rm [filename] -rf
> 
> Train your brain and your fingers to move the rf to the end of the command
> so that you _have_ to check what filename you are typing as you type it.
> 
> If you need to be doubly sure rm [filename] -irf will put in the
> interactive prompt again.

I like all these suggestions, as well as Thomas' addition (although I'm not 
sure I'm ambitious enough to go back and edit the comand buffer).

I do something slightly different than what Andrew suggests, I put the filespec 
on the blank command line, and then go back and insert the rm -rf before the 
filespec.

My approach might be slightly safer -- there is less possibility of deleting 
one (or more than one) (wrong) file by fat fingering something (hitting enter) 
(I'm good at that :-( before getting the filespec correct.

For example, suppose you have files test and test1, and you want to delete 
test1 but not test -- if you accidentally hit enter after typing test but 
before typing the 1 you will be disappointed.

-- 
If you reply: snip, snip, and snip again; leave attributions; avoid top 
posting.

A picture is worth a thousand words -- divide by 10 for each minute of video 
(or audio) or create a transcript and edit it to 10% of the original.  (Oxford 
comma included at no charge.)

Re: Feature request: install package by passing URL to apt-get

[didier gaumet]

Le jeudi 23 juin 2022 à 16:27 -0400, Bijan Soleymani a écrit :
> 
> rm wrongfile
> 
> how do I undelete?
> 
> better put deleted files in the "recycling box" and prompt users on 
> every deletion by default
> 
> rm -rf /
> 
> oops!
> 
> sigh...
> 
> Bijan

Hello,

Please open a new thread when you have a question, do not answer to another 
one's post to ask it :-)

I have never used it but basically just use commands from the trash-cli package 
instead of rm.

Re: Feature request: install package by passing URL to apt-get

[Jonathan Dowland]

On Wed, Jun 22, 2022 at 03:22:30PM -0600, Charles Curley wrote:

How do you handle dependencies where there is a version of the
dependency on the server, and another version on a repo in the user's
sources.list? Multiple nested dependencies?


You don't: just like "apt install ./foo.deb", apt takes its existing
package database, integrates knowledge of foo.deb, and nothing else.
If there are adjacent packages to https://example.com/foo.deb, apt
shouldn't know anything about them, IMHO.


--
Please do not CC me for listmail.

  Jonathan Dowland
✎j...@debian.org
   https://jmtd.net

Re: Feature request: install package by passing URL to apt-get

[Darac Marjal]

On 22/06/2022 18:04, Person the human wrote:
I just want to get everyone's opinion on this before I request it from 
the developers or possibly try to add it myself.


Would it be nice if it was possible to pass a URL to 'apt install' so 
that a package could be installed without first downloading its .deb 
file? I think it's good because it can save time and prevent 
unneeded damage to SSDs. Even if you don't have any info to add, 
please let me know what you think.


Our cousins Ubuntu already have this capability since 7.10: 
https://wiki.ubuntu.com/AptUrl


The fact that this functionality has been available for ~15 years but 
hasn't made it upstream into Debian suggests that Debian is not 
interested in such functionality.




OpenPGP_signature
Description: OpenPGP digital signature

Suggestions for rm [WAS: Re: Feature request: install package by passing URL to apt-get]

[Andrew M.A. Cater]

On Thu, Jun 23, 2022 at 04:27:28PM -0400, Bijan Soleymani wrote:
> On 6/23/2022 12:03 PM, Person the human wrote:
> > The easier something is to do, the more harmless people will think it
> > is, so you're right. Thanks.
> 
> rm wrongfile
> 
> how do I undelete?
> 
> better put deleted files in the "recycling box" and prompt users on every
> deletion by default
> 
> rm -rf /
> 
> oops!
> 
> sigh...
> 
> Bijan
> 
>

For the general "I rm'd something critical" - there is no absolute solution.

Root/superuser privileges can do almost anything and everyone makes that
mistake once.

There are a couple of useful habits to get into when removing things:

There's an 

 rm -i

switch to the rm command. This makes the removal interactive - you get 
an "Are you sure [Y/N]" warning for each file. This is useful for a single 
critical file, less useful when you're removing a hundred logfiles.
On a Red Hat-derived system, this is often aliased to rm by default.
That's a great idea - until you move to a system where rm means rm immediately.

Use the

 pwd

 command to check where you are in the filesystem. (It may be short 
for "print working directory").

If you are deleting one file - change to the directory it is in, check that
it exists there first with the

 ls -al [filename]

command. Since the file is in the current directory,you can use the

 rm ./[filename]

[That's a period and a forward slash - limiting you to a file in the current
directory]

Try and avoid using rm -rf and forced removal. The one exception is that
you have to remove a non-empty directory with -rf. If you first try -r
and it fails, that's a clue that you are actually about to delete a 
directory.

Again, if it's a single directory, change directories to the directory
that it is in and use the ./ The last suggestion, and it's the simplest:

 rm [filename] -rf

Train your brain and your fingers to move the rf to the end of the command so 
that you _have_ to check what filename you are typing as you type it.

If you need to be doubly sure rm [filename] -irf will put in the interactive
prompt again.

Some of this is learnt the hard way from administering one machine that
other people relied on :)

With every good wish, as ever,

Re: Feature request: install package by passing URL to apt-get

[Charles Curley]

On Thu, 23 Jun 2022 16:27:28 -0400
Bijan Soleymani  wrote:

> better put deleted files in the "recycling box" and prompt users on 
> every deletion by default
> 
> rm -rf /
> 
> oops!

Better yet, have proper backups.

https://charlescurley.com/blog/posts/2019/Nov/02/backups-on-linux/

-- 
Does anybody read signatures any more?

https://charlescurley.com
https://charlescurley.com/blog/

Re: Feature request: install package by passing URL to apt-get

[Dan Ritter]

Bijan Soleymani wrote: 
> On 6/23/2022 12:03 PM, Person the human wrote:
> > The easier something is to do, the more harmless people will think it
> > is, so you're right. Thanks.
> 
> rm wrongfile
> 
> how do I undelete?
> 
> better put deleted files in the "recycling box" and prompt users on every
> deletion by default
> 
> rm -rf /
> 
> oops!
> 
> sigh...

Luckily, you weren't root and you didn't use sudo or doas, so
what came back was

rm: failed to remove '/': Permission denied

But sometimes that happens. All we can do is not make it
extremely easy for people to do terrible things by accident
because it looked like a good thing to do.

And people learn from experience, but it's really hard to learn
from experience when the mistake you made happened six days ago
as you tried to install a great game you read about, and now
your ISP has shut your connection off.

-dsr-

Re: Feature request: install package by passing URL to apt-get

[Charles Curley]

On Wed, 22 Jun 2022 14:34:41 -0230
Person the human  wrote:

> Would it be nice if it was possible to pass a URL to 'apt install' so
> that a package could be installed without first downloading its .deb
> file? I think it's good because it can save time and prevent unneeded
> damage to SSDs. Even if you don't have any info to add, please let me
> know what you think.

There is an extension for Chromium called Apt-linker which may do some
of what you want. The description is "Turns "apt-get install" lines
into clickable apturl links for installing Ubuntu and Debian GNU/Linux
packages".

-- 
Does anybody read signatures any more?

https://charlescurley.com
https://charlescurley.com/blog/

Re: Feature request: install package by passing URL to apt-get

[Bijan Soleymani]

On 6/23/2022 12:03 PM, Person the human wrote:
The easier something is to do, the more harmless people will think it 
is, so you're right. Thanks.


rm wrongfile

how do I undelete?

better put deleted files in the "recycling box" and prompt users on 
every deletion by default


rm -rf /

oops!

sigh...

Bijan

Re: Feature request: install package by passing URL to apt-get

[Person the human]

The easier something is to do, the more harmless people will think it is,
so you're right. Thanks.

On Wed, Jun 22, 2022 at 5:56 PM Dan Ritter  wrote:

> Person the human wrote:
> > Having this feature won't help me personally since I'm not using an SSD
> and
> > I don't download packages from the internet often. The reason I wanted
> this
> > to be built into apt is so that websites that would normally link to a
> file
> > and tell users to download it could instead just give users a command to
> > copy-paste into the terminal. It would help all users and change the way
> > people use apt.
>
> That would be terrible.
>
> The last thing Debian needs is people randomly installing things
> that a website told them to install. That's how you get
> malware-infested FrankenDebian.
>
> -dsr-
>

Re: Feature request: install package by passing URL to apt-get

[tomas]

On Wed, Jun 22, 2022 at 06:21:10PM -0400, Bijan Soleymani wrote:

[...]

> > This strikes me as a nightmare waiting to happen.
> 
> There was probably a similar feeling when .deb and dpkg and apt rolled out.
> Users are going to have dependency issues, etc. If libc breaks on upgrade 
> then everything dies, etc.

Actually, no. Because the libc package will be tested once it
arrives at your box (messups do happen, but way less frequently).

I think people underestimate the main service a (good) distribution
offers: a single source of trust. A group of people with a more or
less common set of processes, values, etc. trying to put together
a more or less consistent set of things. When the idea about how to
do things change, there is (sometimes heated) discussion, so you as
user can follow along (or change distro, if you don't like the
outcome).

Look at other package managers which only pose as a technical service
(say, PHP or npm). With npm particularly, it seems a package gets
taken over by a malicious actor every other week or so.

Besides, you /can/ set up a Debian package repo "out there" people
can add to their sources.list. After my past experiences with
debian-multimedia, I'll make sure I really want that.

Cheers
-- 
t


signature.asc
Description: PGP signature

Re: Feature request: install package by passing URL to apt-get

[Bijan Soleymani]

On 6/22/2022 5:22 PM, Charles Curley wrote:

On Wed, 22 Jun 2022 16:25:02 -0400
Bijan Soleymani  wrote:


Anyways it would be nice to do:

apt install http://domain/filename.deb

too. Wonder if there's some optimal/convenient way to handle
dependencies if they're on the same server too.

How do you handle dependencies where there is a version of the
dependency on the server, and another version on a repo in the user's
sources.list? Multiple nested dependencies?


I guess the same way you handle multiple apt sources, based on 
dependency versions declared in the deb and the priority of the sources.


Most packages with complex dependencies or that update often make the 
user edit their apt sources anyways.



Don't forget the public key for the packager, if any.


That could be handled as part of the process.

apt could prompt:

have you verified X signing key

do you want to add Y apt source

vs now users cut and paste:

echo "signing-key" >> /etc/apt/some-keyring.txt

echo "apt-source-url" >> /etc/apt/sources.list

from the developer's website


This strikes me as a nightmare waiting to happen.


There was probably a similar feeling when .deb and dpkg and apt rolled out.
Users are going to have dependency issues, etc. If libc breaks on upgrade then 
everything dies, etc.

Bijan

Re: Feature request: install package by passing URL to apt-get

[Charles Curley]

On Wed, 22 Jun 2022 16:25:02 -0400
Bijan Soleymani  wrote:

> Anyways it would be nice to do:
> 
> apt install http://domain/filename.deb
> 
> too. Wonder if there's some optimal/convenient way to handle 
> dependencies if they're on the same server too.

How do you handle dependencies where there is a version of the
dependency on the server, and another version on a repo in the user's
sources.list? Multiple nested dependencies?

Don't forget the public key for the packager, if any.

This strikes me as a nightmare waiting to happen.


-- 
Does anybody read signatures any more?

https://charlescurley.com
https://charlescurley.com/blog/

Re: Feature request: install package by passing URL to apt-get

[Dan Ritter]

Person the human wrote: 
> Having this feature won't help me personally since I'm not using an SSD and
> I don't download packages from the internet often. The reason I wanted this
> to be built into apt is so that websites that would normally link to a file
> and tell users to download it could instead just give users a command to
> copy-paste into the terminal. It would help all users and change the way
> people use apt.

That would be terrible.

The last thing Debian needs is people randomly installing things
that a website told them to install. That's how you get
malware-infested FrankenDebian.

-dsr-