[Touch-packages] [Bug 1908473] Re: rsyslog-relp: imrelp module leaves sockets in CLOSE_WAIT state which leads to file descriptor leak

2021-01-25 Thread Matthew Ruffell
Performing verification for Focal.

I installed rsyslog-relp 8.2001.0-1ubuntu1.1 and librelp0 1.5.0-1ubuntu2
from -updates.

>From there I set up the configuration file, launched a new rsyslog
instance, and used netcat to set 100 packets to the relp port.

https://paste.ubuntu.com/p/HfSDvNJzpX/

As we can see, there are 100 sockets still open in the CLOSE_WAIT state.

>From there I enabled -proposed and installed librelp
1.5.0-1ubuntu2.20.04.2.

I started a new instance of rsyslog, and used netcat to send another 100
packets to the relp port. This time, all sockets were closed and not
left in CLOSE_WAIT.

https://paste.ubuntu.com/p/tjXHhQ2293/

I also ran the testcase from the upstream testsuite, imrelp-
sessionbreak-vg.sh.

I did this by:

1) pull-lp-source rsyslog focal
2) edit debian/rules, add --enable-valgrind, remove --without-valgrind-tests,
3) wget 
https://github.com/rsyslog/rsyslog/commit/baee0bd5420649329793746f0daf87c4f59fe6a6.patch
4) quilt import baee0bd5420649329793746f0daf87c4f59fe6a6.patch
5) quilt push
6) chmod +x tests/imrelp-sessionbreak-vg.sh
6) debuild -uc -us -b

It will eventually build tests, and imrelp-sessionbreak-vg.sh passes:

make[5]: Entering directory '/home/ubuntu/rsyslog-8.2001.0/tests'
...
PASS: imrelp-sessionbreak-vg.sh
...

We pass both the upstream testsuite and the testcase from the bug
report.

The file descriptor leak has been fixed, happy to mark as verified for
Focal.

** Tags removed: verification-needed verification-needed-focal
** Tags added: verification-done-focal

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to rsyslog in Ubuntu.
https://bugs.launchpad.net/bugs/1908473

Title:
  rsyslog-relp: imrelp module leaves sockets in CLOSE_WAIT state which
  leads to file descriptor leak

Status in librelp package in Ubuntu:
  Fix Released
Status in rsyslog package in Ubuntu:
  Fix Released
Status in librelp source package in Focal:
  Fix Committed
Status in rsyslog source package in Focal:
  Won't Fix
Status in librelp source package in Groovy:
  Fix Committed
Status in rsyslog source package in Groovy:
  Fix Released
Status in librelp source package in Hirsute:
  Fix Released
Status in rsyslog source package in Hirsute:
  Fix Released

Bug description:
  [Impact]

  In recent versions of rsyslog and librelp, the imrelp module leaks
  file descriptors due to a bug where it does not correctly close
  sockets, and instead, leaves them in the CLOSE_WAIT state.

  This causes rsyslogd on busy servers to eventually hit the limit of
  maximum open files allowed, which locks rsyslogd up until it is
  restarted.

  A workaround is to restart rsyslogd every month or so to manually
  close all of the open sockets.

  Only users of the imrelp module are affected, and not rsyslog users in
  general.

  [Testcase]

  Install the rsyslog-relp module like so:

  $ sudo apt install rsyslog rsyslog-relp

  Next, generate a working directory, and make a config file that loads
  the relp module.

  $ sudo mkdir /workdir
  $ cat << EOF >> ./spool.conf
  \$LocalHostName spool
  \$AbortOnUncleanConfig on
  \$PreserveFQDN on

  global(
  workDirectory="/workdir"
  maxMessageSize="256k"
  )

  main_queue(queue.type="Direct")
  module(load="imrelp")
  input(
  type="imrelp"
  name="imrelp"
  port="601"
  ruleset="spool"
  MaxDataSize="256k"
  )

  ruleset(name="spool" queue.type="direct") {
  }

  # Just so rsyslog doesn't whine that we do not have outputs
  ruleset(name="noop" queue.type="direct") {
  action(
  type="omfile"
  name="omfile"
  file="/workdir/spool.log"
  )
  }
  EOF

  Verify that the config is valid, then start a rsyslog server.

  $ sudo rsyslogd -f ./spool.conf -N9
  $ sudo rsyslogd -f ./spool.conf -i /workdir/rsyslogd.pid

  Fetch the rsyslogd PID and check for open files.

  $ RLOGPID=$(cat /workdir/rsyslogd.pid)
  $ sudo ls -l /proc/$RLOGPID/fd
  total 0
  lr-x-- 1 root root 64 Dec 17 01:22 0 -> /dev/urandom
  lrwx-- 1 root root 64 Dec 17 01:22 1 -> 'socket:[41228]'
  lrwx-- 1 root root 64 Dec 17 01:22 3 -> 'socket:[41222]'
  lrwx-- 1 root root 64 Dec 17 01:22 4 -> 'socket:[41223]'
  lrwx-- 1 root root 64 Dec 17 01:22 7 -> 'anon_inode:[eventpoll]'

  We have 3 sockets open by default. Next, use netcat to open 100
  connections:

  $ for i in {1..100} ; do nc -z 127.0.0.1 601 ; done

  Now check for open file descriptors, and there will be an extra 100 sockets
  in the list:

  $ sudo ls -l /proc/$RLOGPID/fd

  https://paste.ubuntu.com/p/f6NQVNbZcR/

  We can check the state of these sockets with:

  $ ss -t

  https://paste.ubuntu.com/p/7Ts2FbxJrg/

  The listening sockets will be in CLOSE-WAIT, and the netcat sockets
  will be in FIN-WAIT-2.

  $ ss -t | grep CLOSE-WAIT | wc -l
  100

  If you install the test package available in the following ppa:

  https://launchpad.net/~mruffell/+archive/ubuntu/sf299578-test

  

[Touch-packages] [Bug 1908473] Re: rsyslog-relp: imrelp module leaves sockets in CLOSE_WAIT state which leads to file descriptor leak

2021-01-24 Thread Matthew Ruffell
Hi Mauricio,

I filed bug 1912969 to fix the FTBFS for librelp on focal. Adjusting the
packets down from 50,000 to 10,000 makes the build succeed on riscv64.

I attached two debdiffs, one an incremental patch from
1.5.0-1ubuntu2.20.04.1, and the other a full patch from 1.5.0-1ubuntu2.

Please review and sponsor.

Thanks,
Matthew

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to rsyslog in Ubuntu.
https://bugs.launchpad.net/bugs/1908473

Title:
  rsyslog-relp: imrelp module leaves sockets in CLOSE_WAIT state which
  leads to file descriptor leak

Status in librelp package in Ubuntu:
  Fix Released
Status in rsyslog package in Ubuntu:
  Fix Released
Status in librelp source package in Focal:
  Fix Committed
Status in rsyslog source package in Focal:
  Won't Fix
Status in librelp source package in Groovy:
  Fix Committed
Status in rsyslog source package in Groovy:
  Fix Released
Status in librelp source package in Hirsute:
  Fix Released
Status in rsyslog source package in Hirsute:
  Fix Released

Bug description:
  [Impact]

  In recent versions of rsyslog and librelp, the imrelp module leaks
  file descriptors due to a bug where it does not correctly close
  sockets, and instead, leaves them in the CLOSE_WAIT state.

  This causes rsyslogd on busy servers to eventually hit the limit of
  maximum open files allowed, which locks rsyslogd up until it is
  restarted.

  A workaround is to restart rsyslogd every month or so to manually
  close all of the open sockets.

  Only users of the imrelp module are affected, and not rsyslog users in
  general.

  [Testcase]

  Install the rsyslog-relp module like so:

  $ sudo apt install rsyslog rsyslog-relp

  Next, generate a working directory, and make a config file that loads
  the relp module.

  $ sudo mkdir /workdir
  $ cat << EOF >> ./spool.conf
  \$LocalHostName spool
  \$AbortOnUncleanConfig on
  \$PreserveFQDN on

  global(
  workDirectory="/workdir"
  maxMessageSize="256k"
  )

  main_queue(queue.type="Direct")
  module(load="imrelp")
  input(
  type="imrelp"
  name="imrelp"
  port="601"
  ruleset="spool"
  MaxDataSize="256k"
  )

  ruleset(name="spool" queue.type="direct") {
  }

  # Just so rsyslog doesn't whine that we do not have outputs
  ruleset(name="noop" queue.type="direct") {
  action(
  type="omfile"
  name="omfile"
  file="/workdir/spool.log"
  )
  }
  EOF

  Verify that the config is valid, then start a rsyslog server.

  $ sudo rsyslogd -f ./spool.conf -N9
  $ sudo rsyslogd -f ./spool.conf -i /workdir/rsyslogd.pid

  Fetch the rsyslogd PID and check for open files.

  $ RLOGPID=$(cat /workdir/rsyslogd.pid)
  $ sudo ls -l /proc/$RLOGPID/fd
  total 0
  lr-x-- 1 root root 64 Dec 17 01:22 0 -> /dev/urandom
  lrwx-- 1 root root 64 Dec 17 01:22 1 -> 'socket:[41228]'
  lrwx-- 1 root root 64 Dec 17 01:22 3 -> 'socket:[41222]'
  lrwx-- 1 root root 64 Dec 17 01:22 4 -> 'socket:[41223]'
  lrwx-- 1 root root 64 Dec 17 01:22 7 -> 'anon_inode:[eventpoll]'

  We have 3 sockets open by default. Next, use netcat to open 100
  connections:

  $ for i in {1..100} ; do nc -z 127.0.0.1 601 ; done

  Now check for open file descriptors, and there will be an extra 100 sockets
  in the list:

  $ sudo ls -l /proc/$RLOGPID/fd

  https://paste.ubuntu.com/p/f6NQVNbZcR/

  We can check the state of these sockets with:

  $ ss -t

  https://paste.ubuntu.com/p/7Ts2FbxJrg/

  The listening sockets will be in CLOSE-WAIT, and the netcat sockets
  will be in FIN-WAIT-2.

  $ ss -t | grep CLOSE-WAIT | wc -l
  100

  If you install the test package available in the following ppa:

  https://launchpad.net/~mruffell/+archive/ubuntu/sf299578-test

  When you open connections with netcat, these will be closed properly,
  and the file descriptor leak will be fixed.

  [Where problems could occur]

  If a regression were to occur, it would be limited to users of the
  imrelp module, which is a part of the rsyslogd-relp package, and
  depends on librelp.

  rsyslog-relp is not part of a default installation of rsyslog, and is
  opt in by changing a configuration file to enable imrelp.

  The changes to rsyslog implement a testcase which exercises the problematic 
code to ensure things are working as expected; this
  can be enabled manually on build, and has been verified to pass (#7).

  [Other]

  Upstream bug list:

  https://github.com/rsyslog/rsyslog/issues/4350
  https://github.com/rsyslog/rsyslog/issues/4005
  https://github.com/rsyslog/librelp/issues/188
  https://github.com/rsyslog/librelp/pull/193

  The following commits fix the problem:

  rsyslogd
  

  commit baee0bd5420649329793746f0daf87c4f59fe6a6
  Author: Andre lorbach 
  Date:   Thu Apr 9 13:00:35 2020 +0200
  Subject: testbench: Add test for imrelp to check broken session handling.
  Link: 

[Touch-packages] [Bug 1908473] Re: rsyslog-relp: imrelp module leaves sockets in CLOSE_WAIT state which leads to file descriptor leak

2021-01-21 Thread Matthew Ruffell
Hi Mauricio,

It seems riscv64 passes on Groovy due to tests being skipped on the
riscv64 architecture.

>From Groovy's build log:

https://paste.ubuntu.com/p/NCJPDVSbSW/

If you look at the man page for dh_auto_test it mentions:

If the DEB_BUILD_OPTIONS environment variable contains nocheck, no tests
will be performed.

nocheck was added to riscv64 by default for all packages in Groovy as a
part of this change to dpkg in bug 1891686.

The test cases basic-realistic.sh and tls-basic-realistic.sh fail on
Focal because they attempt to send 100,000 packets between the server
and the client, and we get to various stages, like 00029000 msgs sent,
and now 00047000 msgs sent with some changes William made to the
builders, before it times out and assumes the channel is dead, and the
test fails.

https://paste.ubuntu.com/p/hwYXSbKPPV/

We aren't going to hit the 100,000 packets on riscv anytime soon. I
think I will open a new bug to adjust the packet counts from 100,000
down to 10,000 for basic-realistic.sh and tls-basic-realistic.sh, which
resembles what has been done for receiver-abort.sh and tls-receiver-
abort.sh.

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to rsyslog in Ubuntu.
https://bugs.launchpad.net/bugs/1908473

Title:
  rsyslog-relp: imrelp module leaves sockets in CLOSE_WAIT state which
  leads to file descriptor leak

Status in librelp package in Ubuntu:
  Fix Released
Status in rsyslog package in Ubuntu:
  Fix Released
Status in librelp source package in Focal:
  Fix Committed
Status in rsyslog source package in Focal:
  Won't Fix
Status in librelp source package in Groovy:
  Fix Committed
Status in rsyslog source package in Groovy:
  Fix Released
Status in librelp source package in Hirsute:
  Fix Released
Status in rsyslog source package in Hirsute:
  Fix Released

Bug description:
  [Impact]

  In recent versions of rsyslog and librelp, the imrelp module leaks
  file descriptors due to a bug where it does not correctly close
  sockets, and instead, leaves them in the CLOSE_WAIT state.

  This causes rsyslogd on busy servers to eventually hit the limit of
  maximum open files allowed, which locks rsyslogd up until it is
  restarted.

  A workaround is to restart rsyslogd every month or so to manually
  close all of the open sockets.

  Only users of the imrelp module are affected, and not rsyslog users in
  general.

  [Testcase]

  Install the rsyslog-relp module like so:

  $ sudo apt install rsyslog rsyslog-relp

  Next, generate a working directory, and make a config file that loads
  the relp module.

  $ sudo mkdir /workdir
  $ cat << EOF >> ./spool.conf
  \$LocalHostName spool
  \$AbortOnUncleanConfig on
  \$PreserveFQDN on

  global(
  workDirectory="/workdir"
  maxMessageSize="256k"
  )

  main_queue(queue.type="Direct")
  module(load="imrelp")
  input(
  type="imrelp"
  name="imrelp"
  port="601"
  ruleset="spool"
  MaxDataSize="256k"
  )

  ruleset(name="spool" queue.type="direct") {
  }

  # Just so rsyslog doesn't whine that we do not have outputs
  ruleset(name="noop" queue.type="direct") {
  action(
  type="omfile"
  name="omfile"
  file="/workdir/spool.log"
  )
  }
  EOF

  Verify that the config is valid, then start a rsyslog server.

  $ sudo rsyslogd -f ./spool.conf -N9
  $ sudo rsyslogd -f ./spool.conf -i /workdir/rsyslogd.pid

  Fetch the rsyslogd PID and check for open files.

  $ RLOGPID=$(cat /workdir/rsyslogd.pid)
  $ sudo ls -l /proc/$RLOGPID/fd
  total 0
  lr-x-- 1 root root 64 Dec 17 01:22 0 -> /dev/urandom
  lrwx-- 1 root root 64 Dec 17 01:22 1 -> 'socket:[41228]'
  lrwx-- 1 root root 64 Dec 17 01:22 3 -> 'socket:[41222]'
  lrwx-- 1 root root 64 Dec 17 01:22 4 -> 'socket:[41223]'
  lrwx-- 1 root root 64 Dec 17 01:22 7 -> 'anon_inode:[eventpoll]'

  We have 3 sockets open by default. Next, use netcat to open 100
  connections:

  $ for i in {1..100} ; do nc -z 127.0.0.1 601 ; done

  Now check for open file descriptors, and there will be an extra 100 sockets
  in the list:

  $ sudo ls -l /proc/$RLOGPID/fd

  https://paste.ubuntu.com/p/f6NQVNbZcR/

  We can check the state of these sockets with:

  $ ss -t

  https://paste.ubuntu.com/p/7Ts2FbxJrg/

  The listening sockets will be in CLOSE-WAIT, and the netcat sockets
  will be in FIN-WAIT-2.

  $ ss -t | grep CLOSE-WAIT | wc -l
  100

  If you install the test package available in the following ppa:

  https://launchpad.net/~mruffell/+archive/ubuntu/sf299578-test

  When you open connections with netcat, these will be closed properly,
  and the file descriptor leak will be fixed.

  [Where problems could occur]

  If a regression were to occur, it would be limited to users of the
  imrelp module, which is a part of the rsyslogd-relp package, and
  depends on librelp.

  rsyslog-relp is not part of a default installation of rsyslog, and is
  opt in by changing a 

[Touch-packages] [Bug 1908473] Re: rsyslog-relp: imrelp module leaves sockets in CLOSE_WAIT state which leads to file descriptor leak

2021-01-20 Thread Matthew Ruffell
Performing verification for Focal

I installed rsyslog-relp 8.2001.0-1ubuntu1.1 and librelp0 1.5.0-1ubuntu2
from -updates.

>From there I set up the configuration file, launched a new rsyslog instance, 
>and
used netcat to set 100 packets to the relp port.

https://paste.ubuntu.com/p/jCs9Dy6FYF/

As we can see, there are 100 sockets still open in the CLOSE_WAIT state.

>From there I enabled -proposed and installed librelp
1.5.0-1ubuntu2.20.04.1.

I started a new instance of rsyslog, and used netcat to send another 100 packets
to the relp port. This time, all sockets were closed and not left in CLOSE_WAIT.

https://paste.ubuntu.com/p/vdzsVTctmf/

I also ran the testcase from the upstream testsuite, imrelp-
sessionbreak-vg.sh.

I did this by:

1) pull-lp-source rsyslog focal
2) edit debian/rules, add --enable-valgrind, remove --without-valgrind-tests,
3) wget 
https://github.com/rsyslog/rsyslog/commit/baee0bd5420649329793746f0daf87c4f59fe6a6.patch
4) quilt import baee0bd5420649329793746f0daf87c4f59fe6a6.patch
5) quilt push
6) chmod +x tests/imrelp-sessionbreak-vg.sh
6) debuild -uc -us -b

It will eventually build tests, and imrelp-sessionbreak-vg.sh passes:

make[5]: Entering directory '/home/ubuntu/rsyslog-8.2001.0/tests'
...
PASS: imrelp-sessionbreak-vg.sh
...

We pass both the upstream testsuite and the testcase from the bug
report.

The file descriptor leak has been fixed, happy to mark as verified for
Focal.

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to rsyslog in Ubuntu.
https://bugs.launchpad.net/bugs/1908473

Title:
  rsyslog-relp: imrelp module leaves sockets in CLOSE_WAIT state which
  leads to file descriptor leak

Status in librelp package in Ubuntu:
  Fix Released
Status in rsyslog package in Ubuntu:
  Fix Released
Status in librelp source package in Focal:
  Fix Committed
Status in rsyslog source package in Focal:
  Won't Fix
Status in librelp source package in Groovy:
  Fix Committed
Status in rsyslog source package in Groovy:
  Fix Released
Status in librelp source package in Hirsute:
  Fix Released
Status in rsyslog source package in Hirsute:
  Fix Released

Bug description:
  [Impact]

  In recent versions of rsyslog and librelp, the imrelp module leaks
  file descriptors due to a bug where it does not correctly close
  sockets, and instead, leaves them in the CLOSE_WAIT state.

  This causes rsyslogd on busy servers to eventually hit the limit of
  maximum open files allowed, which locks rsyslogd up until it is
  restarted.

  A workaround is to restart rsyslogd every month or so to manually
  close all of the open sockets.

  Only users of the imrelp module are affected, and not rsyslog users in
  general.

  [Testcase]

  Install the rsyslog-relp module like so:

  $ sudo apt install rsyslog rsyslog-relp

  Next, generate a working directory, and make a config file that loads
  the relp module.

  $ sudo mkdir /workdir
  $ cat << EOF >> ./spool.conf
  \$LocalHostName spool
  \$AbortOnUncleanConfig on
  \$PreserveFQDN on

  global(
  workDirectory="/workdir"
  maxMessageSize="256k"
  )

  main_queue(queue.type="Direct")
  module(load="imrelp")
  input(
  type="imrelp"
  name="imrelp"
  port="601"
  ruleset="spool"
  MaxDataSize="256k"
  )

  ruleset(name="spool" queue.type="direct") {
  }

  # Just so rsyslog doesn't whine that we do not have outputs
  ruleset(name="noop" queue.type="direct") {
  action(
  type="omfile"
  name="omfile"
  file="/workdir/spool.log"
  )
  }
  EOF

  Verify that the config is valid, then start a rsyslog server.

  $ sudo rsyslogd -f ./spool.conf -N9
  $ sudo rsyslogd -f ./spool.conf -i /workdir/rsyslogd.pid

  Fetch the rsyslogd PID and check for open files.

  $ RLOGPID=$(cat /workdir/rsyslogd.pid)
  $ sudo ls -l /proc/$RLOGPID/fd
  total 0
  lr-x-- 1 root root 64 Dec 17 01:22 0 -> /dev/urandom
  lrwx-- 1 root root 64 Dec 17 01:22 1 -> 'socket:[41228]'
  lrwx-- 1 root root 64 Dec 17 01:22 3 -> 'socket:[41222]'
  lrwx-- 1 root root 64 Dec 17 01:22 4 -> 'socket:[41223]'
  lrwx-- 1 root root 64 Dec 17 01:22 7 -> 'anon_inode:[eventpoll]'

  We have 3 sockets open by default. Next, use netcat to open 100
  connections:

  $ for i in {1..100} ; do nc -z 127.0.0.1 601 ; done

  Now check for open file descriptors, and there will be an extra 100 sockets
  in the list:

  $ sudo ls -l /proc/$RLOGPID/fd

  https://paste.ubuntu.com/p/f6NQVNbZcR/

  We can check the state of these sockets with:

  $ ss -t

  https://paste.ubuntu.com/p/7Ts2FbxJrg/

  The listening sockets will be in CLOSE-WAIT, and the netcat sockets
  will be in FIN-WAIT-2.

  $ ss -t | grep CLOSE-WAIT | wc -l
  100

  If you install the test package available in the following ppa:

  https://launchpad.net/~mruffell/+archive/ubuntu/sf299578-test

  When you open connections with netcat, these will be closed properly,
  and the file descriptor leak 

[Touch-packages] [Bug 1908473] Re: rsyslog-relp: imrelp module leaves sockets in CLOSE_WAIT state which leads to file descriptor leak

2021-01-20 Thread Matthew Ruffell
Performing verification for librelp in Groovy.

I installed rsyslog-relp 8.2006.0-2ubuntu1 and librelp 1.5.0-1ubuntu2
from -updates to reproduce:

https://paste.ubuntu.com/p/gtn4rcXc72/

>From there I set up the configuration script, ran a new instance of
rsyslog, and used netcat to open 100 connections to the relp port.

When I checked the list of file descriptors, there were 100 sockets
open, in the CLOSE_WAIT state.

>From there, I enabled -proposed and installed librelp
1.5.0-1ubuntu2.20.10.1:

https://paste.ubuntu.com/p/nt342PJkQ5/

I started a new rsyslog instance, and used netcat to open 100
connections to the relp port.

All sockets were closed when rsyslog was done with them, and there were
no sockets in CLOSE_WAIT.

I also ran the provided testcase in rsyslog, imrelp-sessionbreak-vg.sh.

I did this by:

1) pull-lp-source rsyslog groovy
2) edit debian/rules, add --enable-valgrind, remove --without-valgrind-tests,
3) debuild -uc -us -b

It will eventually build tests, and imrelp-sessionbreak-vg.sh passes:

make[5]: Entering directory '/home/ubuntu/rsyslog-8.2006.0/tests'
...
PASS: imrelp-sessionbreak-vg.sh
...

We pass both the upstream testsuite and the testcase from the bug
report.

The file descriptor leak has been fixed, happy to mark as verified for
Groovy.

** Tags removed: verification-needed-groovy
** Tags added: verification-done-groovy

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to rsyslog in Ubuntu.
https://bugs.launchpad.net/bugs/1908473

Title:
  rsyslog-relp: imrelp module leaves sockets in CLOSE_WAIT state which
  leads to file descriptor leak

Status in librelp package in Ubuntu:
  Fix Released
Status in rsyslog package in Ubuntu:
  Fix Released
Status in librelp source package in Focal:
  Fix Committed
Status in rsyslog source package in Focal:
  Won't Fix
Status in librelp source package in Groovy:
  Fix Committed
Status in rsyslog source package in Groovy:
  Fix Released
Status in librelp source package in Hirsute:
  Fix Released
Status in rsyslog source package in Hirsute:
  Fix Released

Bug description:
  [Impact]

  In recent versions of rsyslog and librelp, the imrelp module leaks
  file descriptors due to a bug where it does not correctly close
  sockets, and instead, leaves them in the CLOSE_WAIT state.

  This causes rsyslogd on busy servers to eventually hit the limit of
  maximum open files allowed, which locks rsyslogd up until it is
  restarted.

  A workaround is to restart rsyslogd every month or so to manually
  close all of the open sockets.

  Only users of the imrelp module are affected, and not rsyslog users in
  general.

  [Testcase]

  Install the rsyslog-relp module like so:

  $ sudo apt install rsyslog rsyslog-relp

  Next, generate a working directory, and make a config file that loads
  the relp module.

  $ sudo mkdir /workdir
  $ cat << EOF >> ./spool.conf
  \$LocalHostName spool
  \$AbortOnUncleanConfig on
  \$PreserveFQDN on

  global(
  workDirectory="/workdir"
  maxMessageSize="256k"
  )

  main_queue(queue.type="Direct")
  module(load="imrelp")
  input(
  type="imrelp"
  name="imrelp"
  port="601"
  ruleset="spool"
  MaxDataSize="256k"
  )

  ruleset(name="spool" queue.type="direct") {
  }

  # Just so rsyslog doesn't whine that we do not have outputs
  ruleset(name="noop" queue.type="direct") {
  action(
  type="omfile"
  name="omfile"
  file="/workdir/spool.log"
  )
  }
  EOF

  Verify that the config is valid, then start a rsyslog server.

  $ sudo rsyslogd -f ./spool.conf -N9
  $ sudo rsyslogd -f ./spool.conf -i /workdir/rsyslogd.pid

  Fetch the rsyslogd PID and check for open files.

  $ RLOGPID=$(cat /workdir/rsyslogd.pid)
  $ sudo ls -l /proc/$RLOGPID/fd
  total 0
  lr-x-- 1 root root 64 Dec 17 01:22 0 -> /dev/urandom
  lrwx-- 1 root root 64 Dec 17 01:22 1 -> 'socket:[41228]'
  lrwx-- 1 root root 64 Dec 17 01:22 3 -> 'socket:[41222]'
  lrwx-- 1 root root 64 Dec 17 01:22 4 -> 'socket:[41223]'
  lrwx-- 1 root root 64 Dec 17 01:22 7 -> 'anon_inode:[eventpoll]'

  We have 3 sockets open by default. Next, use netcat to open 100
  connections:

  $ for i in {1..100} ; do nc -z 127.0.0.1 601 ; done

  Now check for open file descriptors, and there will be an extra 100 sockets
  in the list:

  $ sudo ls -l /proc/$RLOGPID/fd

  https://paste.ubuntu.com/p/f6NQVNbZcR/

  We can check the state of these sockets with:

  $ ss -t

  https://paste.ubuntu.com/p/7Ts2FbxJrg/

  The listening sockets will be in CLOSE-WAIT, and the netcat sockets
  will be in FIN-WAIT-2.

  $ ss -t | grep CLOSE-WAIT | wc -l
  100

  If you install the test package available in the following ppa:

  https://launchpad.net/~mruffell/+archive/ubuntu/sf299578-test

  When you open connections with netcat, these will be closed properly,
  and the file descriptor leak will be fixed.

  [Where problems could occur]

  If a 

[Touch-packages] [Bug 1908473] Re: rsyslog-relp: imrelp module leaves sockets in CLOSE_WAIT state which leads to file descriptor leak

2021-01-20 Thread Matthew Ruffell
Same one failure on today's rebuild. Strange, since this is the exact
same code as Groovy.


** Attachment added: 
"buildlog_ubuntu-focal-riscv64.librelp_1.5.0-1ubuntu2.20.04.1_BUILDING.txt.gz.5"
   
https://bugs.launchpad.net/ubuntu/+source/rsyslog/+bug/1908473/+attachment/5454976/+files/buildlog_ubuntu-focal-riscv64.librelp_1.5.0-1ubuntu2.20.04.1_BUILDING.txt.gz.5

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to rsyslog in Ubuntu.
https://bugs.launchpad.net/bugs/1908473

Title:
  rsyslog-relp: imrelp module leaves sockets in CLOSE_WAIT state which
  leads to file descriptor leak

Status in librelp package in Ubuntu:
  Fix Released
Status in rsyslog package in Ubuntu:
  Fix Released
Status in librelp source package in Focal:
  Fix Committed
Status in rsyslog source package in Focal:
  Won't Fix
Status in librelp source package in Groovy:
  Fix Committed
Status in rsyslog source package in Groovy:
  Fix Released
Status in librelp source package in Hirsute:
  Fix Released
Status in rsyslog source package in Hirsute:
  Fix Released

Bug description:
  [Impact]

  In recent versions of rsyslog and librelp, the imrelp module leaks
  file descriptors due to a bug where it does not correctly close
  sockets, and instead, leaves them in the CLOSE_WAIT state.

  This causes rsyslogd on busy servers to eventually hit the limit of
  maximum open files allowed, which locks rsyslogd up until it is
  restarted.

  A workaround is to restart rsyslogd every month or so to manually
  close all of the open sockets.

  Only users of the imrelp module are affected, and not rsyslog users in
  general.

  [Testcase]

  Install the rsyslog-relp module like so:

  $ sudo apt install rsyslog rsyslog-relp

  Next, generate a working directory, and make a config file that loads
  the relp module.

  $ sudo mkdir /workdir
  $ cat << EOF >> ./spool.conf
  \$LocalHostName spool
  \$AbortOnUncleanConfig on
  \$PreserveFQDN on

  global(
  workDirectory="/workdir"
  maxMessageSize="256k"
  )

  main_queue(queue.type="Direct")
  module(load="imrelp")
  input(
  type="imrelp"
  name="imrelp"
  port="601"
  ruleset="spool"
  MaxDataSize="256k"
  )

  ruleset(name="spool" queue.type="direct") {
  }

  # Just so rsyslog doesn't whine that we do not have outputs
  ruleset(name="noop" queue.type="direct") {
  action(
  type="omfile"
  name="omfile"
  file="/workdir/spool.log"
  )
  }
  EOF

  Verify that the config is valid, then start a rsyslog server.

  $ sudo rsyslogd -f ./spool.conf -N9
  $ sudo rsyslogd -f ./spool.conf -i /workdir/rsyslogd.pid

  Fetch the rsyslogd PID and check for open files.

  $ RLOGPID=$(cat /workdir/rsyslogd.pid)
  $ sudo ls -l /proc/$RLOGPID/fd
  total 0
  lr-x-- 1 root root 64 Dec 17 01:22 0 -> /dev/urandom
  lrwx-- 1 root root 64 Dec 17 01:22 1 -> 'socket:[41228]'
  lrwx-- 1 root root 64 Dec 17 01:22 3 -> 'socket:[41222]'
  lrwx-- 1 root root 64 Dec 17 01:22 4 -> 'socket:[41223]'
  lrwx-- 1 root root 64 Dec 17 01:22 7 -> 'anon_inode:[eventpoll]'

  We have 3 sockets open by default. Next, use netcat to open 100
  connections:

  $ for i in {1..100} ; do nc -z 127.0.0.1 601 ; done

  Now check for open file descriptors, and there will be an extra 100 sockets
  in the list:

  $ sudo ls -l /proc/$RLOGPID/fd

  https://paste.ubuntu.com/p/f6NQVNbZcR/

  We can check the state of these sockets with:

  $ ss -t

  https://paste.ubuntu.com/p/7Ts2FbxJrg/

  The listening sockets will be in CLOSE-WAIT, and the netcat sockets
  will be in FIN-WAIT-2.

  $ ss -t | grep CLOSE-WAIT | wc -l
  100

  If you install the test package available in the following ppa:

  https://launchpad.net/~mruffell/+archive/ubuntu/sf299578-test

  When you open connections with netcat, these will be closed properly,
  and the file descriptor leak will be fixed.

  [Where problems could occur]

  If a regression were to occur, it would be limited to users of the
  imrelp module, which is a part of the rsyslogd-relp package, and
  depends on librelp.

  rsyslog-relp is not part of a default installation of rsyslog, and is
  opt in by changing a configuration file to enable imrelp.

  The changes to rsyslog implement a testcase which exercises the problematic 
code to ensure things are working as expected; this
  can be enabled manually on build, and has been verified to pass (#7).

  [Other]

  Upstream bug list:

  https://github.com/rsyslog/rsyslog/issues/4350
  https://github.com/rsyslog/rsyslog/issues/4005
  https://github.com/rsyslog/librelp/issues/188
  https://github.com/rsyslog/librelp/pull/193

  The following commits fix the problem:

  rsyslogd
  

  commit baee0bd5420649329793746f0daf87c4f59fe6a6
  Author: Andre lorbach 
  Date:   Thu Apr 9 13:00:35 2020 +0200
  Subject: testbench: Add test for imrelp to check broken session handling.
  Link: 

[Touch-packages] [Bug 1912122] Re: /var/log/dmesg is 0644, should be 0640 to match new DMESG_RESTRICT restrictions

2021-01-20 Thread Matthew Ruffell
Hi Robie, I agree this probably isn't worth a SRU to Groovy, I just made
the packages available in the odd chance that they might be considered.
I will mark Groovy as won't fix.

Hirsute is what really matters in the end.

** Changed in: rsyslog (Ubuntu Groovy)
   Status: In Progress => Won't Fix

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to rsyslog in Ubuntu.
https://bugs.launchpad.net/bugs/1912122

Title:
  /var/log/dmesg is 0644, should be 0640 to match new DMESG_RESTRICT
  restrictions

Status in rsyslog package in Ubuntu:
  In Progress
Status in rsyslog source package in Groovy:
  Won't Fix
Status in rsyslog source package in Hirsute:
  In Progress

Bug description:
  [Impact]

  In bug 1886112, CONFIG_SECURITY_DMESG_RESTRICT was enabled on the
  Ubuntu kernel starting with Groovy and onward, in an effort to
  restrict access to the kernel log buffer from unprivileged users.

  It seems we have overlooked /var/log/dmesg, as it is still mode 0644,
  while /var/log/kern.log, /var/log/syslog are all 0640:

  $ ll /var/log
  -rw-r--r--   1 root  adm 81768 Jan 18 09:09 dmesg
  -rw-r-   1 syslogadm 24538 Jan 18 13:05 
kern.log
  -rw-r-   1 syslogadm213911 Jan 18 13:22 syslog

  Change /var/log/dmesg to 0640 to close the information leak.

  [Testcase]

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ cat /var/log/kern.log
  cat: /var/log/kern.log: Permission denied
  $ cat /var/log/syslog
  cat: /var/log/syslog: Permission denied
  $ cat /var/log/dmesg
  [0.00] kernel: Linux version 5.8.0-36-generic 
(buildd@lgw01-amd64-011) (gcc (Ubuntu 10.2.1-2ubuntu3) 10.2.1 20201221, GNU ld 
(GNU Binutils for Ubuntu) 2.35.50.20210106) #40+21.04.1-Ubuntu SMP Thu Jan 7 
11:35:09 UTC 2021 (Ubuntu 5.8.0-36.40+21.04.1-generic 5.8.18)
  [0.00] kernel: Command line: BOOT_IMAGE=/casper/vmlinuz 
file=/cdrom/preseed/ubuntu.seed maybe-ubiquity quiet splash ---

  If you install the package in the following ppa:

  https://launchpad.net/~mruffell/+archive/ubuntu/lp1912122-test

  $ sudo systemctl daemon-reload
  $ sudo systemctl start dmesg.service

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ cat /var/log/kern.log
  cat: /var/log/kern.log: Permission denied
  $ cat /var/log/syslog
  cat: /var/log/syslog: Permission denied
  $ cat /var/log/dmesg
  cat: /var/log/dmesg: Permission denied

  [Where problems could occur]

  Some users or log scraper programs might need to view the kernel log
  buffers, and in this case, their underlying service accounts should be
  added to the 'adm' group.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/rsyslog/+bug/1912122/+subscriptions

-- 
Mailing list: https://launchpad.net/~touch-packages
Post to : touch-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~touch-packages
More help   : https://help.launchpad.net/ListHelp


[Touch-packages] [Bug 1912122] Re: /var/log/dmesg is 0644, should be 0640 to match new DMESG_RESTRICT restrictions

2021-01-18 Thread Matthew Ruffell
Attached is a patch which changes /var/log/dmesg to 0640 on groovy. It
also contains Steve's recommendation to set the logrotate files to 0640.

** Patch added: "Debdiff for syslog on groovy"
   
https://bugs.launchpad.net/ubuntu/+source/rsyslog/+bug/1912122/+attachment/5454311/+files/lp1912122_groovy_v2.debdiff

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to rsyslog in Ubuntu.
https://bugs.launchpad.net/bugs/1912122

Title:
  /var/log/dmesg is 0644, should be 0640 to match new DMESG_RESTRICT
  restrictions

Status in rsyslog package in Ubuntu:
  In Progress
Status in rsyslog source package in Groovy:
  In Progress
Status in rsyslog source package in Hirsute:
  In Progress

Bug description:
  [Impact]

  In bug 1886112, CONFIG_SECURITY_DMESG_RESTRICT was enabled on the
  Ubuntu kernel starting with Groovy and onward, in an effort to
  restrict access to the kernel log buffer from unprivileged users.

  It seems we have overlooked /var/log/dmesg, as it is still mode 0644,
  while /var/log/kern.log, /var/log/syslog are all 0640:

  $ ll /var/log
  -rw-r--r--   1 root  adm 81768 Jan 18 09:09 dmesg
  -rw-r-   1 syslogadm 24538 Jan 18 13:05 
kern.log
  -rw-r-   1 syslogadm213911 Jan 18 13:22 syslog

  Change /var/log/dmesg to 0640 to close the information leak.

  [Testcase]

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ cat /var/log/kern.log
  cat: /var/log/kern.log: Permission denied
  $ cat /var/log/syslog
  cat: /var/log/syslog: Permission denied
  $ cat /var/log/dmesg
  [0.00] kernel: Linux version 5.8.0-36-generic 
(buildd@lgw01-amd64-011) (gcc (Ubuntu 10.2.1-2ubuntu3) 10.2.1 20201221, GNU ld 
(GNU Binutils for Ubuntu) 2.35.50.20210106) #40+21.04.1-Ubuntu SMP Thu Jan 7 
11:35:09 UTC 2021 (Ubuntu 5.8.0-36.40+21.04.1-generic 5.8.18)
  [0.00] kernel: Command line: BOOT_IMAGE=/casper/vmlinuz 
file=/cdrom/preseed/ubuntu.seed maybe-ubiquity quiet splash ---

  If you install the package in the following ppa:

  https://launchpad.net/~mruffell/+archive/ubuntu/lp1912122-test

  $ sudo systemctl daemon-reload
  $ sudo systemctl start dmesg.service

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ cat /var/log/kern.log
  cat: /var/log/kern.log: Permission denied
  $ cat /var/log/syslog
  cat: /var/log/syslog: Permission denied
  $ cat /var/log/dmesg
  cat: /var/log/dmesg: Permission denied

  [Where problems could occur]

  Some users or log scraper programs might need to view the kernel log
  buffers, and in this case, their underlying service accounts should be
  added to the 'adm' group.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/rsyslog/+bug/1912122/+subscriptions

-- 
Mailing list: https://launchpad.net/~touch-packages
Post to : touch-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~touch-packages
More help   : https://help.launchpad.net/ListHelp


[Touch-packages] [Bug 1912122] Re: /var/log/dmesg is 0644, should be 0640 to match new DMESG_RESTRICT restrictions

2021-01-18 Thread Matthew Ruffell
Attached is a patch which changes /var/log/dmesg to 0640 on hirsute. It
also contains Steve's recommendation to set the logrotate files to 0640.

** Patch removed: "Debdiff for rsyslog on hirsute"
   
https://bugs.launchpad.net/ubuntu/+source/rsyslog/+bug/1912122/+attachment/5454004/+files/lp1912122_hirsute.debdiff

** Patch removed: "Debdiff for syslog on groovy"
   
https://bugs.launchpad.net/ubuntu/+source/rsyslog/+bug/1912122/+attachment/5454005/+files/lp1912122_groovy.debdiff

** Patch added: "Debdiff for rsyslog on hirsute"
   
https://bugs.launchpad.net/ubuntu/+source/rsyslog/+bug/1912122/+attachment/5454310/+files/lp1912122_hirsute_v2.debdiff

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to rsyslog in Ubuntu.
https://bugs.launchpad.net/bugs/1912122

Title:
  /var/log/dmesg is 0644, should be 0640 to match new DMESG_RESTRICT
  restrictions

Status in rsyslog package in Ubuntu:
  In Progress
Status in rsyslog source package in Groovy:
  In Progress
Status in rsyslog source package in Hirsute:
  In Progress

Bug description:
  [Impact]

  In bug 1886112, CONFIG_SECURITY_DMESG_RESTRICT was enabled on the
  Ubuntu kernel starting with Groovy and onward, in an effort to
  restrict access to the kernel log buffer from unprivileged users.

  It seems we have overlooked /var/log/dmesg, as it is still mode 0644,
  while /var/log/kern.log, /var/log/syslog are all 0640:

  $ ll /var/log
  -rw-r--r--   1 root  adm 81768 Jan 18 09:09 dmesg
  -rw-r-   1 syslogadm 24538 Jan 18 13:05 
kern.log
  -rw-r-   1 syslogadm213911 Jan 18 13:22 syslog

  Change /var/log/dmesg to 0640 to close the information leak.

  [Testcase]

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ cat /var/log/kern.log
  cat: /var/log/kern.log: Permission denied
  $ cat /var/log/syslog
  cat: /var/log/syslog: Permission denied
  $ cat /var/log/dmesg
  [0.00] kernel: Linux version 5.8.0-36-generic 
(buildd@lgw01-amd64-011) (gcc (Ubuntu 10.2.1-2ubuntu3) 10.2.1 20201221, GNU ld 
(GNU Binutils for Ubuntu) 2.35.50.20210106) #40+21.04.1-Ubuntu SMP Thu Jan 7 
11:35:09 UTC 2021 (Ubuntu 5.8.0-36.40+21.04.1-generic 5.8.18)
  [0.00] kernel: Command line: BOOT_IMAGE=/casper/vmlinuz 
file=/cdrom/preseed/ubuntu.seed maybe-ubiquity quiet splash ---

  If you install the package in the following ppa:

  https://launchpad.net/~mruffell/+archive/ubuntu/lp1912122-test

  $ sudo systemctl daemon-reload
  $ sudo systemctl start dmesg.service

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ cat /var/log/kern.log
  cat: /var/log/kern.log: Permission denied
  $ cat /var/log/syslog
  cat: /var/log/syslog: Permission denied
  $ cat /var/log/dmesg
  cat: /var/log/dmesg: Permission denied

  [Where problems could occur]

  Some users or log scraper programs might need to view the kernel log
  buffers, and in this case, their underlying service accounts should be
  added to the 'adm' group.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/rsyslog/+bug/1912122/+subscriptions

-- 
Mailing list: https://launchpad.net/~touch-packages
Post to : touch-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~touch-packages
More help   : https://help.launchpad.net/ListHelp


[Touch-packages] [Bug 1912122] Re: /var/log/dmesg is 0644, should be 0640 to match new DMESG_RESTRICT restrictions

2021-01-17 Thread Matthew Ruffell
** Tags added: sts-sponsor

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to rsyslog in Ubuntu.
https://bugs.launchpad.net/bugs/1912122

Title:
  /var/log/dmesg is 0644, should be 0640 to match new DMESG_RESTRICT
  restrictions

Status in rsyslog package in Ubuntu:
  In Progress
Status in rsyslog source package in Groovy:
  In Progress
Status in rsyslog source package in Hirsute:
  In Progress

Bug description:
  [Impact]

  In bug 1886112, CONFIG_SECURITY_DMESG_RESTRICT was enabled on the
  Ubuntu kernel starting with Groovy and onward, in an effort to
  restrict access to the kernel log buffer from unprivileged users.

  It seems we have overlooked /var/log/dmesg, as it is still mode 0644,
  while /var/log/kern.log, /var/log/syslog are all 0640:

  $ ll /var/log
  -rw-r--r--   1 root  adm 81768 Jan 18 09:09 dmesg
  -rw-r-   1 syslogadm 24538 Jan 18 13:05 
kern.log
  -rw-r-   1 syslogadm213911 Jan 18 13:22 syslog

  Change /var/log/dmesg to 0640 to close the information leak.

  [Testcase]

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ cat /var/log/kern.log
  cat: /var/log/kern.log: Permission denied
  $ cat /var/log/syslog
  cat: /var/log/syslog: Permission denied
  $ cat /var/log/dmesg
  [0.00] kernel: Linux version 5.8.0-36-generic 
(buildd@lgw01-amd64-011) (gcc (Ubuntu 10.2.1-2ubuntu3) 10.2.1 20201221, GNU ld 
(GNU Binutils for Ubuntu) 2.35.50.20210106) #40+21.04.1-Ubuntu SMP Thu Jan 7 
11:35:09 UTC 2021 (Ubuntu 5.8.0-36.40+21.04.1-generic 5.8.18)
  [0.00] kernel: Command line: BOOT_IMAGE=/casper/vmlinuz 
file=/cdrom/preseed/ubuntu.seed maybe-ubiquity quiet splash ---

  If you install the package in the following ppa:

  https://launchpad.net/~mruffell/+archive/ubuntu/lp1912122-test

  $ sudo systemctl daemon-reload
  $ sudo systemctl start dmesg.service

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ cat /var/log/kern.log
  cat: /var/log/kern.log: Permission denied
  $ cat /var/log/syslog
  cat: /var/log/syslog: Permission denied
  $ cat /var/log/dmesg
  cat: /var/log/dmesg: Permission denied

  [Where problems could occur]

  Some users or log scraper programs might need to view the kernel log
  buffers, and in this case, their underlying service accounts should be
  added to the 'adm' group.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/rsyslog/+bug/1912122/+subscriptions

-- 
Mailing list: https://launchpad.net/~touch-packages
Post to : touch-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~touch-packages
More help   : https://help.launchpad.net/ListHelp


[Touch-packages] [Bug 1912122] Re: /var/log/dmesg is 0644, should be 0640 to match new DMESG_RESTRICT restrictions

2021-01-17 Thread Matthew Ruffell
Attached is a debdiff for Groovy to change /var/log/dmesg to 0640.

** Patch added: "Debdiff for syslog on groovy"
   
https://bugs.launchpad.net/ubuntu/+source/rsyslog/+bug/1912122/+attachment/5454005/+files/lp1912122_groovy.debdiff

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to rsyslog in Ubuntu.
https://bugs.launchpad.net/bugs/1912122

Title:
  /var/log/dmesg is 0644, should be 0640 to match new DMESG_RESTRICT
  restrictions

Status in rsyslog package in Ubuntu:
  In Progress
Status in rsyslog source package in Groovy:
  In Progress
Status in rsyslog source package in Hirsute:
  In Progress

Bug description:
  [Impact]

  In bug 1886112, CONFIG_SECURITY_DMESG_RESTRICT was enabled on the
  Ubuntu kernel starting with Groovy and onward, in an effort to
  restrict access to the kernel log buffer from unprivileged users.

  It seems we have overlooked /var/log/dmesg, as it is still mode 0644,
  while /var/log/kern.log, /var/log/syslog are all 0640:

  $ ll /var/log
  -rw-r--r--   1 root  adm 81768 Jan 18 09:09 dmesg
  -rw-r-   1 syslogadm 24538 Jan 18 13:05 
kern.log
  -rw-r-   1 syslogadm213911 Jan 18 13:22 syslog

  Change /var/log/dmesg to 0640 to close the information leak.

  [Testcase]

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ cat /var/log/kern.log
  cat: /var/log/kern.log: Permission denied
  $ cat /var/log/syslog
  cat: /var/log/syslog: Permission denied
  $ cat /var/log/dmesg
  [0.00] kernel: Linux version 5.8.0-36-generic 
(buildd@lgw01-amd64-011) (gcc (Ubuntu 10.2.1-2ubuntu3) 10.2.1 20201221, GNU ld 
(GNU Binutils for Ubuntu) 2.35.50.20210106) #40+21.04.1-Ubuntu SMP Thu Jan 7 
11:35:09 UTC 2021 (Ubuntu 5.8.0-36.40+21.04.1-generic 5.8.18)
  [0.00] kernel: Command line: BOOT_IMAGE=/casper/vmlinuz 
file=/cdrom/preseed/ubuntu.seed maybe-ubiquity quiet splash ---

  If you install the package in the following ppa:

  https://launchpad.net/~mruffell/+archive/ubuntu/lp1912122-test

  $ sudo systemctl daemon-reload
  $ sudo systemctl start dmesg.service

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ cat /var/log/kern.log
  cat: /var/log/kern.log: Permission denied
  $ cat /var/log/syslog
  cat: /var/log/syslog: Permission denied
  $ cat /var/log/dmesg
  cat: /var/log/dmesg: Permission denied

  [Where problems could occur]

  Some users or log scraper programs might need to view the kernel log
  buffers, and in this case, their underlying service accounts should be
  added to the 'adm' group.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/rsyslog/+bug/1912122/+subscriptions

-- 
Mailing list: https://launchpad.net/~touch-packages
Post to : touch-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~touch-packages
More help   : https://help.launchpad.net/ListHelp


[Touch-packages] [Bug 1912122] Re: /var/log/dmesg is 0644, should be 0640 to match new DMESG_RESTRICT restrictions

2021-01-17 Thread Matthew Ruffell
Attached is a debdiff for hirsute to set /var/log/dmesg to 0640.

** Patch added: "Debdiff for rsyslog on hirsute"
   
https://bugs.launchpad.net/ubuntu/+source/rsyslog/+bug/1912122/+attachment/5454004/+files/lp1912122_hirsute.debdiff

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to rsyslog in Ubuntu.
https://bugs.launchpad.net/bugs/1912122

Title:
  /var/log/dmesg is 0644, should be 0640 to match new DMESG_RESTRICT
  restrictions

Status in rsyslog package in Ubuntu:
  In Progress
Status in rsyslog source package in Groovy:
  In Progress
Status in rsyslog source package in Hirsute:
  In Progress

Bug description:
  [Impact]

  In bug 1886112, CONFIG_SECURITY_DMESG_RESTRICT was enabled on the
  Ubuntu kernel starting with Groovy and onward, in an effort to
  restrict access to the kernel log buffer from unprivileged users.

  It seems we have overlooked /var/log/dmesg, as it is still mode 0644,
  while /var/log/kern.log, /var/log/syslog are all 0640:

  $ ll /var/log
  -rw-r--r--   1 root  adm 81768 Jan 18 09:09 dmesg
  -rw-r-   1 syslogadm 24538 Jan 18 13:05 
kern.log
  -rw-r-   1 syslogadm213911 Jan 18 13:22 syslog

  Change /var/log/dmesg to 0640 to close the information leak.

  [Testcase]

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ cat /var/log/kern.log
  cat: /var/log/kern.log: Permission denied
  $ cat /var/log/syslog
  cat: /var/log/syslog: Permission denied
  $ cat /var/log/dmesg
  [0.00] kernel: Linux version 5.8.0-36-generic 
(buildd@lgw01-amd64-011) (gcc (Ubuntu 10.2.1-2ubuntu3) 10.2.1 20201221, GNU ld 
(GNU Binutils for Ubuntu) 2.35.50.20210106) #40+21.04.1-Ubuntu SMP Thu Jan 7 
11:35:09 UTC 2021 (Ubuntu 5.8.0-36.40+21.04.1-generic 5.8.18)
  [0.00] kernel: Command line: BOOT_IMAGE=/casper/vmlinuz 
file=/cdrom/preseed/ubuntu.seed maybe-ubiquity quiet splash ---

  If you install the package in the following ppa:

  https://launchpad.net/~mruffell/+archive/ubuntu/lp1912122-test

  $ sudo systemctl daemon-reload
  $ sudo systemctl start dmesg.service

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ cat /var/log/kern.log
  cat: /var/log/kern.log: Permission denied
  $ cat /var/log/syslog
  cat: /var/log/syslog: Permission denied
  $ cat /var/log/dmesg
  cat: /var/log/dmesg: Permission denied

  [Where problems could occur]

  Some users or log scraper programs might need to view the kernel log
  buffers, and in this case, their underlying service accounts should be
  added to the 'adm' group.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/rsyslog/+bug/1912122/+subscriptions

-- 
Mailing list: https://launchpad.net/~touch-packages
Post to : touch-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~touch-packages
More help   : https://help.launchpad.net/ListHelp


[Touch-packages] [Bug 1912122] Re: /var/log/dmesg is 0644, should be 0640 to match new DMESG_RESTRICT restrictions

2021-01-17 Thread Matthew Ruffell
** Tags added: sts

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to rsyslog in Ubuntu.
https://bugs.launchpad.net/bugs/1912122

Title:
  /var/log/dmesg is 0644, should be 0640 to match new DMESG_RESTRICT
  restrictions

Status in rsyslog package in Ubuntu:
  In Progress
Status in rsyslog source package in Groovy:
  In Progress
Status in rsyslog source package in Hirsute:
  In Progress

Bug description:
  [Impact]

  In bug 1886112, CONFIG_SECURITY_DMESG_RESTRICT was enabled on the
  Ubuntu kernel starting with Groovy and onward, in an effort to
  restrict access to the kernel log buffer from unprivileged users.

  It seems we have overlooked /var/log/dmesg, as it is still mode 0644,
  while /var/log/kern.log, /var/log/syslog are all 0640:

  $ ll /var/log
  -rw-r--r--   1 root  adm 81768 Jan 18 09:09 dmesg
  -rw-r-   1 syslogadm 24538 Jan 18 13:05 
kern.log
  -rw-r-   1 syslogadm213911 Jan 18 13:22 syslog

  Change /var/log/dmesg to 0640 to close the information leak.

  [Testcase]

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ cat /var/log/kern.log
  cat: /var/log/kern.log: Permission denied
  $ cat /var/log/syslog
  cat: /var/log/syslog: Permission denied
  $ cat /var/log/dmesg
  [0.00] kernel: Linux version 5.8.0-36-generic 
(buildd@lgw01-amd64-011) (gcc (Ubuntu 10.2.1-2ubuntu3) 10.2.1 20201221, GNU ld 
(GNU Binutils for Ubuntu) 2.35.50.20210106) #40+21.04.1-Ubuntu SMP Thu Jan 7 
11:35:09 UTC 2021 (Ubuntu 5.8.0-36.40+21.04.1-generic 5.8.18)
  [0.00] kernel: Command line: BOOT_IMAGE=/casper/vmlinuz 
file=/cdrom/preseed/ubuntu.seed maybe-ubiquity quiet splash ---

  If you install the package in the following ppa:

  https://launchpad.net/~mruffell/+archive/ubuntu/lp1912122-test

  $ sudo systemctl daemon-reload
  $ sudo systemctl start dmesg.service

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ cat /var/log/kern.log
  cat: /var/log/kern.log: Permission denied
  $ cat /var/log/syslog
  cat: /var/log/syslog: Permission denied
  $ cat /var/log/dmesg
  cat: /var/log/dmesg: Permission denied

  [Where problems could occur]

  Some users or log scraper programs might need to view the kernel log
  buffers, and in this case, their underlying service accounts should be
  added to the 'adm' group.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/rsyslog/+bug/1912122/+subscriptions

-- 
Mailing list: https://launchpad.net/~touch-packages
Post to : touch-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~touch-packages
More help   : https://help.launchpad.net/ListHelp


[Touch-packages] [Bug 1912122] Re: /var/log/dmesg is 0644, should be 0640 to match new DMESG_RESTRICT restrictions

2021-01-17 Thread Matthew Ruffell
** Changed in: rsyslog (Ubuntu Hirsute)
   Status: New => In Progress

** Changed in: rsyslog (Ubuntu Hirsute)
   Importance: Undecided => Medium

** Changed in: rsyslog (Ubuntu Hirsute)
 Assignee: (unassigned) => Matthew Ruffell (mruffell)

** Description changed:

  [Impact]
  
  In bug 1886112, CONFIG_SECURITY_DMESG_RESTRICT was enabled on the Ubuntu
  kernel starting with Groovy and onward, in an effort to restrict access
  to the kernel log buffer from unprivileged users.
  
  It seems we have overlooked /var/log/dmesg, as it is still mode 0644,
  while /var/log/kern.log, /var/log/syslog are all 0640:
  
  $ ll /var/log
  -rw-r--r--   1 root  adm 81768 Jan 18 09:09 dmesg
  -rw-r-   1 syslogadm 24538 Jan 18 13:05 
kern.log
  -rw-r-   1 syslogadm213911 Jan 18 13:22 syslog
  
  Change /var/log/dmesg to 0640 to close the information leak.
  
  [Testcase]
  
  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ cat /var/log/kern.log
  cat: /var/log/kern.log: Permission denied
  $ cat /var/log/syslog
  cat: /var/log/syslog: Permission denied
  $ cat /var/log/dmesg
  [0.00] kernel: Linux version 5.8.0-36-generic 
(buildd@lgw01-amd64-011) (gcc (Ubuntu 10.2.1-2ubuntu3) 10.2.1 20201221, GNU ld 
(GNU Binutils for Ubuntu) 2.35.50.20210106) #40+21.04.1-Ubuntu SMP Thu Jan 7 
11:35:09 UTC 2021 (Ubuntu 5.8.0-36.40+21.04.1-generic 5.8.18)
  [0.00] kernel: Command line: BOOT_IMAGE=/casper/vmlinuz 
file=/cdrom/preseed/ubuntu.seed maybe-ubiquity quiet splash ---
  
  If you install the package in the following ppa:
  
+ https://launchpad.net/~mruffell/+archive/ubuntu/lp1912122-test
+ 
+ $ sudo systemctl daemon-reload
+ $ sudo systemctl start dmesg.service
+ 
  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ cat /var/log/kern.log
  cat: /var/log/kern.log: Permission denied
  $ cat /var/log/syslog
  cat: /var/log/syslog: Permission denied
  $ cat /var/log/dmesg
  cat: /var/log/dmesg: Permission denied
  
  [Where problems could occur]
  
  Some users or log scraper programs might need to view the kernel log
  buffers, and in this case, their underlying service accounts should be
  added to the 'adm' group.

** Changed in: rsyslog (Ubuntu Groovy)
   Status: New => In Progress

** Changed in: rsyslog (Ubuntu Groovy)
   Importance: Undecided => Medium

** Changed in: rsyslog (Ubuntu Groovy)
 Assignee: (unassigned) => Matthew Ruffell (mruffell)

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to rsyslog in Ubuntu.
https://bugs.launchpad.net/bugs/1912122

Title:
  /var/log/dmesg is 0644, should be 0640 to match new DMESG_RESTRICT
  restrictions

Status in rsyslog package in Ubuntu:
  In Progress
Status in rsyslog source package in Groovy:
  In Progress
Status in rsyslog source package in Hirsute:
  In Progress

Bug description:
  [Impact]

  In bug 1886112, CONFIG_SECURITY_DMESG_RESTRICT was enabled on the
  Ubuntu kernel starting with Groovy and onward, in an effort to
  restrict access to the kernel log buffer from unprivileged users.

  It seems we have overlooked /var/log/dmesg, as it is still mode 0644,
  while /var/log/kern.log, /var/log/syslog are all 0640:

  $ ll /var/log
  -rw-r--r--   1 root  adm 81768 Jan 18 09:09 dmesg
  -rw-r-   1 syslogadm 24538 Jan 18 13:05 
kern.log
  -rw-r-   1 syslogadm213911 Jan 18 13:22 syslog

  Change /var/log/dmesg to 0640 to close the information leak.

  [Testcase]

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ cat /var/log/kern.log
  cat: /var/log/kern.log: Permission denied
  $ cat /var/log/syslog
  cat: /var/log/syslog: Permission denied
  $ cat /var/log/dmesg
  [0.00] kernel: Linux version 5.8.0-36-generic 
(buildd@lgw01-amd64-011) (gcc (Ubuntu 10.2.1-2ubuntu3) 10.2.1 20201221, GNU ld 
(GNU Binutils for Ubuntu) 2.35.50.20210106) #40+21.04.1-Ubuntu SMP Thu Jan 7 
11:35:09 UTC 2021 (Ubuntu 5.8.0-36.40+21.04.1-generic 5.8.18)
  [0.00] kernel: Command line: BOOT_IMAGE=/casper/vmlinuz 
file=/cdrom/preseed/ubuntu.seed maybe-ubiquity quiet splash ---

  If you install the package in the following ppa:

  https://launchpad.net/~mruffell/+archive/ubuntu/lp1912122-test

  $ sudo systemctl daemon-reload
  $ sudo systemctl start dmesg.service

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ cat /var/log/kern.log
  cat: /var/log/kern.log: Permission denied
  $ cat /var/log/syslog
  cat: /var/log/syslog: Permission denied
  $ cat /var/log/dmesg
  cat: /var/log/dmesg: Permission denied

  [Where problems could occur]

  Some users or log scraper programs might need to view the kernel log
  buffers, and in this case, their underlying service accounts should be
  added to the 'adm' group.

To manage notifications about this bug go to:
https://bugs.launchpad.

[Touch-packages] [Bug 1912122] [NEW] /var/log/dmesg is 0644, should be 0640 to match new DMESG_RESTRICT restrictions

2021-01-17 Thread Matthew Ruffell
Public bug reported:

[Impact]

In bug 1886112, CONFIG_SECURITY_DMESG_RESTRICT was enabled on the Ubuntu
kernel starting with Groovy and onward, in an effort to restrict access
to the kernel log buffer from unprivileged users.

It seems we have overlooked /var/log/dmesg, as it is still mode 0644,
while /var/log/kern.log, /var/log/syslog are all 0640:

$ ll /var/log
-rw-r--r--   1 root  adm 81768 Jan 18 09:09 dmesg
-rw-r-   1 syslogadm 24538 Jan 18 13:05 kern.log
-rw-r-   1 syslogadm213911 Jan 18 13:22 syslog

Change /var/log/dmesg to 0640 to close the information leak.

[Testcase]

$ sudo adduser dave
$ su dave
$ groups
dave
$ cat /var/log/kern.log
cat: /var/log/kern.log: Permission denied
$ cat /var/log/syslog
cat: /var/log/syslog: Permission denied
$ cat /var/log/dmesg
[0.00] kernel: Linux version 5.8.0-36-generic (buildd@lgw01-amd64-011) 
(gcc (Ubuntu 10.2.1-2ubuntu3) 10.2.1 20201221, GNU ld (GNU Binutils for Ubuntu) 
2.35.50.20210106) #40+21.04.1-Ubuntu SMP Thu Jan 7 11:35:09 UTC 2021 (Ubuntu 
5.8.0-36.40+21.04.1-generic 5.8.18)
[0.00] kernel: Command line: BOOT_IMAGE=/casper/vmlinuz 
file=/cdrom/preseed/ubuntu.seed maybe-ubiquity quiet splash ---

If you install the package in the following ppa:

$ sudo adduser dave
$ su dave
$ groups
dave
$ cat /var/log/kern.log
cat: /var/log/kern.log: Permission denied
$ cat /var/log/syslog
cat: /var/log/syslog: Permission denied
$ cat /var/log/dmesg
cat: /var/log/dmesg: Permission denied

[Where problems could occur]

Some users or log scraper programs might need to view the kernel log
buffers, and in this case, their underlying service accounts should be
added to the 'adm' group.

** Affects: rsyslog (Ubuntu)
 Importance: Medium
 Assignee: Matthew Ruffell (mruffell)
 Status: In Progress

** Affects: rsyslog (Ubuntu Groovy)
 Importance: Undecided
 Status: New

** Affects: rsyslog (Ubuntu Hirsute)
 Importance: Medium
 Assignee: Matthew Ruffell (mruffell)
 Status: In Progress

** Also affects: rsyslog (Ubuntu Groovy)
   Importance: Undecided
   Status: New

** Also affects: rsyslog (Ubuntu Hirsute)
   Importance: Undecided
   Status: New

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to rsyslog in Ubuntu.
https://bugs.launchpad.net/bugs/1912122

Title:
  /var/log/dmesg is 0644, should be 0640 to match new DMESG_RESTRICT
  restrictions

Status in rsyslog package in Ubuntu:
  In Progress
Status in rsyslog source package in Groovy:
  New
Status in rsyslog source package in Hirsute:
  In Progress

Bug description:
  [Impact]

  In bug 1886112, CONFIG_SECURITY_DMESG_RESTRICT was enabled on the
  Ubuntu kernel starting with Groovy and onward, in an effort to
  restrict access to the kernel log buffer from unprivileged users.

  It seems we have overlooked /var/log/dmesg, as it is still mode 0644,
  while /var/log/kern.log, /var/log/syslog are all 0640:

  $ ll /var/log
  -rw-r--r--   1 root  adm 81768 Jan 18 09:09 dmesg
  -rw-r-   1 syslogadm 24538 Jan 18 13:05 
kern.log
  -rw-r-   1 syslogadm213911 Jan 18 13:22 syslog

  Change /var/log/dmesg to 0640 to close the information leak.

  [Testcase]

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ cat /var/log/kern.log
  cat: /var/log/kern.log: Permission denied
  $ cat /var/log/syslog
  cat: /var/log/syslog: Permission denied
  $ cat /var/log/dmesg
  [0.00] kernel: Linux version 5.8.0-36-generic 
(buildd@lgw01-amd64-011) (gcc (Ubuntu 10.2.1-2ubuntu3) 10.2.1 20201221, GNU ld 
(GNU Binutils for Ubuntu) 2.35.50.20210106) #40+21.04.1-Ubuntu SMP Thu Jan 7 
11:35:09 UTC 2021 (Ubuntu 5.8.0-36.40+21.04.1-generic 5.8.18)
  [0.00] kernel: Command line: BOOT_IMAGE=/casper/vmlinuz 
file=/cdrom/preseed/ubuntu.seed maybe-ubiquity quiet splash ---

  If you install the package in the following ppa:

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ cat /var/log/kern.log
  cat: /var/log/kern.log: Permission denied
  $ cat /var/log/syslog
  cat: /var/log/syslog: Permission denied
  $ cat /var/log/dmesg
  cat: /var/log/dmesg: Permission denied

  [Where problems could occur]

  Some users or log scraper programs might need to view the kernel log
  buffers, and in this case, their underlying service accounts should be
  added to the 'adm' group.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/rsyslog/+bug/1912122/+subscriptions

-- 
Mailing list: https://launchpad.net/~touch-packages
Post to : touch-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~touch-packages
More help   : https://help.launchpad.net/ListHelp


[Touch-packages] [Bug 1908473] Re: rsyslog-relp: imrelp module leaves sockets in CLOSE_WAIT state which leads to file descriptor leak

2021-01-05 Thread Matthew Ruffell
** Tags added: sts-sponsor

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to rsyslog in Ubuntu.
https://bugs.launchpad.net/bugs/1908473

Title:
  rsyslog-relp: imrelp module leaves sockets in CLOSE_WAIT state which
  leads to file descriptor leak

Status in librelp package in Ubuntu:
  In Progress
Status in rsyslog package in Ubuntu:
  Fix Released
Status in librelp source package in Focal:
  In Progress
Status in rsyslog source package in Focal:
  Won't Fix
Status in librelp source package in Groovy:
  In Progress
Status in rsyslog source package in Groovy:
  Fix Released
Status in librelp source package in Hirsute:
  In Progress
Status in rsyslog source package in Hirsute:
  Fix Released

Bug description:
  [Impact]

  In recent versions of rsyslog and librelp, the imrelp module leaks
  file descriptors due to a bug where it does not correctly close
  sockets, and instead, leaves them in the CLOSE_WAIT state.

  This causes rsyslogd on busy servers to eventually hit the limit of
  maximum open files allowed, which locks rsyslogd up until it is
  restarted.

  A workaround is to restart rsyslogd every month or so to manually
  close all of the open sockets.

  Only users of the imrelp module are affected, and not rsyslog users in
  general.

  [Testcase]

  Install the rsyslog-relp module like so:

  $ sudo apt install rsyslog rsyslog-relp

  Next, generate a working directory, and make a config file that loads
  the relp module.

  $ sudo mkdir /workdir
  $ cat << EOF >> ./spool.conf
  \$LocalHostName spool
  \$AbortOnUncleanConfig on
  \$PreserveFQDN on

  global(
  workDirectory="/workdir"
  maxMessageSize="256k"
  )

  main_queue(queue.type="Direct")
  module(load="imrelp")
  input(
  type="imrelp"
  name="imrelp"
  port="601"
  ruleset="spool"
  MaxDataSize="256k"
  )

  ruleset(name="spool" queue.type="direct") {
  }

  # Just so rsyslog doesn't whine that we do not have outputs
  ruleset(name="noop" queue.type="direct") {
  action(
  type="omfile"
  name="omfile"
  file="/workdir/spool.log"
  )
  }
  EOF

  Verify that the config is valid, then start a rsyslog server.

  $ sudo rsyslogd -f ./spool.conf -N9
  $ sudo rsyslogd -f ./spool.conf -i /workdir/rsyslogd.pid

  Fetch the rsyslogd PID and check for open files.

  $ RLOGPID=$(cat /workdir/rsyslogd.pid)
  $ sudo ls -l /proc/$RLOGPID/fd
  total 0
  lr-x-- 1 root root 64 Dec 17 01:22 0 -> /dev/urandom
  lrwx-- 1 root root 64 Dec 17 01:22 1 -> 'socket:[41228]'
  lrwx-- 1 root root 64 Dec 17 01:22 3 -> 'socket:[41222]'
  lrwx-- 1 root root 64 Dec 17 01:22 4 -> 'socket:[41223]'
  lrwx-- 1 root root 64 Dec 17 01:22 7 -> 'anon_inode:[eventpoll]'

  We have 3 sockets open by default. Next, use netcat to open 100
  connections:

  $ for i in {1..100} ; do nc -z 127.0.0.1 601 ; done

  Now check for open file descriptors, and there will be an extra 100 sockets
  in the list:

  $ sudo ls -l /proc/$RLOGPID/fd

  https://paste.ubuntu.com/p/f6NQVNbZcR/

  We can check the state of these sockets with:

  $ ss -t

  https://paste.ubuntu.com/p/7Ts2FbxJrg/

  The listening sockets will be in CLOSE-WAIT, and the netcat sockets
  will be in FIN-WAIT-2.

  $ ss -t | grep CLOSE-WAIT | wc -l
  100

  If you install the test package available in the following ppa:

  https://launchpad.net/~mruffell/+archive/ubuntu/sf299578-test

  When you open connections with netcat, these will be closed properly,
  and the file descriptor leak will be fixed.

  [Where problems could occur]

  If a regression were to occur, it would be limited to users of the
  imrelp module, which is a part of the rsyslogd-relp package, and
  depends on librelp.

  rsyslog-relp is not part of a default installation of rsyslog, and is
  opt in by changing a configuration file to enable imrelp.

  The changes to rsyslog implement a testcase which exercises the
  problematic code to ensure things are working as expected, and should
  run during autopkgtest time.

  [Other]

  Upstream bug list:

  https://github.com/rsyslog/rsyslog/issues/4350
  https://github.com/rsyslog/rsyslog/issues/4005
  https://github.com/rsyslog/librelp/issues/188
  https://github.com/rsyslog/librelp/pull/193

  The following commits fix the problem:

  rsyslogd
  

  commit baee0bd5420649329793746f0daf87c4f59fe6a6
  Author: Andre lorbach 
  Date:   Thu Apr 9 13:00:35 2020 +0200
  Subject: testbench: Add test for imrelp to check broken session handling.
  Link: 
https://github.com/rsyslog/rsyslog/commit/baee0bd5420649329793746f0daf87c4f59fe6a6

  librelp
  ===

  commit 7907c9c57f6ed94c8ce5a4e63c3c4e019f71cff0
  Author: Andre lorbach 
  Date:   Mon May 11 14:59:55 2020 +0200
  Subject: fix memory leak on session break.
  Link: 
https://github.com/rsyslog/librelp/commit/7907c9c57f6ed94c8ce5a4e63c3c4e019f71cff0

  commit 4a6ad8637c244fd3a1caeb9a93950826f58e956a
  

[Touch-packages] [Bug 1908473] Re: rsyslog-relp: imrelp module leaves sockets in CLOSE_WAIT state which leads to file descriptor leak

2021-01-05 Thread Matthew Ruffell
Attached is the debdiff for librelp for Focal

** Patch added: "Debdiff for librelp on Focal"
   
https://bugs.launchpad.net/ubuntu/+source/rsyslog/+bug/1908473/+attachment/5449602/+files/librelp_focal.debdiff

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to rsyslog in Ubuntu.
https://bugs.launchpad.net/bugs/1908473

Title:
  rsyslog-relp: imrelp module leaves sockets in CLOSE_WAIT state which
  leads to file descriptor leak

Status in librelp package in Ubuntu:
  In Progress
Status in rsyslog package in Ubuntu:
  Fix Released
Status in librelp source package in Focal:
  In Progress
Status in rsyslog source package in Focal:
  Won't Fix
Status in librelp source package in Groovy:
  In Progress
Status in rsyslog source package in Groovy:
  Fix Released
Status in librelp source package in Hirsute:
  In Progress
Status in rsyslog source package in Hirsute:
  Fix Released

Bug description:
  [Impact]

  In recent versions of rsyslog and librelp, the imrelp module leaks
  file descriptors due to a bug where it does not correctly close
  sockets, and instead, leaves them in the CLOSE_WAIT state.

  This causes rsyslogd on busy servers to eventually hit the limit of
  maximum open files allowed, which locks rsyslogd up until it is
  restarted.

  A workaround is to restart rsyslogd every month or so to manually
  close all of the open sockets.

  Only users of the imrelp module are affected, and not rsyslog users in
  general.

  [Testcase]

  Install the rsyslog-relp module like so:

  $ sudo apt install rsyslog rsyslog-relp

  Next, generate a working directory, and make a config file that loads
  the relp module.

  $ sudo mkdir /workdir
  $ cat << EOF >> ./spool.conf
  \$LocalHostName spool
  \$AbortOnUncleanConfig on
  \$PreserveFQDN on

  global(
  workDirectory="/workdir"
  maxMessageSize="256k"
  )

  main_queue(queue.type="Direct")
  module(load="imrelp")
  input(
  type="imrelp"
  name="imrelp"
  port="601"
  ruleset="spool"
  MaxDataSize="256k"
  )

  ruleset(name="spool" queue.type="direct") {
  }

  # Just so rsyslog doesn't whine that we do not have outputs
  ruleset(name="noop" queue.type="direct") {
  action(
  type="omfile"
  name="omfile"
  file="/workdir/spool.log"
  )
  }
  EOF

  Verify that the config is valid, then start a rsyslog server.

  $ sudo rsyslogd -f ./spool.conf -N9
  $ sudo rsyslogd -f ./spool.conf -i /workdir/rsyslogd.pid

  Fetch the rsyslogd PID and check for open files.

  $ RLOGPID=$(cat /workdir/rsyslogd.pid)
  $ sudo ls -l /proc/$RLOGPID/fd
  total 0
  lr-x-- 1 root root 64 Dec 17 01:22 0 -> /dev/urandom
  lrwx-- 1 root root 64 Dec 17 01:22 1 -> 'socket:[41228]'
  lrwx-- 1 root root 64 Dec 17 01:22 3 -> 'socket:[41222]'
  lrwx-- 1 root root 64 Dec 17 01:22 4 -> 'socket:[41223]'
  lrwx-- 1 root root 64 Dec 17 01:22 7 -> 'anon_inode:[eventpoll]'

  We have 3 sockets open by default. Next, use netcat to open 100
  connections:

  $ for i in {1..100} ; do nc -z 127.0.0.1 601 ; done

  Now check for open file descriptors, and there will be an extra 100 sockets
  in the list:

  $ sudo ls -l /proc/$RLOGPID/fd

  https://paste.ubuntu.com/p/f6NQVNbZcR/

  We can check the state of these sockets with:

  $ ss -t

  https://paste.ubuntu.com/p/7Ts2FbxJrg/

  The listening sockets will be in CLOSE-WAIT, and the netcat sockets
  will be in FIN-WAIT-2.

  $ ss -t | grep CLOSE-WAIT | wc -l
  100

  If you install the test package available in the following ppa:

  https://launchpad.net/~mruffell/+archive/ubuntu/sf299578-test

  When you open connections with netcat, these will be closed properly,
  and the file descriptor leak will be fixed.

  [Where problems could occur]

  If a regression were to occur, it would be limited to users of the
  imrelp module, which is a part of the rsyslogd-relp package, and
  depends on librelp.

  rsyslog-relp is not part of a default installation of rsyslog, and is
  opt in by changing a configuration file to enable imrelp.

  The changes to rsyslog implement a testcase which exercises the
  problematic code to ensure things are working as expected, and should
  run during autopkgtest time.

  [Other]

  Upstream bug list:

  https://github.com/rsyslog/rsyslog/issues/4350
  https://github.com/rsyslog/rsyslog/issues/4005
  https://github.com/rsyslog/librelp/issues/188
  https://github.com/rsyslog/librelp/pull/193

  The following commits fix the problem:

  rsyslogd
  

  commit baee0bd5420649329793746f0daf87c4f59fe6a6
  Author: Andre lorbach 
  Date:   Thu Apr 9 13:00:35 2020 +0200
  Subject: testbench: Add test for imrelp to check broken session handling.
  Link: 
https://github.com/rsyslog/rsyslog/commit/baee0bd5420649329793746f0daf87c4f59fe6a6

  librelp
  ===

  commit 7907c9c57f6ed94c8ce5a4e63c3c4e019f71cff0
  Author: Andre lorbach 
  Date:   Mon May 11 14:59:55 2020 +0200
  Subject: 

[Touch-packages] [Bug 1908473] Re: rsyslog-relp: imrelp module leaves sockets in CLOSE_WAIT state which leads to file descriptor leak

2021-01-05 Thread Matthew Ruffell
Attached is the debdiff for librelp for Groovy

** Patch added: "Debdiff for librelp on Groovy"
   
https://bugs.launchpad.net/ubuntu/+source/rsyslog/+bug/1908473/+attachment/5449601/+files/librelp_groovy.debdiff

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to rsyslog in Ubuntu.
https://bugs.launchpad.net/bugs/1908473

Title:
  rsyslog-relp: imrelp module leaves sockets in CLOSE_WAIT state which
  leads to file descriptor leak

Status in librelp package in Ubuntu:
  In Progress
Status in rsyslog package in Ubuntu:
  Fix Released
Status in librelp source package in Focal:
  In Progress
Status in rsyslog source package in Focal:
  Won't Fix
Status in librelp source package in Groovy:
  In Progress
Status in rsyslog source package in Groovy:
  Fix Released
Status in librelp source package in Hirsute:
  In Progress
Status in rsyslog source package in Hirsute:
  Fix Released

Bug description:
  [Impact]

  In recent versions of rsyslog and librelp, the imrelp module leaks
  file descriptors due to a bug where it does not correctly close
  sockets, and instead, leaves them in the CLOSE_WAIT state.

  This causes rsyslogd on busy servers to eventually hit the limit of
  maximum open files allowed, which locks rsyslogd up until it is
  restarted.

  A workaround is to restart rsyslogd every month or so to manually
  close all of the open sockets.

  Only users of the imrelp module are affected, and not rsyslog users in
  general.

  [Testcase]

  Install the rsyslog-relp module like so:

  $ sudo apt install rsyslog rsyslog-relp

  Next, generate a working directory, and make a config file that loads
  the relp module.

  $ sudo mkdir /workdir
  $ cat << EOF >> ./spool.conf
  \$LocalHostName spool
  \$AbortOnUncleanConfig on
  \$PreserveFQDN on

  global(
  workDirectory="/workdir"
  maxMessageSize="256k"
  )

  main_queue(queue.type="Direct")
  module(load="imrelp")
  input(
  type="imrelp"
  name="imrelp"
  port="601"
  ruleset="spool"
  MaxDataSize="256k"
  )

  ruleset(name="spool" queue.type="direct") {
  }

  # Just so rsyslog doesn't whine that we do not have outputs
  ruleset(name="noop" queue.type="direct") {
  action(
  type="omfile"
  name="omfile"
  file="/workdir/spool.log"
  )
  }
  EOF

  Verify that the config is valid, then start a rsyslog server.

  $ sudo rsyslogd -f ./spool.conf -N9
  $ sudo rsyslogd -f ./spool.conf -i /workdir/rsyslogd.pid

  Fetch the rsyslogd PID and check for open files.

  $ RLOGPID=$(cat /workdir/rsyslogd.pid)
  $ sudo ls -l /proc/$RLOGPID/fd
  total 0
  lr-x-- 1 root root 64 Dec 17 01:22 0 -> /dev/urandom
  lrwx-- 1 root root 64 Dec 17 01:22 1 -> 'socket:[41228]'
  lrwx-- 1 root root 64 Dec 17 01:22 3 -> 'socket:[41222]'
  lrwx-- 1 root root 64 Dec 17 01:22 4 -> 'socket:[41223]'
  lrwx-- 1 root root 64 Dec 17 01:22 7 -> 'anon_inode:[eventpoll]'

  We have 3 sockets open by default. Next, use netcat to open 100
  connections:

  $ for i in {1..100} ; do nc -z 127.0.0.1 601 ; done

  Now check for open file descriptors, and there will be an extra 100 sockets
  in the list:

  $ sudo ls -l /proc/$RLOGPID/fd

  https://paste.ubuntu.com/p/f6NQVNbZcR/

  We can check the state of these sockets with:

  $ ss -t

  https://paste.ubuntu.com/p/7Ts2FbxJrg/

  The listening sockets will be in CLOSE-WAIT, and the netcat sockets
  will be in FIN-WAIT-2.

  $ ss -t | grep CLOSE-WAIT | wc -l
  100

  If you install the test package available in the following ppa:

  https://launchpad.net/~mruffell/+archive/ubuntu/sf299578-test

  When you open connections with netcat, these will be closed properly,
  and the file descriptor leak will be fixed.

  [Where problems could occur]

  If a regression were to occur, it would be limited to users of the
  imrelp module, which is a part of the rsyslogd-relp package, and
  depends on librelp.

  rsyslog-relp is not part of a default installation of rsyslog, and is
  opt in by changing a configuration file to enable imrelp.

  The changes to rsyslog implement a testcase which exercises the
  problematic code to ensure things are working as expected, and should
  run during autopkgtest time.

  [Other]

  Upstream bug list:

  https://github.com/rsyslog/rsyslog/issues/4350
  https://github.com/rsyslog/rsyslog/issues/4005
  https://github.com/rsyslog/librelp/issues/188
  https://github.com/rsyslog/librelp/pull/193

  The following commits fix the problem:

  rsyslogd
  

  commit baee0bd5420649329793746f0daf87c4f59fe6a6
  Author: Andre lorbach 
  Date:   Thu Apr 9 13:00:35 2020 +0200
  Subject: testbench: Add test for imrelp to check broken session handling.
  Link: 
https://github.com/rsyslog/rsyslog/commit/baee0bd5420649329793746f0daf87c4f59fe6a6

  librelp
  ===

  commit 7907c9c57f6ed94c8ce5a4e63c3c4e019f71cff0
  Author: Andre lorbach 
  Date:   Mon May 11 14:59:55 2020 +0200
  

[Touch-packages] [Bug 1908473] Re: rsyslog-relp: imrelp module leaves sockets in CLOSE_WAIT state which leads to file descriptor leak

2021-01-05 Thread Matthew Ruffell
** Description changed:

  [Impact]
  
  In recent versions of rsyslog and librelp, the imrelp module leaks file
  descriptors due to a bug where it does not correctly close sockets, and
  instead, leaves them in the CLOSE_WAIT state.
  
  This causes rsyslogd on busy servers to eventually hit the limit of
  maximum open files allowed, which locks rsyslogd up until it is
  restarted.
  
  A workaround is to restart rsyslogd every month or so to manually close
  all of the open sockets.
  
  Only users of the imrelp module are affected, and not rsyslog users in
  general.
  
  [Testcase]
  
  Install the rsyslog-relp module like so:
  
  $ sudo apt install rsyslog rsyslog-relp
  
  Next, generate a working directory, and make a config file that loads
  the relp module.
  
  $ sudo mkdir /workdir
  $ cat << EOF >> ./spool.conf
  \$LocalHostName spool
  \$AbortOnUncleanConfig on
  \$PreserveFQDN on
  
  global(
  workDirectory="/workdir"
  maxMessageSize="256k"
  )
  
  main_queue(queue.type="Direct")
  module(load="imrelp")
  input(
  type="imrelp"
  name="imrelp"
  port="601"
  ruleset="spool"
  MaxDataSize="256k"
  )
  
  ruleset(name="spool" queue.type="direct") {
  }
  
  # Just so rsyslog doesn't whine that we do not have outputs
  ruleset(name="noop" queue.type="direct") {
  action(
  type="omfile"
  name="omfile"
  file="/workdir/spool.log"
  )
  }
  EOF
  
  Verify that the config is valid, then start a rsyslog server.
  
  $ sudo rsyslogd -f ./spool.conf -N9
  $ sudo rsyslogd -f ./spool.conf -i /workdir/rsyslogd.pid
  
  Fetch the rsyslogd PID and check for open files.
  
  $ RLOGPID=$(cat /workdir/rsyslogd.pid)
  $ sudo ls -l /proc/$RLOGPID/fd
  total 0
  lr-x-- 1 root root 64 Dec 17 01:22 0 -> /dev/urandom
  lrwx-- 1 root root 64 Dec 17 01:22 1 -> 'socket:[41228]'
  lrwx-- 1 root root 64 Dec 17 01:22 3 -> 'socket:[41222]'
  lrwx-- 1 root root 64 Dec 17 01:22 4 -> 'socket:[41223]'
  lrwx-- 1 root root 64 Dec 17 01:22 7 -> 'anon_inode:[eventpoll]'
  
  We have 3 sockets open by default. Next, use netcat to open 100
  connections:
  
  $ for i in {1..100} ; do nc -z 127.0.0.1 601 ; done
  
  Now check for open file descriptors, and there will be an extra 100 sockets
  in the list:
  
  $ sudo ls -l /proc/$RLOGPID/fd
  
  https://paste.ubuntu.com/p/f6NQVNbZcR/
  
  We can check the state of these sockets with:
  
  $ ss -t
  
  https://paste.ubuntu.com/p/7Ts2FbxJrg/
  
  The listening sockets will be in CLOSE-WAIT, and the netcat sockets will
  be in FIN-WAIT-2.
  
+ $ ss -t | grep CLOSE-WAIT | wc -l
+ 100
+ 
  If you install the test package available in the following ppa:
  
  https://launchpad.net/~mruffell/+archive/ubuntu/sf299578-test
  
  When you open connections with netcat, these will be closed properly,
  and the file descriptor leak will be fixed.
  
  [Where problems could occur]
  
  If a regression were to occur, it would be limited to users of the
  imrelp module, which is a part of the rsyslogd-relp package, and depends
  on librelp.
  
  rsyslog-relp is not part of a default installation of rsyslog, and is
  opt in by changing a configuration file to enable imrelp.
  
  The changes to rsyslog implement a testcase which exercises the
  problematic code to ensure things are working as expected, and should
  run during autopkgtest time.
  
  [Other]
  
  Upstream bug list:
  
  https://github.com/rsyslog/rsyslog/issues/4350
  https://github.com/rsyslog/rsyslog/issues/4005
  https://github.com/rsyslog/librelp/issues/188
  https://github.com/rsyslog/librelp/pull/193
  
  The following commits fix the problem:
  
  rsyslogd
  
  
  commit baee0bd5420649329793746f0daf87c4f59fe6a6
  Author: Andre lorbach 
  Date:   Thu Apr 9 13:00:35 2020 +0200
  Subject: testbench: Add test for imrelp to check broken session handling.
  Link: 
https://github.com/rsyslog/rsyslog/commit/baee0bd5420649329793746f0daf87c4f59fe6a6
  
  librelp
  ===
  
  commit 7907c9c57f6ed94c8ce5a4e63c3c4e019f71cff0
  Author: Andre lorbach 
  Date:   Mon May 11 14:59:55 2020 +0200
  Subject: fix memory leak on session break.
  Link: 
https://github.com/rsyslog/librelp/commit/7907c9c57f6ed94c8ce5a4e63c3c4e019f71cff0
  
  commit 4a6ad8637c244fd3a1caeb9a93950826f58e956a
  Author: Andre lorbach 
  Date:   Wed Apr 8 15:55:32 2020 +0200
  Subject: replsess: fix double free of sendbuf in some cases.
  Link: 
https://github.com/rsyslog/librelp/commit/4a6ad8637c244fd3a1caeb9a93950826f58e956a
  
  commit 3797944fb62273fa1164acd3104f0894b337c4d0
  Author: Ognyan Kulev 
  Date:   Mon Jun 15 14:10:08 2020 +0300
  Subject: Fix FD leak when socket shutdown is one-sided
  Link: 
https://github.com/rsyslog/librelp/commit/3797944fb62273fa1164acd3104f0894b337c4d0

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to rsyslog in Ubuntu.
https://bugs.launchpad.net/bugs/1908473

Title:
  

[Touch-packages] [Bug 1908473] Re: rsyslog-relp: imrelp module leaves sockets in CLOSE_WAIT state which leads to file descriptor leak

2021-01-02 Thread Matthew Ruffell
Since we only need to change librelp to fix the problem, we won't SRU
the testcase to Focal's rsyslog, since there is no need to risk a
regression for a testcase, which would effectively be a no-change
rebuild.

The testcase is already present in rsyslog on Groovy and Hirsute,
marking as released.

I found an extra commit we need for librelp, which is "Fix FD leak when
socket shutdown is one-sided". This is needed for the netcat testcase
and for basic heartbeat programs like load balancers etc, which open a
port to the rsyslog relp port, but doesn't necessarily speak relp
protocol over that port.

Building test packages now since my internal tests are now successful
once I included "Fix FD leak when socket shutdown is one-sided".

** Description changed:

  [Impact]
  
  In recent versions of rsyslog and librelp, the imrelp module leaks file
  descriptors due to a bug where it does not correctly close sockets, and
  instead, leaves them in the CLOSE_WAIT state.
  
  This causes rsyslogd on busy servers to eventually hit the limit of
  maximum open files allowed, which locks rsyslogd up until it is
  restarted.
  
  A workaround is to restart rsyslogd every month or so to manually close
  all of the open sockets.
  
  Only users of the imrelp module are affected, and not rsyslog users in
  general.
  
  [Testcase]
  
  Install the rsyslog-relp module like so:
  
  $ sudo apt install rsyslog rsyslog-relp
  
  Next, generate a working directory, and make a config file that loads
  the relp module.
  
  $ sudo mkdir /workdir
  $ cat << EOF >> ./spool.conf
  \$LocalHostName spool
  \$AbortOnUncleanConfig on
  \$PreserveFQDN on
  
  global(
- workDirectory="/workdir"
- maxMessageSize="256k"
+ workDirectory="/workdir"
+ maxMessageSize="256k"
  )
  
  main_queue(queue.type="Direct")
  module(load="imrelp")
  input(
- type="imrelp"
- name="imrelp"
- port="601"
- ruleset="spool"
- MaxDataSize="256k"
+ type="imrelp"
+ name="imrelp"
+ port="601"
+ ruleset="spool"
+ MaxDataSize="256k"
  )
  
  ruleset(name="spool" queue.type="direct") {
  }
  
  # Just so rsyslog doesn't whine that we do not have outputs
  ruleset(name="noop" queue.type="direct") {
- action(
- type="omfile"
- name="omfile"
- file="/workdir/spool.log"
- )
+ action(
+ type="omfile"
+ name="omfile"
+ file="/workdir/spool.log"
+ )
  }
  EOF
  
  Verify that the config is valid, then start a rsyslog server.
  
  $ sudo rsyslogd -f ./spool.conf -N9
  $ sudo rsyslogd -f ./spool.conf -i /workdir/rsyslogd.pid
  
  Fetch the rsyslogd PID and check for open files.
  
  $ RLOGPID=$(cat /workdir/rsyslogd.pid)
  $ sudo ls -l /proc/$RLOGPID/fd
  total 0
  lr-x-- 1 root root 64 Dec 17 01:22 0 -> /dev/urandom
  lrwx-- 1 root root 64 Dec 17 01:22 1 -> 'socket:[41228]'
  lrwx-- 1 root root 64 Dec 17 01:22 3 -> 'socket:[41222]'
  lrwx-- 1 root root 64 Dec 17 01:22 4 -> 'socket:[41223]'
  lrwx-- 1 root root 64 Dec 17 01:22 7 -> 'anon_inode:[eventpoll]'
  
  We have 3 sockets open by default. Next, use netcat to open 100
  connections:
  
  $ for i in {1..100} ; do nc -z 127.0.0.1 601 ; done
  
  Now check for open file descriptors, and there will be an extra 100 sockets
  in the list:
  
  $ sudo ls -l /proc/$RLOGPID/fd
  
  https://paste.ubuntu.com/p/f6NQVNbZcR/
  
  We can check the state of these sockets with:
  
  $ ss -t
  
  https://paste.ubuntu.com/p/7Ts2FbxJrg/
  
  The listening sockets will be in CLOSE-WAIT, and the netcat sockets will
  be in FIN-WAIT-2.
  
  If you install the test package available in the following ppa:
  
  https://launchpad.net/~mruffell/+archive/ubuntu/sf299578-test
  
  When you open connections with netcat, these will be closed properly,
  and the file descriptor leak will be fixed.
  
  [Where problems could occur]
  
  If a regression were to occur, it would be limited to users of the
  imrelp module, which is a part of the rsyslogd-relp package, and depends
  on librelp.
  
  rsyslog-relp is not part of a default installation of rsyslog, and is
  opt in by changing a configuration file to enable imrelp.
  
  The changes to rsyslog implement a testcase which exercises the
  problematic code to ensure things are working as expected, and should
  run during autopkgtest time.
  
  [Other]
  
  Upstream bug list:
  
  https://github.com/rsyslog/rsyslog/issues/4350
  https://github.com/rsyslog/rsyslog/issues/4005
  https://github.com/rsyslog/librelp/issues/188
+ https://github.com/rsyslog/librelp/pull/193
  
  The following commits fix the problem:
  
  rsyslogd
  
  
  commit baee0bd5420649329793746f0daf87c4f59fe6a6
  Author: Andre lorbach 
  Date:   Thu Apr 9 13:00:35 2020 +0200
  Subject: testbench: Add test for imrelp to check broken session handling.
  Link: 
https://github.com/rsyslog/rsyslog/commit/baee0bd5420649329793746f0daf87c4f59fe6a6
  
  librelp
  ===
  
  commit 

[Touch-packages] [Bug 1908473] [NEW] rsyslog-relp: imrelp module leaves sockets in CLOSE_WAIT state which leads to file descriptor leak

2020-12-16 Thread Matthew Ruffell
Public bug reported:

[Impact]

In recent versions of rsyslog and librelp, the imrelp module leaks file
descriptors due to a bug where it does not correctly close sockets, and
instead, leaves them in the CLOSE_WAIT state.

This causes rsyslogd on busy servers to eventually hit the limit of
maximum open files allowed, which locks rsyslogd up until it is
restarted.

A workaround is to restart rsyslogd every month or so to manually close
all of the open sockets.

Only users of the imrelp module are affected, and not rsyslog users in
general.

[Testcase]

Install the rsyslog-relp module like so:

$ sudo apt install rsyslog rsyslog-relp

Next, generate a working directory, and make a config file that loads
the relp module.

$ sudo mkdir /workdir
$ cat << EOF >> ./spool.conf
\$LocalHostName spool
\$AbortOnUncleanConfig on
\$PreserveFQDN on

global(
workDirectory="/workdir"
maxMessageSize="256k"
)

main_queue(queue.type="Direct")
module(load="imrelp")
input(
type="imrelp"
name="imrelp"
port="601"
ruleset="spool"
MaxDataSize="256k"
)

ruleset(name="spool" queue.type="direct") {
}

# Just so rsyslog doesn't whine that we do not have outputs
ruleset(name="noop" queue.type="direct") {
action(
type="omfile"
name="omfile"
file="/workdir/spool.log"
)
}
EOF

Verify that the config is valid, then start a rsyslog server.

$ sudo rsyslogd -f ./spool.conf -N9
$ sudo rsyslogd -f ./spool.conf -i /workdir/rsyslogd.pid

Fetch the rsyslogd PID and check for open files.

$ RLOGPID=$(cat /workdir/rsyslogd.pid)
$ sudo ls -l /proc/$RLOGPID/fd
total 0
lr-x-- 1 root root 64 Dec 17 01:22 0 -> /dev/urandom
lrwx-- 1 root root 64 Dec 17 01:22 1 -> 'socket:[41228]'
lrwx-- 1 root root 64 Dec 17 01:22 3 -> 'socket:[41222]'
lrwx-- 1 root root 64 Dec 17 01:22 4 -> 'socket:[41223]'
lrwx-- 1 root root 64 Dec 17 01:22 7 -> 'anon_inode:[eventpoll]'

We have 3 sockets open by default. Next, use netcat to open 100
connections:

$ for i in {1..100} ; do nc -z 127.0.0.1 601 ; done

Now check for open file descriptors, and there will be an extra 100 sockets
in the list:

$ sudo ls -l /proc/$RLOGPID/fd

https://paste.ubuntu.com/p/f6NQVNbZcR/

We can check the state of these sockets with:

$ ss -t

https://paste.ubuntu.com/p/7Ts2FbxJrg/

The listening sockets will be in CLOSE-WAIT, and the netcat sockets will
be in FIN-WAIT-2.

If you install the test package available in the following ppa:

https://launchpad.net/~mruffell/+archive/ubuntu/sf299578-test

When you open connections with netcat, these will be closed properly,
and the file descriptor leak will be fixed.

[Where problems could occur]

If a regression were to occur, it would be limited to users of the
imrelp module, which is a part of the rsyslogd-relp package, and depends
on librelp.

rsyslog-relp is not part of a default installation of rsyslog, and is
opt in by changing a configuration file to enable imrelp.

The changes to rsyslog implement a testcase which exercises the
problematic code to ensure things are working as expected, and should
run during autopkgtest time.

[Other]

Upstream bug list:

https://github.com/rsyslog/rsyslog/issues/4350
https://github.com/rsyslog/rsyslog/issues/4005
https://github.com/rsyslog/librelp/issues/188

The following commits fix the problem:

rsyslogd


commit baee0bd5420649329793746f0daf87c4f59fe6a6
Author: Andre lorbach 
Date:   Thu Apr 9 13:00:35 2020 +0200
Subject: testbench: Add test for imrelp to check broken session handling.
Link: 
https://github.com/rsyslog/rsyslog/commit/baee0bd5420649329793746f0daf87c4f59fe6a6

librelp
===

commit 7907c9c57f6ed94c8ce5a4e63c3c4e019f71cff0
Author: Andre lorbach 
Date:   Mon May 11 14:59:55 2020 +0200
Subject: fix memory leak on session break.
Link: 
https://github.com/rsyslog/librelp/commit/7907c9c57f6ed94c8ce5a4e63c3c4e019f71cff0

commit 4a6ad8637c244fd3a1caeb9a93950826f58e956a
Author: Andre lorbach 
Date:   Wed Apr 8 15:55:32 2020 +0200
Subject: replsess: fix double free of sendbuf in some cases.
Link: 
https://github.com/rsyslog/librelp/commit/4a6ad8637c244fd3a1caeb9a93950826f58e956a

** Affects: librelp (Ubuntu)
 Importance: Medium
 Assignee: Matthew Ruffell (mruffell)
 Status: In Progress

** Affects: rsyslog (Ubuntu)
 Importance: Medium
 Assignee: Matthew Ruffell (mruffell)
 Status: In Progress

** Affects: librelp (Ubuntu Focal)
 Importance: Medium
 Assignee: Matthew Ruffell (mruffell)
 Status: In Progress

** Affects: rsyslog (Ubuntu Focal)
 Importance: Medium
 Assignee: Matthew Ruffell (mruffell)
 Status: In Progress

** Affects: librelp (Ubuntu Groovy)
 Importance: Medium
 Assignee: Matthew Ruffell (mruffell)
 Status: In Progress

** Affects: rsyslog

[Touch-packages] [Bug 1906627] Re: GSS-SPNEGO implementation in cyrus-sasl2 is incompatible with Active Directory, causing recent adcli regression

2020-12-15 Thread Matthew Ruffell
To anyone following this bug:

As we get ready to re-release the new adcli package which implements the
--use-ldaps flag, if you are happy to spend a few moments testing the
new package, I would really appreciate it. I really don't want to cause
another regression again.

You can install the new adcli package in -proposed like so:

Enable -proposed by running the following command to make a new sources.list.d 
entry:
1) cat << EOF | sudo tee /etc/apt/sources.list.d/ubuntu-$(lsb_release 
-cs)-proposed.list
# Enable Ubuntu proposed archive
deb http://archive.ubuntu.com/ubuntu/ $(lsb_release -cs)-proposed main universe
EOF
2) sudo apt update 
3) sudo apt install adcli
4) sudo apt-cache policy adcli | grep Installed
Installed: 0.8.2-1ubuntu1.2
5) sudo apt-cache policy libsasl2-modules-gssapi-mit | grep Installed
Installed: 2.1.27~101-g0780600+dfsg-3ubuntu2.3
6) sudo rm /etc/apt/sources.list.d/ubuntu-$(lsb_release -cs)-proposed.list
7) sudo apt update

>From there, join your domain like normal, and if you like, try out other
adcli or realm commands to ensure they work.

Let me know how the new adcli package in -proposed goes. In my testing,
it fixes the regression, and works as intended.

To Jason Alavaliant, thanks! I really appreciate the help testing.

Thanks,
Matthew

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to cyrus-sasl2 in Ubuntu.
https://bugs.launchpad.net/bugs/1906627

Title:
  GSS-SPNEGO implementation in cyrus-sasl2 is incompatible with Active
  Directory, causing recent adcli regression

Status in adcli package in Ubuntu:
  Fix Released
Status in cyrus-sasl2 package in Ubuntu:
  Fix Released
Status in adcli source package in Bionic:
  Fix Committed
Status in cyrus-sasl2 source package in Bionic:
  Fix Committed

Bug description:
  [Impact]

  A recent release of adcli 0.8.2-1ubuntu1 to bionic-updates caused a
  regression for some users when attempting to join a Active Directory
  realm. adcli introduced a default behaviour change, moving from GSS-
  API to GSS-SPNEGO as the default channel encryption algorithm.

  adcli uses the GSS-SPNEGO implementation from libsasl2-modules-gssapi-
  mit, a part of cyrus-sasl2. The implementation seems to have some
  compatibility issues with particular configurations of Active
  Directory on recent Windows Server systems.

  Particularly, adcli sends a ldap query to the domain controller, which
  responds with a tcp ack, but never returns a ldap response. The
  connection just hangs at this point and no more traffic is sent.

  You can see it on the packet trace below:

  https://paste.ubuntu.com/p/WRnnRMGBPm/

  On Focal, where the implementation of GSS-SPNEGO is working, we see a
  full exchange, and adcli works as expected:

  https://paste.ubuntu.com/p/8668pJrr2m/

  The fix is to not assume use of confidentiality and integrity modes,
  and instead use the flags negotiated by GSS-API during the initial
  handshake, as required by Microsoft's implementation.

  [Testcase]

  You will need to set up a Windows Server 2019 system, install and
  configure Active Directory and enable LDAP extensions and configure
  LDAPS and import the AD SSL certificate to the Ubuntu client. Create
  some users in Active Directory.

  On the Ubuntu client, set up /etc/hosts with the hostname of the
  Windows Server machine, if your system isn't configured for AD DNS.

  From there, install adcli 0.8.2-1 from -release.

  $ sudo apt install adcli

  Set up a packet trace with tcpdump:

  $ sudo tcpdump -i any port '(389 or 3268 or 636 or 3269)'

  Next, join the AD realm using the normal GSS-API:

  # adcli join --verbose -U Administrator --domain WIN-
  SB6JAS7PH22.testing.local --domain-controller WIN-
  SB6JAS7PH22.testing.local --domain-realm TESTING.LOCAL

  You will be prompted for Administrator's passowrd.

  The output should look like the below:

  https://paste.ubuntu.com/p/NWHGQn746D/

  Next, enable -proposed, and install adcli 0.8.2-1ubuntu1 which caused the 
regression.
  Repeat the above steps. Now you should see the connection hang.

  https://paste.ubuntu.com/p/WRnnRMGBPm/

  Finally, install the fixed cyrus-sasl2 package from -proposed

  https://launchpad.net/~mruffell/+archive/ubuntu/lp1906627-test

  $ sudo apt-get update
  $ sudo apt install libsasl2-2 libsasl2-modules libsasl2-modules-db 
libsasl2-modules-gssapi-mit

  Repeat the steps. GSS-SPNEGO should be working as intended, and you
  should get output like below:

  https://paste.ubuntu.com/p/W5cJNGvCsx/

  [Where problems could occur]

  Since we are changing the implementation of GSS-SPNEGO, and cyrus-
  sasl2 is the library which provides it, we can potentially break any
  package which depends on libsasl2-modules-gssapi-mit for GSS-SPNEGO.

  $ apt rdepends libsasl2-modules-gssapi-mit
  libsasl2-modules-gssapi-mit
  Reverse Depends:
   |Suggests: ldap-utils
    Depends: adcli
    Conflicts: libsasl2-modules-gssapi-heimdal
   

[Touch-packages] [Bug 1906627] Re: GSS-SPNEGO implementation in cyrus-sasl2 is incompatible with Active Directory, causing recent adcli regression

2020-12-15 Thread Matthew Ruffell
Performing verification for Bionic

Firstly, I installed adcli and libsasl2-modules-gssapi-mit from
-updates:

adcli 0.8.2-1
libsasl2-modules-gssapi-mit 2.1.27~101-g0780600+dfsg-3ubuntu2.1

>From there, I joined a Active Directory realm:

https://paste.ubuntu.com/p/zJhvpRzktk/
 
Next, I enabled -proposed and installed the fixed cyrus-sasl2 and adcli 
packages:

https://paste.ubuntu.com/p/cRrbkjjFmw/

We see that installing adcli 0.8.2-1ubuntu1.2 automatically pulls in the
fixed cyrus-sasl2 2.1.27~101-g0780600+dfsg-3ubuntu2.3 packages because
of the depends we set.

Next, I joined a Active Directory realm, using the same commands as
previous, i.e. not using the new --use-ldaps flag, but instead, falling
back to GSS-API and the new GSS-SPNEGO changes:

https://paste.ubuntu.com/p/WdKYxxDBQm/
 
The join succeeds, and does not get stuck. This shows that the implementation 
of GSS-SPNEGO is now compatible with Active Directory, and that the new adcli 
package is using the new implementation.

Looking at the packet trace, we see the full 30 or so packets exchanged,
which matches the expect count.

https://paste.ubuntu.com/p/k9njh3jYHh/

With these changes, the adcli and cyrus-sasl2 packages in -proposed can
join realms in the same ways that the initial packages in -updates can.

These changes fix the recent adcli regression. Happy to mark verified.

** Tags removed: regression-update verification-needed 
verification-needed-bionic
** Tags added: verification-done-bionic

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to cyrus-sasl2 in Ubuntu.
https://bugs.launchpad.net/bugs/1906627

Title:
  GSS-SPNEGO implementation in cyrus-sasl2 is incompatible with Active
  Directory, causing recent adcli regression

Status in adcli package in Ubuntu:
  Fix Released
Status in cyrus-sasl2 package in Ubuntu:
  Fix Released
Status in adcli source package in Bionic:
  Fix Committed
Status in cyrus-sasl2 source package in Bionic:
  Fix Committed

Bug description:
  [Impact]

  A recent release of adcli 0.8.2-1ubuntu1 to bionic-updates caused a
  regression for some users when attempting to join a Active Directory
  realm. adcli introduced a default behaviour change, moving from GSS-
  API to GSS-SPNEGO as the default channel encryption algorithm.

  adcli uses the GSS-SPNEGO implementation from libsasl2-modules-gssapi-
  mit, a part of cyrus-sasl2. The implementation seems to have some
  compatibility issues with particular configurations of Active
  Directory on recent Windows Server systems.

  Particularly, adcli sends a ldap query to the domain controller, which
  responds with a tcp ack, but never returns a ldap response. The
  connection just hangs at this point and no more traffic is sent.

  You can see it on the packet trace below:

  https://paste.ubuntu.com/p/WRnnRMGBPm/

  On Focal, where the implementation of GSS-SPNEGO is working, we see a
  full exchange, and adcli works as expected:

  https://paste.ubuntu.com/p/8668pJrr2m/

  The fix is to not assume use of confidentiality and integrity modes,
  and instead use the flags negotiated by GSS-API during the initial
  handshake, as required by Microsoft's implementation.

  [Testcase]

  You will need to set up a Windows Server 2019 system, install and
  configure Active Directory and enable LDAP extensions and configure
  LDAPS and import the AD SSL certificate to the Ubuntu client. Create
  some users in Active Directory.

  On the Ubuntu client, set up /etc/hosts with the hostname of the
  Windows Server machine, if your system isn't configured for AD DNS.

  From there, install adcli 0.8.2-1 from -release.

  $ sudo apt install adcli

  Set up a packet trace with tcpdump:

  $ sudo tcpdump -i any port '(389 or 3268 or 636 or 3269)'

  Next, join the AD realm using the normal GSS-API:

  # adcli join --verbose -U Administrator --domain WIN-
  SB6JAS7PH22.testing.local --domain-controller WIN-
  SB6JAS7PH22.testing.local --domain-realm TESTING.LOCAL

  You will be prompted for Administrator's passowrd.

  The output should look like the below:

  https://paste.ubuntu.com/p/NWHGQn746D/

  Next, enable -proposed, and install adcli 0.8.2-1ubuntu1 which caused the 
regression.
  Repeat the above steps. Now you should see the connection hang.

  https://paste.ubuntu.com/p/WRnnRMGBPm/

  Finally, install the fixed cyrus-sasl2 package from -proposed

  https://launchpad.net/~mruffell/+archive/ubuntu/lp1906627-test

  $ sudo apt-get update
  $ sudo apt install libsasl2-2 libsasl2-modules libsasl2-modules-db 
libsasl2-modules-gssapi-mit

  Repeat the steps. GSS-SPNEGO should be working as intended, and you
  should get output like below:

  https://paste.ubuntu.com/p/W5cJNGvCsx/

  [Where problems could occur]

  Since we are changing the implementation of GSS-SPNEGO, and cyrus-
  sasl2 is the library which provides it, we can potentially break any
  package which depends on libsasl2-modules-gssapi-mit for 

[Touch-packages] [Bug 1906627] Re: GSS-SPNEGO implementation in cyrus-sasl2 is incompatible with Active Directory, causing recent adcli regression

2020-12-07 Thread Matthew Ruffell
Attached is option two: a debdiff for adcli, which builds on
0.8.2-1ubuntu2, which re-introduces all of the --use-ldaps patches, and
also adds a depends to the fixed libsasl2-modules-gssapi-mit at greater
or equal to relationship. Use this if option 1 is a no go.

** Patch added: "debdiff for adcli on Bionic option two"
   
https://bugs.launchpad.net/ubuntu/+source/adcli/+bug/1906627/+attachment/5441873/+files/lp1906627_adcli_option_two.debdiff

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to cyrus-sasl2 in Ubuntu.
https://bugs.launchpad.net/bugs/1906627

Title:
  GSS-SPNEGO implementation in cyrus-sasl2 is incompatible with Active
  Directory, causing recent adcli regression

Status in adcli package in Ubuntu:
  Fix Released
Status in cyrus-sasl2 package in Ubuntu:
  Fix Released
Status in adcli source package in Bionic:
  In Progress
Status in cyrus-sasl2 source package in Bionic:
  In Progress

Bug description:
  [Impact]

  A recent release of adcli 0.8.2-1ubuntu1 to bionic-updates caused a
  regression for some users when attempting to join a Active Directory
  realm. adcli introduced a default behaviour change, moving from GSS-
  API to GSS-SPNEGO as the default channel encryption algorithm.

  adcli uses the GSS-SPNEGO implementation from libsasl2-modules-gssapi-
  mit, a part of cyrus-sasl2. The implementation seems to have some
  compatibility issues with particular configurations of Active
  Directory on recent Windows Server systems.

  Particularly, adcli sends a ldap query to the domain controller, which
  responds with a tcp ack, but never returns a ldap response. The
  connection just hangs at this point and no more traffic is sent.

  You can see it on the packet trace below:

  https://paste.ubuntu.com/p/WRnnRMGBPm/

  On Focal, where the implementation of GSS-SPNEGO is working, we see a
  full exchange, and adcli works as expected:

  https://paste.ubuntu.com/p/8668pJrr2m/

  The fix is to not assume use of confidentiality and integrity modes,
  and instead use the flags negotiated by GSS-API during the initial
  handshake, as required by Microsoft's implementation.

  [Testcase]

  You will need to set up a Windows Server 2019 system, install and
  configure Active Directory and enable LDAP extensions and configure
  LDAPS and import the AD SSL certificate to the Ubuntu client. Create
  some users in Active Directory.

  On the Ubuntu client, set up /etc/hosts with the hostname of the
  Windows Server machine, if your system isn't configured for AD DNS.

  From there, install adcli 0.8.2-1 from -release.

  $ sudo apt install adcli

  Set up a packet trace with tcpdump:

  $ sudo tcpdump -i any port '(389 or 3268 or 636 or 3269)'

  Next, join the AD realm using the normal GSS-API:

  # adcli join --verbose -U Administrator --domain WIN-
  SB6JAS7PH22.testing.local --domain-controller WIN-
  SB6JAS7PH22.testing.local --domain-realm TESTING.LOCAL

  You will be prompted for Administrator's passowrd.

  The output should look like the below:

  https://paste.ubuntu.com/p/NWHGQn746D/

  Next, enable -proposed, and install adcli 0.8.2-1ubuntu1 which caused the 
regression.
  Repeat the above steps. Now you should see the connection hang.

  https://paste.ubuntu.com/p/WRnnRMGBPm/

  Finally, install the fixed cyrus-sasl2 package, which is available from the
  below ppa:

  https://launchpad.net/~mruffell/+archive/ubuntu/lp1906627-test

  $ sudo add-apt-repository ppa:mruffell/lp1906627-test
  $ sudo apt-get update
  $ sudo apt install libsasl2-2 libsasl2-modules libsasl2-modules-db 
libsasl2-modules-gssapi-mit

  Repeat the steps. GSS-SPNEGO should be working as intended, and you
  should get output like below:

  https://paste.ubuntu.com/p/W5cJNGvCsx/

  [Where problems could occur]

  Since we are changing the implementation of GSS-SPNEGO, and cyrus-
  sasl2 is the library which provides it, we can potentially break any
  package which depends on libsasl2-modules-gssapi-mit for GSS-SPNEGO.

  $ apt rdepends libsasl2-modules-gssapi-mit
  libsasl2-modules-gssapi-mit
  Reverse Depends:
   |Suggests: ldap-utils
Depends: adcli
Conflicts: libsasl2-modules-gssapi-heimdal
   |Suggests: libsasl2-modules
Conflicts: libsasl2-modules-gssapi-heimdal
   |Recommends: sssd-krb5-common
   |Suggests: slapd
   |Suggests: libsasl2-modules
   |Suggests: ldap-utils
   |Depends: msktutil
Conflicts: libsasl2-modules-gssapi-heimdal
   |Depends: libapache2-mod-webauthldap
Depends: freeipa-server
Depends: freeipa-client
Depends: adcli
Depends: 389-ds-base
   |Recommends: sssd-krb5-common
   |Suggests: slapd
   |Suggests: libsasl2-modules
   
  While this SRU makes cyrus-sasl2 work with Microsoft implementations of 
GSS-SPNEGO, which will be the more common usecase, it may change the behaviour  
when connecting to a MIT krb5 server with the GSS-SPNEGO protocol, as krb5 
assumes use of confidentiality and integrity 

[Touch-packages] [Bug 1906627] Re: GSS-SPNEGO implementation in cyrus-sasl2 is incompatible with Active Directory, causing recent adcli regression

2020-12-07 Thread Matthew Ruffell
Attached is option one: a debdiff for adcli, which builds on
0.8.2-1ubuntu1 and simply adds a depends to the fixed libsasl2-modules-
gssapi-mit at greater or equal to relationship. This will require the
0.8.2-1ubuntu2 package in -unapproved queue to be deleted.

** Patch added: "debdiff for adcli on Bionic"
   
https://bugs.launchpad.net/ubuntu/+source/adcli/+bug/1906627/+attachment/5441872/+files/lp1906627_adcli_option_one.debdiff

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to cyrus-sasl2 in Ubuntu.
https://bugs.launchpad.net/bugs/1906627

Title:
  GSS-SPNEGO implementation in cyrus-sasl2 is incompatible with Active
  Directory, causing recent adcli regression

Status in adcli package in Ubuntu:
  Fix Released
Status in cyrus-sasl2 package in Ubuntu:
  Fix Released
Status in adcli source package in Bionic:
  In Progress
Status in cyrus-sasl2 source package in Bionic:
  In Progress

Bug description:
  [Impact]

  A recent release of adcli 0.8.2-1ubuntu1 to bionic-updates caused a
  regression for some users when attempting to join a Active Directory
  realm. adcli introduced a default behaviour change, moving from GSS-
  API to GSS-SPNEGO as the default channel encryption algorithm.

  adcli uses the GSS-SPNEGO implementation from libsasl2-modules-gssapi-
  mit, a part of cyrus-sasl2. The implementation seems to have some
  compatibility issues with particular configurations of Active
  Directory on recent Windows Server systems.

  Particularly, adcli sends a ldap query to the domain controller, which
  responds with a tcp ack, but never returns a ldap response. The
  connection just hangs at this point and no more traffic is sent.

  You can see it on the packet trace below:

  https://paste.ubuntu.com/p/WRnnRMGBPm/

  On Focal, where the implementation of GSS-SPNEGO is working, we see a
  full exchange, and adcli works as expected:

  https://paste.ubuntu.com/p/8668pJrr2m/

  The fix is to not assume use of confidentiality and integrity modes,
  and instead use the flags negotiated by GSS-API during the initial
  handshake, as required by Microsoft's implementation.

  [Testcase]

  You will need to set up a Windows Server 2019 system, install and
  configure Active Directory and enable LDAP extensions and configure
  LDAPS and import the AD SSL certificate to the Ubuntu client. Create
  some users in Active Directory.

  On the Ubuntu client, set up /etc/hosts with the hostname of the
  Windows Server machine, if your system isn't configured for AD DNS.

  From there, install adcli 0.8.2-1 from -release.

  $ sudo apt install adcli

  Set up a packet trace with tcpdump:

  $ sudo tcpdump -i any port '(389 or 3268 or 636 or 3269)'

  Next, join the AD realm using the normal GSS-API:

  # adcli join --verbose -U Administrator --domain WIN-
  SB6JAS7PH22.testing.local --domain-controller WIN-
  SB6JAS7PH22.testing.local --domain-realm TESTING.LOCAL

  You will be prompted for Administrator's passowrd.

  The output should look like the below:

  https://paste.ubuntu.com/p/NWHGQn746D/

  Next, enable -proposed, and install adcli 0.8.2-1ubuntu1 which caused the 
regression.
  Repeat the above steps. Now you should see the connection hang.

  https://paste.ubuntu.com/p/WRnnRMGBPm/

  Finally, install the fixed cyrus-sasl2 package, which is available from the
  below ppa:

  https://launchpad.net/~mruffell/+archive/ubuntu/lp1906627-test

  $ sudo add-apt-repository ppa:mruffell/lp1906627-test
  $ sudo apt-get update
  $ sudo apt install libsasl2-2 libsasl2-modules libsasl2-modules-db 
libsasl2-modules-gssapi-mit

  Repeat the steps. GSS-SPNEGO should be working as intended, and you
  should get output like below:

  https://paste.ubuntu.com/p/W5cJNGvCsx/

  [Where problems could occur]

  Since we are changing the implementation of GSS-SPNEGO, and cyrus-
  sasl2 is the library which provides it, we can potentially break any
  package which depends on libsasl2-modules-gssapi-mit for GSS-SPNEGO.

  $ apt rdepends libsasl2-modules-gssapi-mit
  libsasl2-modules-gssapi-mit
  Reverse Depends:
   |Suggests: ldap-utils
Depends: adcli
Conflicts: libsasl2-modules-gssapi-heimdal
   |Suggests: libsasl2-modules
Conflicts: libsasl2-modules-gssapi-heimdal
   |Recommends: sssd-krb5-common
   |Suggests: slapd
   |Suggests: libsasl2-modules
   |Suggests: ldap-utils
   |Depends: msktutil
Conflicts: libsasl2-modules-gssapi-heimdal
   |Depends: libapache2-mod-webauthldap
Depends: freeipa-server
Depends: freeipa-client
Depends: adcli
Depends: 389-ds-base
   |Recommends: sssd-krb5-common
   |Suggests: slapd
   |Suggests: libsasl2-modules
   
  While this SRU makes cyrus-sasl2 work with Microsoft implementations of 
GSS-SPNEGO, which will be the more common usecase, it may change the behaviour  
when connecting to a MIT krb5 server with the GSS-SPNEGO protocol, as krb5 
assumes use of confidentiality and integrity modes. This 

[Touch-packages] [Bug 1906627] Re: GSS-SPNEGO implementation in cyrus-sasl2 is incompatible with Active Directory, causing recent adcli regression

2020-12-06 Thread Matthew Ruffell
** Tags added: sts-sponsor

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to cyrus-sasl2 in Ubuntu.
https://bugs.launchpad.net/bugs/1906627

Title:
  GSS-SPNEGO implementation in cyrus-sasl2 is incompatible with Active
  Directory, causing recent adcli regression

Status in adcli package in Ubuntu:
  Fix Released
Status in cyrus-sasl2 package in Ubuntu:
  Fix Released
Status in adcli source package in Bionic:
  In Progress
Status in cyrus-sasl2 source package in Bionic:
  In Progress

Bug description:
  [Impact]

  A recent release of adcli 0.8.2-1ubuntu1 to bionic-updates caused a
  regression for some users when attempting to join a Active Directory
  realm. adcli introduced a default behaviour change, moving from GSS-
  API to GSS-SPNEGO as the default channel encryption algorithm.

  adcli uses the GSS-SPNEGO implementation from libsasl2-modules-gssapi-
  mit, a part of cyrus-sasl2. The implementation seems to have some
  compatibility issues with particular configurations of Active
  Directory on recent Windows Server systems.

  Particularly, adcli sends a ldap query to the domain controller, which
  responds with a tcp ack, but never returns a ldap response. The
  connection just hangs at this point and no more traffic is sent.

  You can see it on the packet trace below:

  https://paste.ubuntu.com/p/WRnnRMGBPm/

  On Focal, where the implementation of GSS-SPNEGO is working, we see a
  full exchange, and adcli works as expected:

  https://paste.ubuntu.com/p/8668pJrr2m/

  The fix is to not assume use of confidentiality and integrity modes,
  and instead use the flags negotiated by GSS-API during the initial
  handshake, as required by Microsoft's implementation.

  [Testcase]

  You will need to set up a Windows Server 2019 system, install and
  configure Active Directory and enable LDAP extensions and configure
  LDAPS and import the AD SSL certificate to the Ubuntu client. Create
  some users in Active Directory.

  On the Ubuntu client, set up /etc/hosts with the hostname of the
  Windows Server machine, if your system isn't configured for AD DNS.

  From there, install adcli 0.8.2-1 from -release.

  $ sudo apt install adcli

  Set up a packet trace with tcpdump:

  $ sudo tcpdump -i any port '(389 or 3268 or 636 or 3269)'

  Next, join the AD realm using the normal GSS-API:

  # adcli join --verbose -U Administrator --domain WIN-
  SB6JAS7PH22.testing.local --domain-controller WIN-
  SB6JAS7PH22.testing.local --domain-realm TESTING.LOCAL

  You will be prompted for Administrator's passowrd.

  The output should look like the below:

  https://paste.ubuntu.com/p/NWHGQn746D/

  Next, enable -proposed, and install adcli 0.8.2-1ubuntu1 which caused the 
regression.
  Repeat the above steps. Now you should see the connection hang.

  https://paste.ubuntu.com/p/WRnnRMGBPm/

  Finally, install the fixed cyrus-sasl2 package, which is available from the
  below ppa:

  https://launchpad.net/~mruffell/+archive/ubuntu/lp1906627-test

  $ sudo add-apt-repository ppa:mruffell/lp1906627-test
  $ sudo apt-get update
  $ sudo apt install libsasl2-2 libsasl2-modules libsasl2-modules-db 
libsasl2-modules-gssapi-mit

  Repeat the steps. GSS-SPNEGO should be working as intended, and you
  should get output like below:

  https://paste.ubuntu.com/p/W5cJNGvCsx/

  [Where problems could occur]

  Since we are changing the implementation of GSS-SPNEGO, and cyrus-
  sasl2 is the library which provides it, we can potentially break any
  package which depends on libsasl2-modules-gssapi-mit for GSS-SPNEGO.

  $ apt rdepends libsasl2-modules-gssapi-mit
  libsasl2-modules-gssapi-mit
  Reverse Depends:
   |Suggests: ldap-utils
Depends: adcli
Conflicts: libsasl2-modules-gssapi-heimdal
   |Suggests: libsasl2-modules
Conflicts: libsasl2-modules-gssapi-heimdal
   |Recommends: sssd-krb5-common
   |Suggests: slapd
   |Suggests: libsasl2-modules
   |Suggests: ldap-utils
   |Depends: msktutil
Conflicts: libsasl2-modules-gssapi-heimdal
   |Depends: libapache2-mod-webauthldap
Depends: freeipa-server
Depends: freeipa-client
Depends: adcli
Depends: 389-ds-base
   |Recommends: sssd-krb5-common
   |Suggests: slapd
   |Suggests: libsasl2-modules
   
  While this SRU makes cyrus-sasl2 work with Microsoft implementations of 
GSS-SPNEGO, which will be the more common usecase, it may change the behaviour  
when connecting to a MIT krb5 server with the GSS-SPNEGO protocol, as krb5 
assumes use of confidentiality and integrity modes. This shouldn't be a problem 
as the krb5 implementation signals its intentions by setting the correct flags 
during handshake, which these patches to cyrus-sasl2 should now parse correctly.

  [Other Info]

  The below two commits are needed. The first fixes the problem, the second 
fixes
  some unused parameter warnings.

  commit 816e529043de08f3f9dcc4097380de39478b0b16
  Author: Simo Sorce 
  Date:   Thu Feb 

[Touch-packages] [Bug 1906627] Re: GSS-SPNEGO implementation in cyrus-sasl2 is incompatible with Active Directory, causing recent adcli regression

2020-12-06 Thread Matthew Ruffell
Attached is a debdiff for cyrus-sasl2 on Bionic, which resolves the
incompatibilities of the GSS-SPNEGO implementation with the one in
Active Directory.

** Patch added: "cyrus-sasl2 debdiff for Bionic"
   
https://bugs.launchpad.net/ubuntu/+source/adcli/+bug/1906627/+attachment/5441530/+files/lp1906627_cyrus_sasl2_bionic.debdiff

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to cyrus-sasl2 in Ubuntu.
https://bugs.launchpad.net/bugs/1906627

Title:
  GSS-SPNEGO implementation in cyrus-sasl2 is incompatible with Active
  Directory, causing recent adcli regression

Status in adcli package in Ubuntu:
  Fix Released
Status in cyrus-sasl2 package in Ubuntu:
  Fix Released
Status in adcli source package in Bionic:
  In Progress
Status in cyrus-sasl2 source package in Bionic:
  In Progress

Bug description:
  [Impact]

  A recent release of adcli 0.8.2-1ubuntu1 to bionic-updates caused a
  regression for some users when attempting to join a Active Directory
  realm. adcli introduced a default behaviour change, moving from GSS-
  API to GSS-SPNEGO as the default channel encryption algorithm.

  adcli uses the GSS-SPNEGO implementation from libsasl2-modules-gssapi-
  mit, a part of cyrus-sasl2. The implementation seems to have some
  compatibility issues with particular configurations of Active
  Directory on recent Windows Server systems.

  Particularly, adcli sends a ldap query to the domain controller, which
  responds with a tcp ack, but never returns a ldap response. The
  connection just hangs at this point and no more traffic is sent.

  You can see it on the packet trace below:

  https://paste.ubuntu.com/p/WRnnRMGBPm/

  On Focal, where the implementation of GSS-SPNEGO is working, we see a
  full exchange, and adcli works as expected:

  https://paste.ubuntu.com/p/8668pJrr2m/

  The fix is to not assume use of confidentiality and integrity modes,
  and instead use the flags negotiated by GSS-API during the initial
  handshake, as required by Microsoft's implementation.

  [Testcase]

  You will need to set up a Windows Server 2019 system, install and
  configure Active Directory and enable LDAP extensions and configure
  LDAPS and import the AD SSL certificate to the Ubuntu client. Create
  some users in Active Directory.

  On the Ubuntu client, set up /etc/hosts with the hostname of the
  Windows Server machine, if your system isn't configured for AD DNS.

  From there, install adcli 0.8.2-1 from -release.

  $ sudo apt install adcli

  Set up a packet trace with tcpdump:

  $ sudo tcpdump -i any port '(389 or 3268 or 636 or 3269)'

  Next, join the AD realm using the normal GSS-API:

  # adcli join --verbose -U Administrator --domain WIN-
  SB6JAS7PH22.testing.local --domain-controller WIN-
  SB6JAS7PH22.testing.local --domain-realm TESTING.LOCAL

  You will be prompted for Administrator's passowrd.

  The output should look like the below:

  https://paste.ubuntu.com/p/NWHGQn746D/

  Next, enable -proposed, and install adcli 0.8.2-1ubuntu1 which caused the 
regression.
  Repeat the above steps. Now you should see the connection hang.

  https://paste.ubuntu.com/p/WRnnRMGBPm/

  Finally, install the fixed cyrus-sasl2 package, which is available from the
  below ppa:

  https://launchpad.net/~mruffell/+archive/ubuntu/lp1906627-test

  $ sudo add-apt-repository ppa:mruffell/lp1906627-test
  $ sudo apt-get update
  $ sudo apt install libsasl2-2 libsasl2-modules libsasl2-modules-db 
libsasl2-modules-gssapi-mit

  Repeat the steps. GSS-SPNEGO should be working as intended, and you
  should get output like below:

  https://paste.ubuntu.com/p/W5cJNGvCsx/

  [Where problems could occur]

  Since we are changing the implementation of GSS-SPNEGO, and cyrus-
  sasl2 is the library which provides it, we can potentially break any
  package which depends on libsasl2-modules-gssapi-mit for GSS-SPNEGO.

  $ apt rdepends libsasl2-modules-gssapi-mit
  libsasl2-modules-gssapi-mit
  Reverse Depends:
   |Suggests: ldap-utils
Depends: adcli
Conflicts: libsasl2-modules-gssapi-heimdal
   |Suggests: libsasl2-modules
Conflicts: libsasl2-modules-gssapi-heimdal
   |Recommends: sssd-krb5-common
   |Suggests: slapd
   |Suggests: libsasl2-modules
   |Suggests: ldap-utils
   |Depends: msktutil
Conflicts: libsasl2-modules-gssapi-heimdal
   |Depends: libapache2-mod-webauthldap
Depends: freeipa-server
Depends: freeipa-client
Depends: adcli
Depends: 389-ds-base
   |Recommends: sssd-krb5-common
   |Suggests: slapd
   |Suggests: libsasl2-modules
   
  While this SRU makes cyrus-sasl2 work with Microsoft implementations of 
GSS-SPNEGO, which will be the more common usecase, it may change the behaviour  
when connecting to a MIT krb5 server with the GSS-SPNEGO protocol, as krb5 
assumes use of confidentiality and integrity modes. This shouldn't be a problem 
as the krb5 implementation signals its intentions by setting the correct flags 

[Touch-packages] [Bug 1906627] Re: GSS-SPNEGO implementation in cyrus-sasl2 is incompatible with Active Directory, causing recent adcli regression

2020-12-06 Thread Matthew Ruffell
** Summary changed:

- adcli fails, can't contact LDAP server
+ GSS-SPNEGO implementation in cyrus-sasl2 is incompatible with Active 
Directory, causing recent adcli regression

** Description changed:

- Package: adcli
- Version: 0.8.2-1ubuntu1
- Release: Ubuntu 18.04 LTS
+ [Impact]
  
- When trying to join the domain with this new version of adcli, it gets
- to the point of 'Using GSS-SPNEGO for SASL bind' and then it will not do
- anything for 10 minutes. It will then fail, complaining it can't reach
- the LDAP server.
+ A recent release of adcli 0.8.2-1ubuntu1 to bionic-updates caused a
+ regression for some users when attempting to join a Active Directory
+ realm. adcli introduced a default behaviour change, moving from GSS-API
+ to GSS-SPNEGO as the default channel encryption algorithm.
  
- Logs:
- Dec 03 01:39:50 example001.domain.com realmd[6419]:  * Authenticated as user: 
domain-join-acco...@domain.com
- Dec 03 01:39:50 example001.domain.com adcli[6459]: GSSAPI client step 1
- Dec 03 01:39:50 example001.domain.com realmd[6419]:  * Authenticated as user: 
domain-join-acco...@domain.com
- Dec 03 01:39:50 example001.domain.com adcli[6459]: GSSAPI client step 1
- Dec 03 01:39:50 example001.domain.com realmd[6419]:  * Using GSS-SPNEGO for 
SASL bind
- Dec 03 01:39:50 example001.domain.com realmd[6419]:  * Using GSS-SPNEGO for 
SASL bind
- Dec 03 01:39:50 example001.domain.com adcli[6459]: GSSAPI client step 1
- Dec 03 01:55:27 example001.domain.com realmd[6419]:  ! Couldn't lookup domain 
short name: Can't contact LDAP server
- Dec 03 01:55:27 example001.domain.com realmd[6419]:  ! Couldn't lookup domain 
short name: Can't contact LDAP server
- Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using fully qualified 
name: example001.domain.com
- Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using fully qualified 
name: example001.domain.com
- Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using domain name: 
domain.com
- Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using domain name: 
domain.com
- Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using computer account 
name: EXAMPLE001
- Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using computer account 
name: EXAMPLE001
- Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using domain realm: 
domain.com
- Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using domain realm: 
domain.com
- Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Calculated computer 
account name from fqdn: EXAMPLE001
- Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Calculated computer 
account name from fqdn: EXAMPLE001
- Dec 03 01:55:27 example001.domain.com realmd[6419]:  * With user principal: 
host/example001.domain@domain.com
- Dec 03 01:55:27 example001.domain.com realmd[6419]:  * With user principal: 
host/example001.domain@domain.com
- Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Generated 120 
character computer password
- Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Generated 120 
character computer password
- Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using keytab: 
FILE:/etc/krb5.keytab
- Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using keytab: 
FILE:/etc/krb5.keytab
- Dec 03 01:55:27 example001.domain.com realmd[6419]:  ! Couldn't lookup 
computer account: EXAMPLE001$: Can't contact LDAP server
- Dec 03 01:55:27 example001.domain.com realmd[6419]:  ! Couldn't lookup 
computer account: EXAMPLE001$: Can't contact LDAP server
- Dec 03 01:55:27 example001.domain.com realmd[6419]: adcli: joining domain 
domain.com failed: Couldn't lookup computer account: EXAMPLE001$: Can't contact 
LDAP server
- Dec 03 01:55:27 example001.domain.com realmd[6419]: adcli: joining domain 
domain.com failed: Couldn't lookup computer account: EXAMPLE001$: Can't contact 
LDAP server
- Dec 03 01:55:27 example001.domain.com realmd[6419]: process exited: 6459
- Dec 03 01:55:27 example001.domain.com realmd[6419]:  ! Failed to join the 
domain
- Dec 03 01:55:27 example001.domain.com realmd[6419]:  ! Failed to join the 
domain
+ adcli uses the GSS-SPNEGO implementation from libsasl2-modules-gssapi-
+ mit, a part of cyrus-sasl2. The implementation seems to have some
+ compatibility issues with particular configurations of Active Directory
+ on recent Windows Server systems.
  
- On the network level, adcli gets to the point of send an ldap query to
- the domain controller and the domain controller returns an ack tcp
- packet, but then there is no more traffic between the domain controller
- and the server except for ntp packets until it fails.
+ Particularly, adcli sends a ldap query to the domain controller, which
+ responds with a tcp ack, but never returns a ldap response. The
+ connection just hangs at this point and no more traffic is sent.
  
- The domain controller traffic also shows that it is receiving the ldap
- query packet from the server but it never sends a 

[Touch-packages] [Bug 1906627] Re: adcli fails, can't contact LDAP server

2020-12-04 Thread Matthew Ruffell
** Changed in: cyrus-sasl2 (Ubuntu)
   Status: Confirmed => Fix Released

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to cyrus-sasl2 in Ubuntu.
https://bugs.launchpad.net/bugs/1906627

Title:
  adcli fails, can't contact LDAP server

Status in adcli package in Ubuntu:
  Fix Released
Status in cyrus-sasl2 package in Ubuntu:
  Fix Released
Status in adcli source package in Bionic:
  In Progress
Status in cyrus-sasl2 source package in Bionic:
  In Progress

Bug description:
  Package: adcli
  Version: 0.8.2-1ubuntu1
  Release: Ubuntu 18.04 LTS

  When trying to join the domain with this new version of adcli, it gets
  to the point of 'Using GSS-SPNEGO for SASL bind' and then it will not
  do anything for 10 minutes. It will then fail, complaining it can't
  reach the LDAP server.

  Logs:
  Dec 03 01:39:50 example001.domain.com realmd[6419]:  * Authenticated as user: 
domain-join-acco...@domain.com
  Dec 03 01:39:50 example001.domain.com adcli[6459]: GSSAPI client step 1
  Dec 03 01:39:50 example001.domain.com realmd[6419]:  * Authenticated as user: 
domain-join-acco...@domain.com
  Dec 03 01:39:50 example001.domain.com adcli[6459]: GSSAPI client step 1
  Dec 03 01:39:50 example001.domain.com realmd[6419]:  * Using GSS-SPNEGO for 
SASL bind
  Dec 03 01:39:50 example001.domain.com realmd[6419]:  * Using GSS-SPNEGO for 
SASL bind
  Dec 03 01:39:50 example001.domain.com adcli[6459]: GSSAPI client step 1
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  ! Couldn't lookup domain 
short name: Can't contact LDAP server
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  ! Couldn't lookup domain 
short name: Can't contact LDAP server
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using fully qualified 
name: example001.domain.com
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using fully qualified 
name: example001.domain.com
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using domain name: 
domain.com
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using domain name: 
domain.com
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using computer account 
name: EXAMPLE001
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using computer account 
name: EXAMPLE001
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using domain realm: 
domain.com
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using domain realm: 
domain.com
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Calculated computer 
account name from fqdn: EXAMPLE001
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Calculated computer 
account name from fqdn: EXAMPLE001
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * With user principal: 
host/example001.domain@domain.com
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * With user principal: 
host/example001.domain@domain.com
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Generated 120 
character computer password
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Generated 120 
character computer password
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using keytab: 
FILE:/etc/krb5.keytab
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using keytab: 
FILE:/etc/krb5.keytab
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  ! Couldn't lookup 
computer account: EXAMPLE001$: Can't contact LDAP server
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  ! Couldn't lookup 
computer account: EXAMPLE001$: Can't contact LDAP server
  Dec 03 01:55:27 example001.domain.com realmd[6419]: adcli: joining domain 
domain.com failed: Couldn't lookup computer account: EXAMPLE001$: Can't contact 
LDAP server
  Dec 03 01:55:27 example001.domain.com realmd[6419]: adcli: joining domain 
domain.com failed: Couldn't lookup computer account: EXAMPLE001$: Can't contact 
LDAP server
  Dec 03 01:55:27 example001.domain.com realmd[6419]: process exited: 6459
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  ! Failed to join the 
domain
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  ! Failed to join the 
domain

  On the network level, adcli gets to the point of send an ldap query to
  the domain controller and the domain controller returns an ack tcp
  packet, but then there is no more traffic between the domain
  controller and the server except for ntp packets until it fails.

  The domain controller traffic also shows that it is receiving the ldap
  query packet from the server but it never sends a reply and there is
  no log in directory services regarding the query. We also couldn't
  find anything in procmon regarding this query either.

  Workaround/Fix:
  Downgrading the adcli package back to version 0.8.2-1 fixes the issues and 
domain join works properly again.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/adcli/+bug/1906627/+subscriptions

-- 

[Touch-packages] [Bug 1906627] Re: adcli fails, can't contact LDAP server

2020-12-04 Thread Matthew Ruffell
** Changed in: cyrus-sasl2 (Ubuntu Bionic)
   Status: Confirmed => In Progress

** Changed in: cyrus-sasl2 (Ubuntu Bionic)
   Importance: Undecided => Medium

** Changed in: cyrus-sasl2 (Ubuntu Bionic)
 Assignee: (unassigned) => Matthew Ruffell (mruffell)

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to cyrus-sasl2 in Ubuntu.
https://bugs.launchpad.net/bugs/1906627

Title:
  adcli fails, can't contact LDAP server

Status in adcli package in Ubuntu:
  Fix Released
Status in cyrus-sasl2 package in Ubuntu:
  Confirmed
Status in adcli source package in Bionic:
  In Progress
Status in cyrus-sasl2 source package in Bionic:
  In Progress

Bug description:
  Package: adcli
  Version: 0.8.2-1ubuntu1
  Release: Ubuntu 18.04 LTS

  When trying to join the domain with this new version of adcli, it gets
  to the point of 'Using GSS-SPNEGO for SASL bind' and then it will not
  do anything for 10 minutes. It will then fail, complaining it can't
  reach the LDAP server.

  Logs:
  Dec 03 01:39:50 example001.domain.com realmd[6419]:  * Authenticated as user: 
domain-join-acco...@domain.com
  Dec 03 01:39:50 example001.domain.com adcli[6459]: GSSAPI client step 1
  Dec 03 01:39:50 example001.domain.com realmd[6419]:  * Authenticated as user: 
domain-join-acco...@domain.com
  Dec 03 01:39:50 example001.domain.com adcli[6459]: GSSAPI client step 1
  Dec 03 01:39:50 example001.domain.com realmd[6419]:  * Using GSS-SPNEGO for 
SASL bind
  Dec 03 01:39:50 example001.domain.com realmd[6419]:  * Using GSS-SPNEGO for 
SASL bind
  Dec 03 01:39:50 example001.domain.com adcli[6459]: GSSAPI client step 1
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  ! Couldn't lookup domain 
short name: Can't contact LDAP server
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  ! Couldn't lookup domain 
short name: Can't contact LDAP server
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using fully qualified 
name: example001.domain.com
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using fully qualified 
name: example001.domain.com
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using domain name: 
domain.com
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using domain name: 
domain.com
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using computer account 
name: EXAMPLE001
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using computer account 
name: EXAMPLE001
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using domain realm: 
domain.com
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using domain realm: 
domain.com
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Calculated computer 
account name from fqdn: EXAMPLE001
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Calculated computer 
account name from fqdn: EXAMPLE001
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * With user principal: 
host/example001.domain@domain.com
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * With user principal: 
host/example001.domain@domain.com
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Generated 120 
character computer password
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Generated 120 
character computer password
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using keytab: 
FILE:/etc/krb5.keytab
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using keytab: 
FILE:/etc/krb5.keytab
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  ! Couldn't lookup 
computer account: EXAMPLE001$: Can't contact LDAP server
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  ! Couldn't lookup 
computer account: EXAMPLE001$: Can't contact LDAP server
  Dec 03 01:55:27 example001.domain.com realmd[6419]: adcli: joining domain 
domain.com failed: Couldn't lookup computer account: EXAMPLE001$: Can't contact 
LDAP server
  Dec 03 01:55:27 example001.domain.com realmd[6419]: adcli: joining domain 
domain.com failed: Couldn't lookup computer account: EXAMPLE001$: Can't contact 
LDAP server
  Dec 03 01:55:27 example001.domain.com realmd[6419]: process exited: 6459
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  ! Failed to join the 
domain
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  ! Failed to join the 
domain

  On the network level, adcli gets to the point of send an ldap query to
  the domain controller and the domain controller returns an ack tcp
  packet, but then there is no more traffic between the domain
  controller and the server except for ntp packets until it fails.

  The domain controller traffic also shows that it is receiving the ldap
  query packet from the server but it never sends a reply and there is
  no log in directory services regarding the query. We also couldn't
  find anything in procmon regarding this query either.

  Workaround/Fix:
  Downgrading the adcli package back to versio

[Touch-packages] [Bug 1906627] Re: adcli fails, can't contact LDAP server

2020-12-04 Thread Matthew Ruffell
Attached is a debdiff to revert the changes we made to adcli to restore
functionality to GSS-API.

** Patch added: "Debdiff for adcli on Bionic"
   
https://bugs.launchpad.net/ubuntu/+source/adcli/+bug/1906627/+attachment/5441133/+files/lp1906627_adcli_bionic.debdiff

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to cyrus-sasl2 in Ubuntu.
https://bugs.launchpad.net/bugs/1906627

Title:
  adcli fails, can't contact LDAP server

Status in adcli package in Ubuntu:
  Fix Released
Status in cyrus-sasl2 package in Ubuntu:
  New
Status in adcli source package in Bionic:
  In Progress
Status in cyrus-sasl2 source package in Bionic:
  New

Bug description:
  Package: adcli
  Version: 0.8.2-1ubuntu1
  Release: Ubuntu 18.04 LTS

  When trying to join the domain with this new version of adcli, it gets
  to the point of 'Using GSS-SPNEGO for SASL bind' and then it will not
  do anything for 10 minutes. It will then fail, complaining it can't
  reach the LDAP server.

  Logs:
  Dec 03 01:39:50 example001.domain.com realmd[6419]:  * Authenticated as user: 
domain-join-acco...@domain.com
  Dec 03 01:39:50 example001.domain.com adcli[6459]: GSSAPI client step 1
  Dec 03 01:39:50 example001.domain.com realmd[6419]:  * Authenticated as user: 
domain-join-acco...@domain.com
  Dec 03 01:39:50 example001.domain.com adcli[6459]: GSSAPI client step 1
  Dec 03 01:39:50 example001.domain.com realmd[6419]:  * Using GSS-SPNEGO for 
SASL bind
  Dec 03 01:39:50 example001.domain.com realmd[6419]:  * Using GSS-SPNEGO for 
SASL bind
  Dec 03 01:39:50 example001.domain.com adcli[6459]: GSSAPI client step 1
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  ! Couldn't lookup domain 
short name: Can't contact LDAP server
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  ! Couldn't lookup domain 
short name: Can't contact LDAP server
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using fully qualified 
name: example001.domain.com
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using fully qualified 
name: example001.domain.com
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using domain name: 
domain.com
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using domain name: 
domain.com
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using computer account 
name: EXAMPLE001
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using computer account 
name: EXAMPLE001
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using domain realm: 
domain.com
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using domain realm: 
domain.com
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Calculated computer 
account name from fqdn: EXAMPLE001
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Calculated computer 
account name from fqdn: EXAMPLE001
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * With user principal: 
host/example001.domain@domain.com
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * With user principal: 
host/example001.domain@domain.com
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Generated 120 
character computer password
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Generated 120 
character computer password
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using keytab: 
FILE:/etc/krb5.keytab
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using keytab: 
FILE:/etc/krb5.keytab
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  ! Couldn't lookup 
computer account: EXAMPLE001$: Can't contact LDAP server
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  ! Couldn't lookup 
computer account: EXAMPLE001$: Can't contact LDAP server
  Dec 03 01:55:27 example001.domain.com realmd[6419]: adcli: joining domain 
domain.com failed: Couldn't lookup computer account: EXAMPLE001$: Can't contact 
LDAP server
  Dec 03 01:55:27 example001.domain.com realmd[6419]: adcli: joining domain 
domain.com failed: Couldn't lookup computer account: EXAMPLE001$: Can't contact 
LDAP server
  Dec 03 01:55:27 example001.domain.com realmd[6419]: process exited: 6459
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  ! Failed to join the 
domain
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  ! Failed to join the 
domain

  On the network level, adcli gets to the point of send an ldap query to
  the domain controller and the domain controller returns an ack tcp
  packet, but then there is no more traffic between the domain
  controller and the server except for ntp packets until it fails.

  The domain controller traffic also shows that it is receiving the ldap
  query packet from the server but it never sends a reply and there is
  no log in directory services regarding the query. We also couldn't
  find anything in procmon regarding this query either.

  Workaround/Fix:
  Downgrading the adcli package back to version 0.8.2-1 fixes the 

[Touch-packages] [Bug 1906627] Re: adcli fails, can't contact LDAP server

2020-12-03 Thread Matthew Ruffell
Yes, when --use-ldaps is specified, adcli will make a TLS connection to
the domain controller, and speak LDAPS. This works, and is the reason
why this bug slipped through our regression testing. I should have
tested without the --use-ldaps flag as well.

Regardless, this bug seems to be caused by the GSS-SPNEGO implementation
in the cyrus-sasl2 package being broken. adcli links to libsasl2
-modules-gssapi-mit, which is a part of cyrus-sasl2, since adcli does
not implement GSS-SPNEGO itself, and relies on cyrus-sasl libraries.

I downloaded the source package of cyrus-sasl2 2.1.27+dfsg-2 from Focal,
and I built it on Bionic, and installed it. I then tried a adcli join,
and it worked:

https://paste.ubuntu.com/p/R8PyHJMNtT/

Looking at the cyrus-sasl2 source repo, it seems the Bionic version is
missing a lot of commits related to GSS-SPNEGO support.

Commit 816e529043de08f3f9dcc4097380de39478b0b16
From: Simo Sorce 
Date: Thu, 16 Feb 2017 15:25:56 -0500
Subject: Fix GSS-SPNEGO mechanism's incompatible behavior
Link: 
https://github.com/cyrusimap/cyrus-sasl/commit/816e529043de08f3f9dcc4097380de39478b0b16

Commit 4b0306dcd76031460246b2dabcb7db766d6b04d8
From: Simo Sorce 
Date: Mon, 10 Apr 2017 19:54:19 -0400
Subject: Add support for retrieving the mech_ssf
Link: 
https://github.com/cyrusimap/cyrus-sasl/commit/4b0306dcd76031460246b2dabcb7db766d6b04d8

Commit 31b68a9438c24fc9e3e52f626462bf514de31757
From: Ryan Tandy 
Date: Mon, 24 Dec 2018 15:07:02 -0800
Subject: Restore LIBS after checking gss_inquire_sec_context_by_oid
Link: 
https://github.com/cyrusimap/cyrus-sasl/commit/31b68a9438c24fc9e3e52f626462bf514de31757

This doesn't even seem to be a complete list either, and if we backport
these patches to the Bionic cyrus-sasl2 package, it fails to build for
numerous reasons.

I also found a similar bug report in Debian, which features the above third 
commit: 
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=917129

>From what I can tell, GSS-SPNEGO in cyrus-sasl2 for Bionic has never
worked, and changing it to the default was a bad idea.

So, we have a decision to make. If supporting the new Active Directory
requirements in ADV190023 [1][2] which adds --use-ldaps for adcli, as a
part of bug 1868703 is important, and something the community wants, we
need to fix up cyrus-sasl2 to have a working GSS-SPNEGO implementation.

[1] https://msrc.microsoft.com/update-guide/en-us/vulnerability/ADV190023
[2] 
https://support.microsoft.com/en-us/help/4520412/2020-ldap-channel-binding-and-ldap-signing-requirements-for-windows

If we don't want --use-ldaps for adcli, then we can revert the patches
for adcli on Bionic, and go back to what was working previously, with
GSS-API.

** Bug watch added: Debian Bug tracker #917129
   https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=917129

** Also affects: cyrus-sasl2 (Ubuntu)
   Importance: Undecided
   Status: New

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to cyrus-sasl2 in Ubuntu.
https://bugs.launchpad.net/bugs/1906627

Title:
  adcli fails, can't contact LDAP server

Status in adcli package in Ubuntu:
  Fix Released
Status in cyrus-sasl2 package in Ubuntu:
  New
Status in adcli source package in Bionic:
  In Progress
Status in cyrus-sasl2 source package in Bionic:
  New

Bug description:
  Package: adcli
  Version: 0.8.2-1ubuntu1
  Release: Ubuntu 18.04 LTS

  When trying to join the domain with this new version of adcli, it gets
  to the point of 'Using GSS-SPNEGO for SASL bind' and then it will not
  do anything for 10 minutes. It will then fail, complaining it can't
  reach the LDAP server.

  Logs:
  Dec 03 01:39:50 example001.domain.com realmd[6419]:  * Authenticated as user: 
domain-join-acco...@domain.com
  Dec 03 01:39:50 example001.domain.com adcli[6459]: GSSAPI client step 1
  Dec 03 01:39:50 example001.domain.com realmd[6419]:  * Authenticated as user: 
domain-join-acco...@domain.com
  Dec 03 01:39:50 example001.domain.com adcli[6459]: GSSAPI client step 1
  Dec 03 01:39:50 example001.domain.com realmd[6419]:  * Using GSS-SPNEGO for 
SASL bind
  Dec 03 01:39:50 example001.domain.com realmd[6419]:  * Using GSS-SPNEGO for 
SASL bind
  Dec 03 01:39:50 example001.domain.com adcli[6459]: GSSAPI client step 1
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  ! Couldn't lookup domain 
short name: Can't contact LDAP server
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  ! Couldn't lookup domain 
short name: Can't contact LDAP server
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using fully qualified 
name: example001.domain.com
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using fully qualified 
name: example001.domain.com
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using domain name: 
domain.com
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using domain name: 
domain.com
  Dec 03 01:55:27 example001.domain.com realmd[6419]:  * Using 

[Touch-packages] [Bug 1886112] Re: Enabling DMESG_RESTRICT in Groovy Onward

2020-08-30 Thread Matthew Ruffell
As per my most recent email to ubuntu-devel, I am marking the changes to
util-linux as Won't Fix.

Relevant mailing list discussion (for future reference):

Ansgar responded on debian-devel mentioning that adding cap_syslog to
dmesg enables the user to clear the kernel log buffer:

https://lists.debian.org/debian-devel/2020/08/msg00121.html>>

> That grants additional rights to the `adm` group that it did not have
> before, for example to clear the dmesg buffer:
>
> $ dmesg --clear
>
> works after adding `cap_syslog` to the dmesg binary whereas it did not
> work before.

Chris Hofstaedtler, the maintainer of util-linux, mentions that granting
such powers to members of adm is more or less unacceptable:

https://lists.ubuntu.com/archives/ubuntu-devel/2020-August/041151.html

> Re-enabling dmesg for the %adm group does not seem to add value for
> Debian now, and granting the --clear (and other) permissions seems
> to be too much.

This was further acked by Steve Langasek:

https://lists.ubuntu.com/archives/ubuntu-devel/2020-August/041152.html

> I agree, and on that basis I also do not believe we should include this
> change to util-linux in Ubuntu.

Because of this, I will no longer pursue opening dmesg up to users in
the adm group, or at least until cap_syslog gets a read-only sister
capability.

Hopefully Ubuntu users won't be too inconvenienced by having to run
dmesg as superuser.

Users can always turn off the behaviour, by setting
"kernel.dmesg_restrict = 0" in /etc/sysctl.d/10-kernel-hardening.conf

** Changed in: util-linux (Ubuntu Groovy)
   Status: In Progress => Won't Fix

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to util-linux in Ubuntu.
https://bugs.launchpad.net/bugs/1886112

Title:
  Enabling DMESG_RESTRICT in Groovy Onward

Status in linux package in Ubuntu:
  Fix Released
Status in procps package in Ubuntu:
  In Progress
Status in util-linux package in Ubuntu:
  In Progress
Status in linux source package in Groovy:
  Fix Released
Status in procps source package in Groovy:
  In Progress
Status in util-linux source package in Groovy:
  Won't Fix

Bug description:
  [Impact]

  This bug implements the enablement of CONFIG_SECURITY_DMESG_RESTRICT
  feature by default for Groovy onward, proposed to ubuntu-devel:

  https://lists.ubuntu.com/archives/ubuntu-devel/2020-June/041063.html

  The kernel log buffer contains a wealth of sensitive information, such
  as detailed call traces and kernel addresses found in register dumps
  in kernel oops messages.

  Exploit developers and attackers can leverage these information leaks
  to get past KASLR, and they can use the kernel log buffer to get
  instant feedback on their privilege escalation attacks, as failures
  will be shown as further oops messages, which attackers can use to fix
  and tune their programs until they work.

  Currently, if I create a new, unprivileged user on a Focal system,
  they cannot access /var/log/kern.log, /var/log/syslog or see system
  events in journalctl. But yet, they are given free reign to the kernel
  log buffer.

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ cat /var/log/kern.log
  cat: /var/log/kern.log: Permission denied
  $ cat /var/log/syslog
  cat: /var/log/syslog: Permission denied
  $ journalctl
  Hint: You are currently not seeing messages from other users and the system.
    Users in groups 'adm', 'systemd-journal' can see all messages.
    Pass -q to turn off this notice.
  Jun 16 23:44:59 ubuntu systemd[2328]: Reached target Main User Target.
  Jun 16 23:44:59 ubuntu systemd[2328]: Startup finished in 69ms.
  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
  ...

  I propose that we restrict access to dmesg to users in group 'adm'
  like so:

  1) Add kernel.dmesg_restrict = 1 to
 /etc/sysctl.d/10-kernel-hardening.conf
  2) Following changes to /bin/dmesg permissions in package 'util-linux'
  - Ownership changes to root:adm
  - Permissions changed to 0750 (-rwxr-x---)
  - Add cap_syslog capability to binary.

  For most users, they will use the initial admin account, which is in
  the 'adm' group already, and will see no impact to these changes. If a
  log scraper type program needs access to dmesg, the user the daemon
  runs as can simply be added to the 'adm' group.

  [Testcase]

  Currently, all users can run /usr/bin/dmesg to view the kernel log
  buffer:

  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: 

[Touch-packages] [Bug 1886112] Re: Enabling DMESG_RESTRICT in Groovy Onward

2020-08-30 Thread Matthew Ruffell
As 5.8.0-16-generic has now been released to the -release pocket,
CONFIG_SECURITY_DMESG_RESTRICT is now enabled in Groovy. Marking the
changes to the kernel as Fix Released.

** Changed in: linux (Ubuntu Groovy)
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to util-linux in Ubuntu.
https://bugs.launchpad.net/bugs/1886112

Title:
  Enabling DMESG_RESTRICT in Groovy Onward

Status in linux package in Ubuntu:
  Fix Released
Status in procps package in Ubuntu:
  In Progress
Status in util-linux package in Ubuntu:
  In Progress
Status in linux source package in Groovy:
  Fix Released
Status in procps source package in Groovy:
  In Progress
Status in util-linux source package in Groovy:
  Won't Fix

Bug description:
  [Impact]

  This bug implements the enablement of CONFIG_SECURITY_DMESG_RESTRICT
  feature by default for Groovy onward, proposed to ubuntu-devel:

  https://lists.ubuntu.com/archives/ubuntu-devel/2020-June/041063.html

  The kernel log buffer contains a wealth of sensitive information, such
  as detailed call traces and kernel addresses found in register dumps
  in kernel oops messages.

  Exploit developers and attackers can leverage these information leaks
  to get past KASLR, and they can use the kernel log buffer to get
  instant feedback on their privilege escalation attacks, as failures
  will be shown as further oops messages, which attackers can use to fix
  and tune their programs until they work.

  Currently, if I create a new, unprivileged user on a Focal system,
  they cannot access /var/log/kern.log, /var/log/syslog or see system
  events in journalctl. But yet, they are given free reign to the kernel
  log buffer.

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ cat /var/log/kern.log
  cat: /var/log/kern.log: Permission denied
  $ cat /var/log/syslog
  cat: /var/log/syslog: Permission denied
  $ journalctl
  Hint: You are currently not seeing messages from other users and the system.
    Users in groups 'adm', 'systemd-journal' can see all messages.
    Pass -q to turn off this notice.
  Jun 16 23:44:59 ubuntu systemd[2328]: Reached target Main User Target.
  Jun 16 23:44:59 ubuntu systemd[2328]: Startup finished in 69ms.
  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
  ...

  I propose that we restrict access to dmesg to users in group 'adm'
  like so:

  1) Add kernel.dmesg_restrict = 1 to
 /etc/sysctl.d/10-kernel-hardening.conf
  2) Following changes to /bin/dmesg permissions in package 'util-linux'
  - Ownership changes to root:adm
  - Permissions changed to 0750 (-rwxr-x---)
  - Add cap_syslog capability to binary.

  For most users, they will use the initial admin account, which is in
  the 'adm' group already, and will see no impact to these changes. If a
  log scraper type program needs access to dmesg, the user the daemon
  runs as can simply be added to the 'adm' group.

  [Testcase]

  Currently, all users can run /usr/bin/dmesg to view the kernel log
  buffer:

  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
  ...

  When the changes are applied, the default admin user will be able to
  view dmesg (since they are in group 'adm'), while new unprivileged
  users will not.

  Test packages are available in the following ppa:
  https://launchpad.net/~mruffell/+archive/ubuntu/lp1886112-test

  $ whoami
  ubuntu
  $ groups
  ubuntu adm cdrom sudo dip plugdev
  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
  ...

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ dmesg
  -bash: /usr/bin/dmesg: Permission denied

  [Regression Potential]

  Some users or log scraper type programs may need to view the kernel
  log buffer, or have access to dmesg. In this case, the underlying
  service user would need to be added to the 'adm' group.

  Users have the ability to disable DMESG_RESTRICT by changing
  kernel.dmesg_restrict sysctl in /etc/sysctl.d/10-kernel-hardening.conf
  from '1' to '0', followed by a reboot.

To manage notifications about this bug go to:

[Touch-packages] [Bug 1886112] Re: Enabling DMESG_RESTRICT in Groovy Onward

2020-08-16 Thread Matthew Ruffell
Wrote to debian-devel to see if upstream is interested in carrying the
debian postinstall changes for util-linux: https://lists.debian.org
/debian-devel/2020/08/msg00107.html

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to util-linux in Ubuntu.
https://bugs.launchpad.net/bugs/1886112

Title:
  Enabling DMESG_RESTRICT in Groovy Onward

Status in linux package in Ubuntu:
  Fix Committed
Status in procps package in Ubuntu:
  In Progress
Status in util-linux package in Ubuntu:
  In Progress
Status in linux source package in Groovy:
  Fix Committed
Status in procps source package in Groovy:
  In Progress
Status in util-linux source package in Groovy:
  In Progress

Bug description:
  [Impact]

  This bug implements the enablement of CONFIG_SECURITY_DMESG_RESTRICT
  feature by default for Groovy onward, proposed to ubuntu-devel:

  https://lists.ubuntu.com/archives/ubuntu-devel/2020-June/041063.html

  The kernel log buffer contains a wealth of sensitive information, such
  as detailed call traces and kernel addresses found in register dumps
  in kernel oops messages.

  Exploit developers and attackers can leverage these information leaks
  to get past KASLR, and they can use the kernel log buffer to get
  instant feedback on their privilege escalation attacks, as failures
  will be shown as further oops messages, which attackers can use to fix
  and tune their programs until they work.

  Currently, if I create a new, unprivileged user on a Focal system,
  they cannot access /var/log/kern.log, /var/log/syslog or see system
  events in journalctl. But yet, they are given free reign to the kernel
  log buffer.

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ cat /var/log/kern.log
  cat: /var/log/kern.log: Permission denied
  $ cat /var/log/syslog
  cat: /var/log/syslog: Permission denied
  $ journalctl
  Hint: You are currently not seeing messages from other users and the system.
    Users in groups 'adm', 'systemd-journal' can see all messages.
    Pass -q to turn off this notice.
  Jun 16 23:44:59 ubuntu systemd[2328]: Reached target Main User Target.
  Jun 16 23:44:59 ubuntu systemd[2328]: Startup finished in 69ms.
  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
  ...

  I propose that we restrict access to dmesg to users in group 'adm'
  like so:

  1) Add kernel.dmesg_restrict = 1 to
 /etc/sysctl.d/10-kernel-hardening.conf
  2) Following changes to /bin/dmesg permissions in package 'util-linux'
  - Ownership changes to root:adm
  - Permissions changed to 0750 (-rwxr-x---)
  - Add cap_syslog capability to binary.

  For most users, they will use the initial admin account, which is in
  the 'adm' group already, and will see no impact to these changes. If a
  log scraper type program needs access to dmesg, the user the daemon
  runs as can simply be added to the 'adm' group.

  [Testcase]

  Currently, all users can run /usr/bin/dmesg to view the kernel log
  buffer:

  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
  ...

  When the changes are applied, the default admin user will be able to
  view dmesg (since they are in group 'adm'), while new unprivileged
  users will not.

  Test packages are available in the following ppa:
  https://launchpad.net/~mruffell/+archive/ubuntu/lp1886112-test

  $ whoami
  ubuntu
  $ groups
  ubuntu adm cdrom sudo dip plugdev
  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
  ...

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ dmesg
  -bash: /usr/bin/dmesg: Permission denied

  [Regression Potential]

  Some users or log scraper type programs may need to view the kernel
  log buffer, or have access to dmesg. In this case, the underlying
  service user would need to be added to the 'adm' group.

  Users have the ability to disable DMESG_RESTRICT by changing
  kernel.dmesg_restrict sysctl in /etc/sysctl.d/10-kernel-hardening.conf
  from '1' to '0', followed by a reboot.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1886112/+subscriptions

-- 
Mailing 

[Touch-packages] [Bug 1886112] Re: Enabling DMESG_RESTRICT in Groovy Onward

2020-08-10 Thread Matthew Ruffell
Attached is a rebased debdiff for util-linux, which implements the
permission changes to the dmesg binary.

** Patch removed: "util-linux debdiff for Groovy"
   
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1886112/+attachment/5395389/+files/lp1886112_util-linux_groovy.debdiff

** Patch added: "util-linux debdiff for Groovy"
   
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1886112/+attachment/5400541/+files/lp1886112_util-linux_groovy.debdiff

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to util-linux in Ubuntu.
https://bugs.launchpad.net/bugs/1886112

Title:
  Enabling DMESG_RESTRICT in Groovy Onward

Status in linux package in Ubuntu:
  Fix Committed
Status in procps package in Ubuntu:
  In Progress
Status in util-linux package in Ubuntu:
  In Progress
Status in linux source package in Groovy:
  Fix Committed
Status in procps source package in Groovy:
  In Progress
Status in util-linux source package in Groovy:
  In Progress

Bug description:
  [Impact]

  This bug implements the enablement of CONFIG_SECURITY_DMESG_RESTRICT
  feature by default for Groovy onward, proposed to ubuntu-devel:

  https://lists.ubuntu.com/archives/ubuntu-devel/2020-June/041063.html

  The kernel log buffer contains a wealth of sensitive information, such
  as detailed call traces and kernel addresses found in register dumps
  in kernel oops messages.

  Exploit developers and attackers can leverage these information leaks
  to get past KASLR, and they can use the kernel log buffer to get
  instant feedback on their privilege escalation attacks, as failures
  will be shown as further oops messages, which attackers can use to fix
  and tune their programs until they work.

  Currently, if I create a new, unprivileged user on a Focal system,
  they cannot access /var/log/kern.log, /var/log/syslog or see system
  events in journalctl. But yet, they are given free reign to the kernel
  log buffer.

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ cat /var/log/kern.log
  cat: /var/log/kern.log: Permission denied
  $ cat /var/log/syslog
  cat: /var/log/syslog: Permission denied
  $ journalctl
  Hint: You are currently not seeing messages from other users and the system.
    Users in groups 'adm', 'systemd-journal' can see all messages.
    Pass -q to turn off this notice.
  Jun 16 23:44:59 ubuntu systemd[2328]: Reached target Main User Target.
  Jun 16 23:44:59 ubuntu systemd[2328]: Startup finished in 69ms.
  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
  ...

  I propose that we restrict access to dmesg to users in group 'adm'
  like so:

  1) Add kernel.dmesg_restrict = 1 to
 /etc/sysctl.d/10-kernel-hardening.conf
  2) Following changes to /bin/dmesg permissions in package 'util-linux'
  - Ownership changes to root:adm
  - Permissions changed to 0750 (-rwxr-x---)
  - Add cap_syslog capability to binary.

  For most users, they will use the initial admin account, which is in
  the 'adm' group already, and will see no impact to these changes. If a
  log scraper type program needs access to dmesg, the user the daemon
  runs as can simply be added to the 'adm' group.

  [Testcase]

  Currently, all users can run /usr/bin/dmesg to view the kernel log
  buffer:

  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
  ...

  When the changes are applied, the default admin user will be able to
  view dmesg (since they are in group 'adm'), while new unprivileged
  users will not.

  Test packages are available in the following ppa:
  https://launchpad.net/~mruffell/+archive/ubuntu/lp1886112-test

  $ whoami
  ubuntu
  $ groups
  ubuntu adm cdrom sudo dip plugdev
  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
  ...

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ dmesg
  -bash: /usr/bin/dmesg: Permission denied

  [Regression Potential]

  Some users or log scraper type programs may need to view the kernel
  log buffer, or have access to dmesg. In this case, the underlying
  service user would need to be added to the 'adm' group.

  Users have the ability 

[Touch-packages] [Bug 1886112] Re: Enabling DMESG_RESTRICT in Groovy Onward

2020-07-23 Thread Matthew Ruffell
Attached is a debdiff for util-linux which implements the permission and
capability changes to the dmesg binary.

** Patch added: "util-linux debdiff for Groovy"
   
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1886112/+attachment/5395389/+files/lp1886112_util-linux_groovy.debdiff

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to util-linux in Ubuntu.
https://bugs.launchpad.net/bugs/1886112

Title:
  Enabling DMESG_RESTRICT in Groovy Onward

Status in linux package in Ubuntu:
  Fix Committed
Status in procps package in Ubuntu:
  In Progress
Status in util-linux package in Ubuntu:
  In Progress
Status in linux source package in Groovy:
  Fix Committed
Status in procps source package in Groovy:
  In Progress
Status in util-linux source package in Groovy:
  In Progress

Bug description:
  [Impact]

  This bug implements the enablement of CONFIG_SECURITY_DMESG_RESTRICT
  feature by default for Groovy onward, proposed to ubuntu-devel:

  https://lists.ubuntu.com/archives/ubuntu-devel/2020-June/041063.html

  The kernel log buffer contains a wealth of sensitive information, such
  as detailed call traces and kernel addresses found in register dumps
  in kernel oops messages.

  Exploit developers and attackers can leverage these information leaks
  to get past KASLR, and they can use the kernel log buffer to get
  instant feedback on their privilege escalation attacks, as failures
  will be shown as further oops messages, which attackers can use to fix
  and tune their programs until they work.

  Currently, if I create a new, unprivileged user on a Focal system,
  they cannot access /var/log/kern.log, /var/log/syslog or see system
  events in journalctl. But yet, they are given free reign to the kernel
  log buffer.

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ cat /var/log/kern.log
  cat: /var/log/kern.log: Permission denied
  $ cat /var/log/syslog
  cat: /var/log/syslog: Permission denied
  $ journalctl
  Hint: You are currently not seeing messages from other users and the system.
    Users in groups 'adm', 'systemd-journal' can see all messages.
    Pass -q to turn off this notice.
  Jun 16 23:44:59 ubuntu systemd[2328]: Reached target Main User Target.
  Jun 16 23:44:59 ubuntu systemd[2328]: Startup finished in 69ms.
  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
  ...

  I propose that we restrict access to dmesg to users in group 'adm'
  like so:

  1) Add kernel.dmesg_restrict = 1 to
 /etc/sysctl.d/10-kernel-hardening.conf
  2) Following changes to /bin/dmesg permissions in package 'util-linux'
  - Ownership changes to root:adm
  - Permissions changed to 0750 (-rwxr-x---)
  - Add cap_syslog capability to binary.

  For most users, they will use the initial admin account, which is in
  the 'adm' group already, and will see no impact to these changes. If a
  log scraper type program needs access to dmesg, the user the daemon
  runs as can simply be added to the 'adm' group.

  [Testcase]

  Currently, all users can run /usr/bin/dmesg to view the kernel log
  buffer:

  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
  ...

  When the changes are applied, the default admin user will be able to
  view dmesg (since they are in group 'adm'), while new unprivileged
  users will not.

  Test packages are available in the following ppa:
  https://launchpad.net/~mruffell/+archive/ubuntu/lp1886112-test

  $ whoami
  ubuntu
  $ groups
  ubuntu adm cdrom sudo dip plugdev
  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
  ...

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ dmesg
  -bash: /usr/bin/dmesg: Permission denied

  [Regression Potential]

  Some users or log scraper type programs may need to view the kernel
  log buffer, or have access to dmesg. In this case, the underlying
  service user would need to be added to the 'adm' group.

  Users have the ability to disable DMESG_RESTRICT by changing
  kernel.dmesg_restrict sysctl in /etc/sysctl.d/10-kernel-hardening.conf
  from '1' to '0', followed by a reboot.

To manage 

[Touch-packages] [Bug 1886112] Re: Enabling DMESG_RESTRICT in Groovy Onward

2020-07-23 Thread Matthew Ruffell
I emailed Seth Forshee asking about what happens when Groovy's kernel
becomes Focal's HWE kernel, and he mentioned that the kernel team has
processes in place to handle config changes, and that it isn't a
problem.

So we will go with the more secure by default way, and enable
CONFIG_SECURITY_DMESG_RESTRICT in the kernel config.

I rebased the patches for procps and util-linux to the latest versions
in groovy and their debdiffs are attached below. Since things are quiet
on this bug I will write to ubuntu-devel shortly.

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to util-linux in Ubuntu.
https://bugs.launchpad.net/bugs/1886112

Title:
  Enabling DMESG_RESTRICT in Groovy Onward

Status in linux package in Ubuntu:
  Fix Committed
Status in procps package in Ubuntu:
  In Progress
Status in util-linux package in Ubuntu:
  In Progress
Status in linux source package in Groovy:
  Fix Committed
Status in procps source package in Groovy:
  In Progress
Status in util-linux source package in Groovy:
  In Progress

Bug description:
  [Impact]

  This bug implements the enablement of CONFIG_SECURITY_DMESG_RESTRICT
  feature by default for Groovy onward, proposed to ubuntu-devel:

  https://lists.ubuntu.com/archives/ubuntu-devel/2020-June/041063.html

  The kernel log buffer contains a wealth of sensitive information, such
  as detailed call traces and kernel addresses found in register dumps
  in kernel oops messages.

  Exploit developers and attackers can leverage these information leaks
  to get past KASLR, and they can use the kernel log buffer to get
  instant feedback on their privilege escalation attacks, as failures
  will be shown as further oops messages, which attackers can use to fix
  and tune their programs until they work.

  Currently, if I create a new, unprivileged user on a Focal system,
  they cannot access /var/log/kern.log, /var/log/syslog or see system
  events in journalctl. But yet, they are given free reign to the kernel
  log buffer.

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ cat /var/log/kern.log
  cat: /var/log/kern.log: Permission denied
  $ cat /var/log/syslog
  cat: /var/log/syslog: Permission denied
  $ journalctl
  Hint: You are currently not seeing messages from other users and the system.
    Users in groups 'adm', 'systemd-journal' can see all messages.
    Pass -q to turn off this notice.
  Jun 16 23:44:59 ubuntu systemd[2328]: Reached target Main User Target.
  Jun 16 23:44:59 ubuntu systemd[2328]: Startup finished in 69ms.
  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
  ...

  I propose that we restrict access to dmesg to users in group 'adm'
  like so:

  1) Add kernel.dmesg_restrict = 1 to
 /etc/sysctl.d/10-kernel-hardening.conf
  2) Following changes to /bin/dmesg permissions in package 'util-linux'
  - Ownership changes to root:adm
  - Permissions changed to 0750 (-rwxr-x---)
  - Add cap_syslog capability to binary.

  For most users, they will use the initial admin account, which is in
  the 'adm' group already, and will see no impact to these changes. If a
  log scraper type program needs access to dmesg, the user the daemon
  runs as can simply be added to the 'adm' group.

  [Testcase]

  Currently, all users can run /usr/bin/dmesg to view the kernel log
  buffer:

  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
  ...

  When the changes are applied, the default admin user will be able to
  view dmesg (since they are in group 'adm'), while new unprivileged
  users will not.

  Test packages are available in the following ppa:
  https://launchpad.net/~mruffell/+archive/ubuntu/lp1886112-test

  $ whoami
  ubuntu
  $ groups
  ubuntu adm cdrom sudo dip plugdev
  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
  ...

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ dmesg
  -bash: /usr/bin/dmesg: Permission denied

  [Regression Potential]

  Some users or log scraper type programs may need to view the kernel
  log buffer, or have access to dmesg. In this case, the underlying
  service user would 

[Touch-packages] [Bug 1886112] Re: Enabling DMESG_RESTRICT in Groovy Onward

2020-07-23 Thread Matthew Ruffell
Attached is a procps debdiff for groovy, which adds documentation to
/etc/sysctl.d/10-kernel-hardening.conf and a commented out way to
disable DMESG_RESTRICT.

** Patch added: "procps debdiff for Groovy"
   
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1886112/+attachment/5395388/+files/lp1886112_procps_groovy.debdiff

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to util-linux in Ubuntu.
https://bugs.launchpad.net/bugs/1886112

Title:
  Enabling DMESG_RESTRICT in Groovy Onward

Status in linux package in Ubuntu:
  Fix Committed
Status in procps package in Ubuntu:
  In Progress
Status in util-linux package in Ubuntu:
  In Progress
Status in linux source package in Groovy:
  Fix Committed
Status in procps source package in Groovy:
  In Progress
Status in util-linux source package in Groovy:
  In Progress

Bug description:
  [Impact]

  This bug implements the enablement of CONFIG_SECURITY_DMESG_RESTRICT
  feature by default for Groovy onward, proposed to ubuntu-devel:

  https://lists.ubuntu.com/archives/ubuntu-devel/2020-June/041063.html

  The kernel log buffer contains a wealth of sensitive information, such
  as detailed call traces and kernel addresses found in register dumps
  in kernel oops messages.

  Exploit developers and attackers can leverage these information leaks
  to get past KASLR, and they can use the kernel log buffer to get
  instant feedback on their privilege escalation attacks, as failures
  will be shown as further oops messages, which attackers can use to fix
  and tune their programs until they work.

  Currently, if I create a new, unprivileged user on a Focal system,
  they cannot access /var/log/kern.log, /var/log/syslog or see system
  events in journalctl. But yet, they are given free reign to the kernel
  log buffer.

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ cat /var/log/kern.log
  cat: /var/log/kern.log: Permission denied
  $ cat /var/log/syslog
  cat: /var/log/syslog: Permission denied
  $ journalctl
  Hint: You are currently not seeing messages from other users and the system.
    Users in groups 'adm', 'systemd-journal' can see all messages.
    Pass -q to turn off this notice.
  Jun 16 23:44:59 ubuntu systemd[2328]: Reached target Main User Target.
  Jun 16 23:44:59 ubuntu systemd[2328]: Startup finished in 69ms.
  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
  ...

  I propose that we restrict access to dmesg to users in group 'adm'
  like so:

  1) Add kernel.dmesg_restrict = 1 to
 /etc/sysctl.d/10-kernel-hardening.conf
  2) Following changes to /bin/dmesg permissions in package 'util-linux'
  - Ownership changes to root:adm
  - Permissions changed to 0750 (-rwxr-x---)
  - Add cap_syslog capability to binary.

  For most users, they will use the initial admin account, which is in
  the 'adm' group already, and will see no impact to these changes. If a
  log scraper type program needs access to dmesg, the user the daemon
  runs as can simply be added to the 'adm' group.

  [Testcase]

  Currently, all users can run /usr/bin/dmesg to view the kernel log
  buffer:

  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
  ...

  When the changes are applied, the default admin user will be able to
  view dmesg (since they are in group 'adm'), while new unprivileged
  users will not.

  Test packages are available in the following ppa:
  https://launchpad.net/~mruffell/+archive/ubuntu/lp1886112-test

  $ whoami
  ubuntu
  $ groups
  ubuntu adm cdrom sudo dip plugdev
  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
  ...

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ dmesg
  -bash: /usr/bin/dmesg: Permission denied

  [Regression Potential]

  Some users or log scraper type programs may need to view the kernel
  log buffer, or have access to dmesg. In this case, the underlying
  service user would need to be added to the 'adm' group.

  Users have the ability to disable DMESG_RESTRICT by changing
  kernel.dmesg_restrict sysctl in /etc/sysctl.d/10-kernel-hardening.conf
  from '1' to '0', 

[Touch-packages] [Bug 1886112] Re: Enabling DMESG_RESTRICT in Groovy Onward

2020-07-05 Thread Matthew Ruffell
I have created patches for both the procps package and the util-linux
package which implements the proposed changes.

You can find test packages in the following ppa:

https://launchpad.net/~mruffell/+archive/ubuntu/lp1886112-test

Debdiff for procps: https://paste.ubuntu.com/p/qvmHgMhXSj/
Debdiff for util-linux: https://paste.ubuntu.com/p/SYrK9xrwnP/

I have tested the packages on a Groovy daily build, and the changes
function as intended. The default user, who is in the adm group can use
dmesg freely, and unprivileged users can no longer access dmesg.

I am looking for feedback about the maintainability of the util-linux
patches, particularly about the additional burden it would place on
merges performed in the future. My main worry is the additional
dependency of libcap2-bin needed to set CAP_SYSLOG on /bin/dmesg.

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to util-linux in Ubuntu.
https://bugs.launchpad.net/bugs/1886112

Title:
  Enabling DMESG_RESTRICT in Groovy Onward

Status in linux package in Ubuntu:
  Fix Committed
Status in procps package in Ubuntu:
  In Progress
Status in util-linux package in Ubuntu:
  In Progress
Status in linux source package in Groovy:
  Fix Committed
Status in procps source package in Groovy:
  In Progress
Status in util-linux source package in Groovy:
  In Progress

Bug description:
  [Impact]

  This bug implements the enablement of CONFIG_SECURITY_DMESG_RESTRICT
  feature by default for Groovy onward, proposed to ubuntu-devel:

  https://lists.ubuntu.com/archives/ubuntu-devel/2020-June/041063.html

  The kernel log buffer contains a wealth of sensitive information, such
  as detailed call traces and kernel addresses found in register dumps
  in kernel oops messages.

  Exploit developers and attackers can leverage these information leaks
  to get past KASLR, and they can use the kernel log buffer to get
  instant feedback on their privilege escalation attacks, as failures
  will be shown as further oops messages, which attackers can use to fix
  and tune their programs until they work.

  Currently, if I create a new, unprivileged user on a Focal system,
  they cannot access /var/log/kern.log, /var/log/syslog or see system
  events in journalctl. But yet, they are given free reign to the kernel
  log buffer.

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ cat /var/log/kern.log
  cat: /var/log/kern.log: Permission denied
  $ cat /var/log/syslog
  cat: /var/log/syslog: Permission denied
  $ journalctl
  Hint: You are currently not seeing messages from other users and the system.
    Users in groups 'adm', 'systemd-journal' can see all messages.
    Pass -q to turn off this notice.
  Jun 16 23:44:59 ubuntu systemd[2328]: Reached target Main User Target.
  Jun 16 23:44:59 ubuntu systemd[2328]: Startup finished in 69ms.
  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
  ...

  I propose that we restrict access to dmesg to users in group 'adm'
  like so:

  1) Add kernel.dmesg_restrict = 1 to
 /etc/sysctl.d/10-kernel-hardening.conf
  2) Following changes to /bin/dmesg permissions in package 'util-linux'
  - Ownership changes to root:adm
  - Permissions changed to 0750 (-rwxr-x---)
  - Add cap_syslog capability to binary.

  For most users, they will use the initial admin account, which is in
  the 'adm' group already, and will see no impact to these changes. If a
  log scraper type program needs access to dmesg, the user the daemon
  runs as can simply be added to the 'adm' group.

  [Testcase]

  Currently, all users can run /usr/bin/dmesg to view the kernel log
  buffer:

  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
  ...

  When the changes are applied, the default admin user will be able to
  view dmesg (since they are in group 'adm'), while new unprivileged
  users will not.

  Test packages are available in the following ppa:
  https://launchpad.net/~mruffell/+archive/ubuntu/lp1886112-test

  $ whoami
  ubuntu
  $ groups
  ubuntu adm cdrom sudo dip plugdev
  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 

[Touch-packages] [Bug 1886112] Re: Enabling DMESG_RESTRICT in Groovy Onward

2020-07-05 Thread Matthew Ruffell
I was thinking about this over the weekend, and I think we overlooked
the impact of setting CONFIG_SECURITY_DMESG_RESTRICT in the kernel
config has on downstream users of Groovy's kernel, namely when it
becomes Focal's HWE kernel.

Focal won't be receiving any patches for /usr/bin/dmesg, so I think it
is better to not set CONFIG_SECURITY_DMESG_RESTRICT in kernel config,
but to instead set kernel.dmesg_restrict systctl to 1 in
/etc/sysctl.d/10-kernel-hardening.conf. This would ensure it only
changes Groovy onward, and doesn't cause any regressions for Focal HWE
users.

I have emailed Seth Forshee asking to revert the config change.

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to util-linux in Ubuntu.
https://bugs.launchpad.net/bugs/1886112

Title:
  Enabling DMESG_RESTRICT in Groovy Onward

Status in linux package in Ubuntu:
  Fix Committed
Status in procps package in Ubuntu:
  In Progress
Status in util-linux package in Ubuntu:
  In Progress
Status in linux source package in Groovy:
  Fix Committed
Status in procps source package in Groovy:
  In Progress
Status in util-linux source package in Groovy:
  In Progress

Bug description:
  [Impact]

  This bug implements the enablement of CONFIG_SECURITY_DMESG_RESTRICT
  feature by default for Groovy onward, proposed to ubuntu-devel:

  https://lists.ubuntu.com/archives/ubuntu-devel/2020-June/041063.html

  The kernel log buffer contains a wealth of sensitive information, such
  as detailed call traces and kernel addresses found in register dumps
  in kernel oops messages.

  Exploit developers and attackers can leverage these information leaks
  to get past KASLR, and they can use the kernel log buffer to get
  instant feedback on their privilege escalation attacks, as failures
  will be shown as further oops messages, which attackers can use to fix
  and tune their programs until they work.

  Currently, if I create a new, unprivileged user on a Focal system,
  they cannot access /var/log/kern.log, /var/log/syslog or see system
  events in journalctl. But yet, they are given free reign to the kernel
  log buffer.

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ cat /var/log/kern.log
  cat: /var/log/kern.log: Permission denied
  $ cat /var/log/syslog
  cat: /var/log/syslog: Permission denied
  $ journalctl
  Hint: You are currently not seeing messages from other users and the system.
    Users in groups 'adm', 'systemd-journal' can see all messages.
    Pass -q to turn off this notice.
  Jun 16 23:44:59 ubuntu systemd[2328]: Reached target Main User Target.
  Jun 16 23:44:59 ubuntu systemd[2328]: Startup finished in 69ms.
  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
  ...

  I propose that we restrict access to dmesg to users in group 'adm'
  like so:

  1) Add kernel.dmesg_restrict = 1 to
 /etc/sysctl.d/10-kernel-hardening.conf
  2) Following changes to /bin/dmesg permissions in package 'util-linux'
  - Ownership changes to root:adm
  - Permissions changed to 0750 (-rwxr-x---)
  - Add cap_syslog capability to binary.

  For most users, they will use the initial admin account, which is in
  the 'adm' group already, and will see no impact to these changes. If a
  log scraper type program needs access to dmesg, the user the daemon
  runs as can simply be added to the 'adm' group.

  [Testcase]

  Currently, all users can run /usr/bin/dmesg to view the kernel log
  buffer:

  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
  ...

  When the changes are applied, the default admin user will be able to
  view dmesg (since they are in group 'adm'), while new unprivileged
  users will not.

  Test packages are available in the following ppa:
  https://launchpad.net/~mruffell/+archive/ubuntu/lp1886112-test

  $ whoami
  ubuntu
  $ groups
  ubuntu adm cdrom sudo dip plugdev
  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
  ...

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ dmesg
  -bash: /usr/bin/dmesg: Permission denied

  [Regression Potential]

  Some users or log scraper type programs may 

[Touch-packages] [Bug 1886112] Re: Enabling DMESG_RESTRICT in Groovy Onward

2020-07-05 Thread Matthew Ruffell
** Description changed:

  [Impact]
  
  This bug implements the enablement of CONFIG_SECURITY_DMESG_RESTRICT
  feature by default for Groovy onward, proposed to ubuntu-devel:
  
  https://lists.ubuntu.com/archives/ubuntu-devel/2020-June/041063.html
  
  The kernel log buffer contains a wealth of sensitive information, such
  as detailed call traces and kernel addresses found in register dumps in
  kernel oops messages.
  
  Exploit developers and attackers can leverage these information leaks to
  get past KASLR, and they can use the kernel log buffer to get instant
  feedback on their privilege escalation attacks, as failures will be
  shown as further oops messages, which attackers can use to fix and tune
  their programs until they work.
  
  Currently, if I create a new, unprivileged user on a Focal system, they
  cannot access /var/log/kern.log, /var/log/syslog or see system events in
  journalctl. But yet, they are given free reign to the kernel log buffer.
  
  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ cat /var/log/kern.log
  cat: /var/log/kern.log: Permission denied
  $ cat /var/log/syslog
  cat: /var/log/syslog: Permission denied
  $ journalctl
  Hint: You are currently not seeing messages from other users and the system.
-   Users in groups 'adm', 'systemd-journal' can see all messages.
-   Pass -q to turn off this notice.
+   Users in groups 'adm', 'systemd-journal' can see all messages.
+   Pass -q to turn off this notice.
  Jun 16 23:44:59 ubuntu systemd[2328]: Reached target Main User Target.
  Jun 16 23:44:59 ubuntu systemd[2328]: Startup finished in 69ms.
  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
  ...
  
  I propose that we restrict access to dmesg to users in group 'adm' like
  so:
  
- 1) CONFIG_SECURITY_DMESG_RESTRICT=y in the kernel.
+ 1) Add kernel.dmesg_restrict = 1 to
+/etc/sysctl.d/10-kernel-hardening.conf
  2) Following changes to /bin/dmesg permissions in package 'util-linux'
- - Ownership changes to root:adm
- - Permissions changed to 0750 (-rwxr-x---)
- - Add cap_syslog capability to binary.
- 3) Add a commented out '# kernel.dmesg_restrict = 0' to
-/etc/sysctl.d/10-kernel-hardening.conf
+ - Ownership changes to root:adm
+ - Permissions changed to 0750 (-rwxr-x---)
+ - Add cap_syslog capability to binary.
  
  For most users, they will use the initial admin account, which is in the
  'adm' group already, and will see no impact to these changes. If a log
  scraper type program needs access to dmesg, the user the daemon runs as
  can simply be added to the 'adm' group.
  
  [Testcase]
  
  Currently, all users can run /usr/bin/dmesg to view the kernel log
  buffer:
  
  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
  ...
  
  When the changes are applied, the default admin user will be able to
  view dmesg (since they are in group 'adm'), while new unprivileged users
  will not.
+ 
+ Test packages are available in the following ppa:
+ https://launchpad.net/~mruffell/+archive/ubuntu/lp1886112-test
  
  $ whoami
  ubuntu
  $ groups
  ubuntu adm cdrom sudo dip plugdev
  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
  ...
  
  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ dmesg
  -bash: /usr/bin/dmesg: Permission denied
  
  [Regression Potential]
  
  Some users or log scraper type programs may need to view the kernel log
  buffer, or have access to dmesg. In this case, the underlying service
  user would need to be added to the 'adm' group.
  
- Users have the ability to disable DMESG_RESTRICT by uncommenting the
- sysctl in /etc/sysctl.d/10-kernel-hardening.conf.
+ Users have the ability to disable DMESG_RESTRICT by changing
+ kernel.dmesg_restrict sysctl in /etc/sysctl.d/10-kernel-hardening.conf
+ from '1' to '0', followed by a reboot.

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to util-linux in Ubuntu.
https://bugs.launchpad.net/bugs/1886112

Title:
  Enabling DMESG_RESTRICT in Groovy Onward

Status in linux package in Ubuntu:
  Fix Committed
Status in procps package in 

[Touch-packages] [Bug 1886112] Re: Enabling DMESG_RESTRICT in Groovy Onward

2020-07-03 Thread Matthew Ruffell
** Patch removed: "procps debdiff for Groovy"
   
https://bugs.launchpad.net/ubuntu/+source/util-linux/+bug/1886112/+attachment/5389194/+files/lp1886112_procps_groovy.debdiff

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to util-linux in Ubuntu.
https://bugs.launchpad.net/bugs/1886112

Title:
  Enabling DMESG_RESTRICT in Groovy Onward

Status in linux package in Ubuntu:
  Fix Committed
Status in procps package in Ubuntu:
  In Progress
Status in util-linux package in Ubuntu:
  In Progress
Status in linux source package in Groovy:
  Fix Committed
Status in procps source package in Groovy:
  In Progress
Status in util-linux source package in Groovy:
  In Progress

Bug description:
  [Impact]

  This bug implements the enablement of CONFIG_SECURITY_DMESG_RESTRICT
  feature by default for Groovy onward, proposed to ubuntu-devel:

  https://lists.ubuntu.com/archives/ubuntu-devel/2020-June/041063.html

  The kernel log buffer contains a wealth of sensitive information, such
  as detailed call traces and kernel addresses found in register dumps
  in kernel oops messages.

  Exploit developers and attackers can leverage these information leaks
  to get past KASLR, and they can use the kernel log buffer to get
  instant feedback on their privilege escalation attacks, as failures
  will be shown as further oops messages, which attackers can use to fix
  and tune their programs until they work.

  Currently, if I create a new, unprivileged user on a Focal system,
  they cannot access /var/log/kern.log, /var/log/syslog or see system
  events in journalctl. But yet, they are given free reign to the kernel
  log buffer.

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ cat /var/log/kern.log
  cat: /var/log/kern.log: Permission denied
  $ cat /var/log/syslog
  cat: /var/log/syslog: Permission denied
  $ journalctl
  Hint: You are currently not seeing messages from other users and the system.
Users in groups 'adm', 'systemd-journal' can see all messages.
Pass -q to turn off this notice.
  Jun 16 23:44:59 ubuntu systemd[2328]: Reached target Main User Target.
  Jun 16 23:44:59 ubuntu systemd[2328]: Startup finished in 69ms.
  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
  ...

  I propose that we restrict access to dmesg to users in group 'adm'
  like so:

  1) CONFIG_SECURITY_DMESG_RESTRICT=y in the kernel.
  2) Following changes to /bin/dmesg permissions in package 'util-linux'
  - Ownership changes to root:adm
  - Permissions changed to 0750 (-rwxr-x---)
  - Add cap_syslog capability to binary.
  3) Add a commented out '# kernel.dmesg_restrict = 0' to
 /etc/sysctl.d/10-kernel-hardening.conf

  For most users, they will use the initial admin account, which is in
  the 'adm' group already, and will see no impact to these changes. If a
  log scraper type program needs access to dmesg, the user the daemon
  runs as can simply be added to the 'adm' group.

  [Testcase]

  Currently, all users can run /usr/bin/dmesg to view the kernel log
  buffer:

  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
  ...

  When the changes are applied, the default admin user will be able to
  view dmesg (since they are in group 'adm'), while new unprivileged
  users will not.

  $ whoami
  ubuntu
  $ groups
  ubuntu adm cdrom sudo dip plugdev
  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
  ...

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ dmesg
  -bash: /usr/bin/dmesg: Permission denied

  [Regression Potential]

  Some users or log scraper type programs may need to view the kernel
  log buffer, or have access to dmesg. In this case, the underlying
  service user would need to be added to the 'adm' group.

  Users have the ability to disable DMESG_RESTRICT by uncommenting the
  sysctl in /etc/sysctl.d/10-kernel-hardening.conf.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1886112/+subscriptions

-- 
Mailing list: https://launchpad.net/~touch-packages
Post to : touch-packages@lists.launchpad.net

[Touch-packages] [Bug 1886112] Re: Enabling DMESG_RESTRICT in Groovy Onward

2020-07-02 Thread Matthew Ruffell
Attached is a debdiff for procps on Groovy. It adds a commented out
entry to 10-kernel-hardening.conf which users can use to disable the
setting if they wish.

** Patch added: "procps debdiff for Groovy"
   
https://bugs.launchpad.net/ubuntu/+source/util-linux/+bug/1886112/+attachment/5389194/+files/lp1886112_procps_groovy.debdiff

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to procps in Ubuntu.
https://bugs.launchpad.net/bugs/1886112

Title:
  Enabling DMESG_RESTRICT in Groovy Onward

Status in linux package in Ubuntu:
  Fix Committed
Status in procps package in Ubuntu:
  In Progress
Status in util-linux package in Ubuntu:
  In Progress
Status in linux source package in Groovy:
  Fix Committed
Status in procps source package in Groovy:
  In Progress
Status in util-linux source package in Groovy:
  In Progress

Bug description:
  [Impact]

  This bug implements the enablement of CONFIG_SECURITY_DMESG_RESTRICT
  feature by default for Groovy onward, proposed to ubuntu-devel:

  https://lists.ubuntu.com/archives/ubuntu-devel/2020-June/041063.html

  The kernel log buffer contains a wealth of sensitive information, such
  as detailed call traces and kernel addresses found in register dumps
  in kernel oops messages.

  Exploit developers and attackers can leverage these information leaks
  to get past KASLR, and they can use the kernel log buffer to get
  instant feedback on their privilege escalation attacks, as failures
  will be shown as further oops messages, which attackers can use to fix
  and tune their programs until they work.

  Currently, if I create a new, unprivileged user on a Focal system,
  they cannot access /var/log/kern.log, /var/log/syslog or see system
  events in journalctl. But yet, they are given free reign to the kernel
  log buffer.

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ cat /var/log/kern.log
  cat: /var/log/kern.log: Permission denied
  $ cat /var/log/syslog
  cat: /var/log/syslog: Permission denied
  $ journalctl
  Hint: You are currently not seeing messages from other users and the system.
Users in groups 'adm', 'systemd-journal' can see all messages.
Pass -q to turn off this notice.
  Jun 16 23:44:59 ubuntu systemd[2328]: Reached target Main User Target.
  Jun 16 23:44:59 ubuntu systemd[2328]: Startup finished in 69ms.
  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
  ...

  I propose that we restrict access to dmesg to users in group 'adm'
  like so:

  1) CONFIG_SECURITY_DMESG_RESTRICT=y in the kernel.
  2) Following changes to /bin/dmesg permissions in package 'util-linux'
  - Ownership changes to root:adm
  - Permissions changed to 0750 (-rwxr-x---)
  - Add cap_syslog capability to binary.
  3) Add a commented out '# kernel.dmesg_restrict = 0' to
 /etc/sysctl.d/10-kernel-hardening.conf

  For most users, they will use the initial admin account, which is in
  the 'adm' group already, and will see no impact to these changes. If a
  log scraper type program needs access to dmesg, the user the daemon
  runs as can simply be added to the 'adm' group.

  [Testcase]

  Currently, all users can run /usr/bin/dmesg to view the kernel log
  buffer:

  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
  ...

  When the changes are applied, the default admin user will be able to
  view dmesg (since they are in group 'adm'), while new unprivileged
  users will not.

  $ whoami
  ubuntu
  $ groups
  ubuntu adm cdrom sudo dip plugdev
  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
  ...

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ dmesg
  -bash: /usr/bin/dmesg: Permission denied

  [Regression Potential]

  Some users or log scraper type programs may need to view the kernel
  log buffer, or have access to dmesg. In this case, the underlying
  service user would need to be added to the 'adm' group.

  Users have the ability to disable DMESG_RESTRICT by uncommenting the
  sysctl in /etc/sysctl.d/10-kernel-hardening.conf.

To manage notifications about this bug go to:

[Touch-packages] [Bug 1886112] [NEW] Enabling DMESG_RESTRICT in Groovy Onward

2020-07-02 Thread Matthew Ruffell
Public bug reported:

[Impact]

This bug implements the enablement of CONFIG_SECURITY_DMESG_RESTRICT
feature by default for Groovy onward, proposed to ubuntu-devel:

https://lists.ubuntu.com/archives/ubuntu-devel/2020-June/041063.html

The kernel log buffer contains a wealth of sensitive information, such
as detailed call traces and kernel addresses found in register dumps in
kernel oops messages.

Exploit developers and attackers can leverage these information leaks to
get past KASLR, and they can use the kernel log buffer to get instant
feedback on their privilege escalation attacks, as failures will be
shown as further oops messages, which attackers can use to fix and tune
their programs until they work.

Currently, if I create a new, unprivileged user on a Focal system, they
cannot access /var/log/kern.log, /var/log/syslog or see system events in
journalctl. But yet, they are given free reign to the kernel log buffer.

$ sudo adduser dave
$ su dave
$ groups
dave
$ cat /var/log/kern.log
cat: /var/log/kern.log: Permission denied
$ cat /var/log/syslog
cat: /var/log/syslog: Permission denied
$ journalctl
Hint: You are currently not seeing messages from other users and the system.
  Users in groups 'adm', 'systemd-journal' can see all messages.
  Pass -q to turn off this notice.
Jun 16 23:44:59 ubuntu systemd[2328]: Reached target Main User Target.
Jun 16 23:44:59 ubuntu systemd[2328]: Startup finished in 69ms.
$ dmesg
[0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
(gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 15:46:55
UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
[0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
...

I propose that we restrict access to dmesg to users in group 'adm' like
so:

1) CONFIG_SECURITY_DMESG_RESTRICT=y in the kernel.
2) Following changes to /bin/dmesg permissions in package 'util-linux'
- Ownership changes to root:adm
- Permissions changed to 0750 (-rwxr-x---)
- Add cap_syslog capability to binary.
3) Add a commented out '# kernel.dmesg_restrict = 0' to
   /etc/sysctl.d/10-kernel-hardening.conf

For most users, they will use the initial admin account, which is in the
'adm' group already, and will see no impact to these changes. If a log
scraper type program needs access to dmesg, the user the daemon runs as
can simply be added to the 'adm' group.

[Testcase]

Currently, all users can run /usr/bin/dmesg to view the kernel log
buffer:

$ dmesg
[0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
(gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 15:46:55
UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
[0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
...

When the changes are applied, the default admin user will be able to
view dmesg (since they are in group 'adm'), while new unprivileged users
will not.

$ whoami
ubuntu
$ groups
ubuntu adm cdrom sudo dip plugdev
$ dmesg
[0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
(gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 15:46:55
UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
[0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
...

$ sudo adduser dave
$ su dave
$ groups
dave
$ dmesg
-bash: /usr/bin/dmesg: Permission denied

[Regression Potential]

Some users or log scraper type programs may need to view the kernel log
buffer, or have access to dmesg. In this case, the underlying service
user would need to be added to the 'adm' group.

Users have the ability to disable DMESG_RESTRICT by uncommenting the
sysctl in /etc/sysctl.d/10-kernel-hardening.conf.

** Affects: linux (Ubuntu)
 Importance: Undecided
 Status: Fix Committed

** Affects: procps (Ubuntu)
 Importance: Undecided
 Assignee: Matthew Ruffell (mruffell)
 Status: In Progress

** Affects: util-linux (Ubuntu)
 Importance: Undecided
 Assignee: Matthew Ruffell (mruffell)
 Status: In Progress

** Affects: linux (Ubuntu Groovy)
 Importance: Undecided
 Status: Fix Committed

** Affects: procps (Ubuntu Groovy)
 Importance: Undecided
 Assignee: Matthew Ruffell (mruffell)
 Status: In Progress

** Affects: util-linux (Ubuntu Groovy)
 Importance: Undecided
 Assignee: Matthew Ruffell (mruffell)
 Status: In Progress

** Also affects: linux (Ubuntu Groovy)
   Importance: Undecided
   Status: New

** Changed in: linux (Ubuntu Groovy)
   Status: New => Fix Committed

** Also affects: procps (Ubuntu)
   Importance: Undecided
   Status: New

** Changed in: procps (Ubuntu Groovy)
   Status: New => In Progress

** Changed in: procps (Ubuntu Groovy)
 Assignee: (unassigned) => Matthew Ruffell (mruffell)

** Also affects: u

[Touch-packages] [Bug 1886112] Re: Enabling DMESG_RESTRICT in Groovy Onward

2020-07-02 Thread Matthew Ruffell
Kernel is fix-committed as per:

Mailing list:
https://lists.ubuntu.com/archives/ubuntu-devel/2020-July/041079.html

Commit:
https://kernel.ubuntu.com/git/ubuntu/unstable.git/commit/?id=25e6c851704a47c81e78e1a82530ac4b328098a6

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to procps in Ubuntu.
https://bugs.launchpad.net/bugs/1886112

Title:
  Enabling DMESG_RESTRICT in Groovy Onward

Status in linux package in Ubuntu:
  Fix Committed
Status in procps package in Ubuntu:
  In Progress
Status in util-linux package in Ubuntu:
  In Progress
Status in linux source package in Groovy:
  Fix Committed
Status in procps source package in Groovy:
  In Progress
Status in util-linux source package in Groovy:
  In Progress

Bug description:
  [Impact]

  This bug implements the enablement of CONFIG_SECURITY_DMESG_RESTRICT
  feature by default for Groovy onward, proposed to ubuntu-devel:

  https://lists.ubuntu.com/archives/ubuntu-devel/2020-June/041063.html

  The kernel log buffer contains a wealth of sensitive information, such
  as detailed call traces and kernel addresses found in register dumps
  in kernel oops messages.

  Exploit developers and attackers can leverage these information leaks
  to get past KASLR, and they can use the kernel log buffer to get
  instant feedback on their privilege escalation attacks, as failures
  will be shown as further oops messages, which attackers can use to fix
  and tune their programs until they work.

  Currently, if I create a new, unprivileged user on a Focal system,
  they cannot access /var/log/kern.log, /var/log/syslog or see system
  events in journalctl. But yet, they are given free reign to the kernel
  log buffer.

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ cat /var/log/kern.log
  cat: /var/log/kern.log: Permission denied
  $ cat /var/log/syslog
  cat: /var/log/syslog: Permission denied
  $ journalctl
  Hint: You are currently not seeing messages from other users and the system.
Users in groups 'adm', 'systemd-journal' can see all messages.
Pass -q to turn off this notice.
  Jun 16 23:44:59 ubuntu systemd[2328]: Reached target Main User Target.
  Jun 16 23:44:59 ubuntu systemd[2328]: Startup finished in 69ms.
  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
  ...

  I propose that we restrict access to dmesg to users in group 'adm'
  like so:

  1) CONFIG_SECURITY_DMESG_RESTRICT=y in the kernel.
  2) Following changes to /bin/dmesg permissions in package 'util-linux'
  - Ownership changes to root:adm
  - Permissions changed to 0750 (-rwxr-x---)
  - Add cap_syslog capability to binary.
  3) Add a commented out '# kernel.dmesg_restrict = 0' to
 /etc/sysctl.d/10-kernel-hardening.conf

  For most users, they will use the initial admin account, which is in
  the 'adm' group already, and will see no impact to these changes. If a
  log scraper type program needs access to dmesg, the user the daemon
  runs as can simply be added to the 'adm' group.

  [Testcase]

  Currently, all users can run /usr/bin/dmesg to view the kernel log
  buffer:

  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
  ...

  When the changes are applied, the default admin user will be able to
  view dmesg (since they are in group 'adm'), while new unprivileged
  users will not.

  $ whoami
  ubuntu
  $ groups
  ubuntu adm cdrom sudo dip plugdev
  $ dmesg
  [0.00] Linux version 5.4.0-34-generic (buildd at lcy01-amd64-014)
  (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #38-Ubuntu SMP Mon May 25 
15:46:55
  UTC 2020 (Ubuntu 5.4.0-34.38-generic 5.4.41)
  [0.00] Command line: BOOT_IMAGE=/boot/vmlinuz-5.4.0-34-generic
  root=UUID=f9f909c3-782a-43c2-a59d-c789656b4188 ro
  ...

  $ sudo adduser dave
  $ su dave
  $ groups
  dave
  $ dmesg
  -bash: /usr/bin/dmesg: Permission denied

  [Regression Potential]

  Some users or log scraper type programs may need to view the kernel
  log buffer, or have access to dmesg. In this case, the underlying
  service user would need to be added to the 'adm' group.

  Users have the ability to disable DMESG_RESTRICT by uncommenting the
  sysctl in /etc/sysctl.d/10-kernel-hardening.conf.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1886112/+subscriptions

-- 
Mailing list: https://launchpad.net/~touch-packages
Post to  

[Touch-packages] [Bug 1876230] Re: liburcu: Enable MEMBARRIER_CMD_PRIVATE_EXPEDITED to address performance problems with MEMBARRIER_CMD_SHARED

2020-05-11 Thread Matthew Ruffell
> - apart from feedback given by @mruffell, to also check if any of
librcu consumers are depending on a full membarrier - driven by kernel -
for ** shared pages among different processes **

I agree with @ddstreet, I don't think liburcu gives that sort of
guarantee when it comes to cross process synchronisation. It was my
belief that liburcu targets synchronisation across a set of threads
within the current process only.

Proof by contradiction.

Assume that a program compiles against liburcu and uses it to
synchronise access to shared memory pages for IPC between a sister
process.

If the program links against liburcu 0.9 or lower, the sys_membarrier
syscall did not exist yet, and liburcu will use the default compiler
based membarrier, which is only good within the current process.
Synchronisation across shared memory pages fails. This is the case on
Xenial, Trusty and the like.

If the program links against liburcu 0.11 or newer, the sys_membarrier
syscall does exist, but MEMBARRIER_CMD_SHARED is only used if the
current running kernel does not support
MEMBARRIER_CMD_PRIVATE_EXPEDITED. There is no toggle option in the API
at all, so for users with a kernel 4.14 or higher,
MEMBARRIER_CMD_PRIVATE_EXPEDITED will be used, and synchronisation
across shared memory pages will fail. This is the case on Eoan, Focal,
Groovy.

If the program links against liburcu 0.10, and uses the -qsbr, -md and
-signal variants, sys_membarrier is not used at all, and it falls back
to the compiler based membarrier, which is only good within the current
process. Synchronisation across shared memory pages will fail.

If the program links against liburcu 0.10, and is used within a
container, with a kernel version less than 4.3 that does not support
sys_membarrier, such as a Bionic container on a Trusty 3.13 host, or on
a 3.10 RHEL host, the sys_membarrier syscall fails, and it falls back to
the compiler based membarrier. Synchronisation across shared memory
pages will fail.

Now, the upstream developers added MEMBARRIER_CMD_PRIVATE_EXPEDITED as
the default in liburcu 0.11. They did not change the API to accommodate
both MEMBARRIER_CMD_SHARED and MEMBARRIER_CMD_PRIVATE_EXPEDITED, and
instead, if the kernel is greater than 4.14,
MEMBARRIER_CMD_PRIVATE_EXPEDITED will be used. Upstream are well aware
of their consumers, and they would not break everyone's usages out of
the blue, without adding some sort of API provision for legacy users.

Thus, our initial assumption that liburcu can be used to synchronise
access to shared memory pages for IPC between a sister process is wrong,
since no one will create a program that potentially only works in one
specific environment, which is bionic on bare metal and liburcu 0.10
only. I'm not even sure how you would co-ordinate liburcu over multiple
processes either.

So, because of the above, I don't think any librcu consumers are
depending on a full membarrier, driven by the kernel, for shared pages
among different processes.

I still think this is safe to SRU.

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to liburcu in Ubuntu.
https://bugs.launchpad.net/bugs/1876230

Title:
  liburcu: Enable MEMBARRIER_CMD_PRIVATE_EXPEDITED to address
  performance problems with MEMBARRIER_CMD_SHARED

Status in liburcu package in Ubuntu:
  Fix Released
Status in liburcu source package in Bionic:
  In Progress

Bug description:
  [Impact]

  In Linux 4.3, a new syscall was defined, called "membarrier". This
  systemcall was defined specifically for use in userspace-rcu (liburcu)
  to speed up the fast path / reader side of the library. The original
  implementation in Linux 4.3 only supported the MEMBARRIER_CMD_SHARED
  subcommand of the membarrier syscall.

  MEMBARRIER_CMD_SHARED executes a memory barrier on all threads from
  all processes running on the system. When it exits, the userspace
  thread which called it is guaranteed that all running threads share
  the same world view in regards to userspace addresses which are
  consumed by readers and writers.

  The problem with MEMBARRIER_CMD_SHARED is system calls made in this
  fashion can block, since it deploys a barrier across all threads in a
  system, and some other threads can be waiting on blocking operations,
  and take time to reach the barrier.

  In Linux 4.14, this was addressed by adding the
  MEMBARRIER_CMD_PRIVATE_EXPEDITED command to the membarrier syscall. It
  only targets threads which share the same mm as the thread calling the
  membarrier syscall, aka, threads in the current process, and not all
  threads / processes in the system.

  Calls to membarrier with the MEMBARRIER_CMD_PRIVATE_EXPEDITED command
  are guaranteed non-blocking, due to using inter-processor interrupts
  to implement memory barriers.

  Because of this, membarrier calls that use
  MEMBARRIER_CMD_PRIVATE_EXPEDITED are much faster than those that use
  MEMBARRIER_CMD_SHARED.

  Since Bionic uses a 

[Touch-packages] [Bug 1872863] Re: QEMU/KVM display is garbled when booting from kernel EFI stub due to missing bochs-drm module

2020-05-10 Thread Matthew Ruffell
I went and looked at the autopkgtest regressions, and they aren't actual
regressions, just transient network problems and slow build timeouts.

chrony autopkgtest regression on armhf:

Need to get 263 kB of archives.
After this operation, 2,048 B of additional disk space will be used.
Err:1 http://ftpmaster.internal/ubuntu bionic-proposed/main armhf kmod armhf 
24-1ubuntu3.4
  Could not connect to ftpmaster.internal:80 (91.189.89.99), connection timed 
out
Err:2 http://ftpmaster.internal/ubuntu bionic-proposed/main armhf libkmod2 
armhf 24-1ubuntu3.4
  Unable to connect to ftpmaster.internal:http:
  
Network problems for this one.

  
All the linux packages for arm64:

autopkgtest [15:44:38]: ERROR: timed out on command 
/tmp/autopkgtest.M7KUyX/build.f2S/src/debian/tests/rebuild
autopkgtest [15:48:48]: test rebuild: ---]
rebuild  FAIL timed out

Slow builds for these packages. Maybe the arm64 build host was having a
bad day.

I think the autopkgtests just need to be re-run.

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to kmod in Ubuntu.
https://bugs.launchpad.net/bugs/1872863

Title:
  QEMU/KVM display is garbled when booting from kernel EFI stub due to
  missing bochs-drm module

Status in kmod package in Ubuntu:
  Fix Released
Status in linux package in Ubuntu:
  Fix Released
Status in kmod source package in Bionic:
  Fix Committed
Status in linux source package in Bionic:
  Fix Committed

Bug description:
  BugLink: https://bugs.launchpad.net/bugs/1872863

  [Impact]

  A recent grub2 SRU, LP #1864533, now forces the kernel to boot via the
  kernel EFI stub whenever EFI is enabled. This causes problems for
  QEMU/KVM virtual machines which use the VGA=std video device, as the
  efifb driver yields an unreadable garbled screen. See the attached
  image.

  The correct framebuffer driver to use in this situation is bochs-drm,
  and modprobing it from a HWE kernel fixes the issues.

  bochs-drm is missing from Bionic since CONFIG_DRM_BOCHS was disabled
  in LP #1378648 due to bochs-drm causing problems in a PowerKVM
  machine. This problem appears to be fixed now, and bochs-drm has been
  re-enabled for Disco and up, in LP #1795857 and has been proven safe.

  This has also come up again in LP #1823152, as well as chatter on LP
  #1795857 to get this enabled on Bionic.

  The customer which is experiencing this issue cannot switch to VGA=qxl
  as a workaround, and must use VGA=std, hence I suggest we re-enable
  bochs-drm for Bionic.

  [Fix]

  I noticed on Focal, if you boot, the framebuffer is initially efifb:

  [ 0.603716] efifb: probing for efifb
  [ 0.603733] efifb: framebuffer at 0xc000, using 1876k, total 1875k
  [ 0.603735] efifb: mode is 800x600x32, linelength=3200, pages=1
  [ 0.603736] efifb: scrolling: redraw
  [ 0.603738] efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0
  [ 0.604462] Console: switching to colour frame buffer device 100x37
  [ 0.605829] fb0: EFI VGA frame buffer device

  This soon changes to bochs-drm about a second later:

  [ 0.935988] bochs-drm :00:01.0: remove_conflicting_pci_framebuffers: bar 
0: 0xc000 -> 0xc0ff
  [ 0.937023] bochs-drm :00:01.0: remove_conflicting_pci_framebuffers: bar 
2: 0xc1c8c000 -> 0xc1c8cfff
  [ 0.937776] checking generic (c000 1d5000) vs hw (c000 100)
  [ 0.937776] fb0: switching to bochsdrmfb from EFI VGA
  [ 0.939085] Console: switching to colour dummy device 80x25
  [ 0.939117] bochs-drm :00:01.0: vgaarb: deactivate vga console
  [ 0.939210] [drm] Found bochs VGA, ID 0xb0c5.
  [ 0.939212] [drm] Framebuffer size 16384 kB @ 0xc000, mmio @ 0xc1c8c000.
  [ 0.941955] lpc_ich :00:1f.0: I/O space for GPIO uninitialized
  [ 0.942069] [TTM] Zone kernel: Available graphics memory: 2006780 KiB
  [ 0.942081] [TTM] Initializing pool allocator
  [ 0.942089] [TTM] Initializing DMA pool allocator
  [ 0.943026] virtio_blk virtio2: [vda] 20971520 512-byte logical blocks (10.7 
GB/10.0 GiB)
  [ 0.944019] [drm] Found EDID data blob.
  [ 0.944162] [drm] Initialized bochs-drm 1.0.0 20130925 for :00:01.0 on 
minor 0
  [ 0.944979] fbcon: bochs-drmdrmfb (fb0) is primary device
  [ 0.947712] Console: switching to colour frame buffer device 128x48

  On bionic, the framebuffer never changes from efifb, since the bochs-
  drm kernel module is not built, and it is also present on the module
  banlist in /etc/modprobe.d/blacklist-framebuffer.conf

  bochs-drm needs to be enabled to be built in the kernel config, and
  removed from the module blacklist in kmod.

  [Testcase]

  Create a new QEMU/KVM virtual machine, I used virt-manager. Before you
  install the OS, check the box to modify settings before install. In
  the "Overview" tab, enable EFI by setting the firmware to "UEFI
  x86_64: /usr/share/OVMF/OVMF_CODE.secboot.fd".

  Set the video device to qxl while you install Bionic. Once the install
  is done, reboot, do 

[Touch-packages] [Bug 1872863] Re: QEMU/KVM display is garbled when booting from kernel EFI stub due to missing bochs-drm module

2020-05-10 Thread Matthew Ruffell
I repeated the same SRU verification as before, this time using kmod
from -proposed.

I installed a fresh Bionic KVM, with EFI enabled. I upgraded the system
to use the latest -updates packages. Linux 4.15.0-99-generic and Kmod
24-1ubuntu3.3 were installed. I shut the machine down, set the VGA
device to vga=std, and rebooted. The screen was garbled and unreadable.

I then ssh'd in, enabled -proposed, and installed Linux
4.15.0-100-generic, and kmod 24-1ubuntu3.4. I rebooted, and the display
came up great, and was perfectly readable. Attached screenshot as proof.

Relevent details from dmesg:

[1.137544] checking generic (c000 1d5000) vs hw (c000 100)
[1.137545] fb: switching to bochsdrmfb from EFI VGA
[1.137918] Console: switching to colour dummy device 80x25
[1.140305] [drm] Found bochs VGA, ID 0xb0c5.
[1.140308] [drm] Framebuffer size 16384 kB @ 0xc000, mmio @ 0xc1c8c000.
[1.140616] [TTM] Zone  kernel: Available graphics memory: 2011058 kiB
[1.140619] [TTM] Initializing pool allocator
[1.140622] [TTM] Initializing DMA pool allocator
[1.146542] fbcon: bochsdrmfb (fb0) is primary device
[1.149357] Console: switching to colour frame buffer device 128x48
[1.151830] bochs-drm :00:01.0: fb0: bochsdrmfb frame buffer device
[1.168977] [drm] Initialized bochs-drm 1.0.0 20130925 for :00:01.0 on 
minor 0

lsmod:

$ lsmod | grep bochs
bochs_drm  20480  1
ttm   106496  1 bochs_drm
drm_kms_helper172032  1 bochs_drm
drm   401408  4 drm_kms_helper,bochs_drm,ttm

Since the Linux package builds bochs-drm, and the kmod package removes
the blacklist, I am happy to mark this verified for both packages.

** Attachment added: "screenshot of vga=std with 4.15.0-100-generic and kmod 
24-1ubuntu3.4"
   
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1872863/+attachment/5369638/+files/Screenshot%20from%202020-05-11%2010-41-12.png

** Tags removed: verification-needed verification-needed-bionic
** Tags added: verification-done-bionic

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to kmod in Ubuntu.
https://bugs.launchpad.net/bugs/1872863

Title:
  QEMU/KVM display is garbled when booting from kernel EFI stub due to
  missing bochs-drm module

Status in kmod package in Ubuntu:
  Fix Released
Status in linux package in Ubuntu:
  Fix Released
Status in kmod source package in Bionic:
  Fix Committed
Status in linux source package in Bionic:
  Fix Committed

Bug description:
  BugLink: https://bugs.launchpad.net/bugs/1872863

  [Impact]

  A recent grub2 SRU, LP #1864533, now forces the kernel to boot via the
  kernel EFI stub whenever EFI is enabled. This causes problems for
  QEMU/KVM virtual machines which use the VGA=std video device, as the
  efifb driver yields an unreadable garbled screen. See the attached
  image.

  The correct framebuffer driver to use in this situation is bochs-drm,
  and modprobing it from a HWE kernel fixes the issues.

  bochs-drm is missing from Bionic since CONFIG_DRM_BOCHS was disabled
  in LP #1378648 due to bochs-drm causing problems in a PowerKVM
  machine. This problem appears to be fixed now, and bochs-drm has been
  re-enabled for Disco and up, in LP #1795857 and has been proven safe.

  This has also come up again in LP #1823152, as well as chatter on LP
  #1795857 to get this enabled on Bionic.

  The customer which is experiencing this issue cannot switch to VGA=qxl
  as a workaround, and must use VGA=std, hence I suggest we re-enable
  bochs-drm for Bionic.

  [Fix]

  I noticed on Focal, if you boot, the framebuffer is initially efifb:

  [ 0.603716] efifb: probing for efifb
  [ 0.603733] efifb: framebuffer at 0xc000, using 1876k, total 1875k
  [ 0.603735] efifb: mode is 800x600x32, linelength=3200, pages=1
  [ 0.603736] efifb: scrolling: redraw
  [ 0.603738] efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0
  [ 0.604462] Console: switching to colour frame buffer device 100x37
  [ 0.605829] fb0: EFI VGA frame buffer device

  This soon changes to bochs-drm about a second later:

  [ 0.935988] bochs-drm :00:01.0: remove_conflicting_pci_framebuffers: bar 
0: 0xc000 -> 0xc0ff
  [ 0.937023] bochs-drm :00:01.0: remove_conflicting_pci_framebuffers: bar 
2: 0xc1c8c000 -> 0xc1c8cfff
  [ 0.937776] checking generic (c000 1d5000) vs hw (c000 100)
  [ 0.937776] fb0: switching to bochsdrmfb from EFI VGA
  [ 0.939085] Console: switching to colour dummy device 80x25
  [ 0.939117] bochs-drm :00:01.0: vgaarb: deactivate vga console
  [ 0.939210] [drm] Found bochs VGA, ID 0xb0c5.
  [ 0.939212] [drm] Framebuffer size 16384 kB @ 0xc000, mmio @ 0xc1c8c000.
  [ 0.941955] lpc_ich :00:1f.0: I/O space for GPIO uninitialized
  [ 0.942069] [TTM] Zone kernel: Available graphics memory: 2006780 KiB
  [ 0.942081] [TTM] Initializing pool allocator
  [ 0.942089] [TTM] Initializing DMA 

[Touch-packages] [Bug 1876230] Re: liburcu: Enable MEMBARRIER_CMD_PRIVATE_EXPEDITED to address performance problems with MEMBARRIER_CMD_SHARED

2020-05-05 Thread Matthew Ruffell
Answering question 2. I have done a comprehensive performance analysis
based on the benchmark application.

Note: The SRU changes how the sys_membarrier syscall is used. The
implementation that we want to change to in this SRU never blocks, while
the previous implementation does. This makes performance analysis
entirely workload dependant. On busy servers with lots of background
processes, sys_membarrier will block more often, compared to quiet
servers with no background processes.

The following is based on a quiet server with no background processes.

Test parameters
===
Ubuntu 18.04.4
KVM, 2 vcpus
0.10.1 liburcu
4.15.0-99-generic
Test program "test_urcu[_bp]": http://paste.ubuntu.com/p/5vXVycQjYk/
(only difference is #include  or #include )

No changes to source code
=

ubuntu@ubuntu:~/userspace-rcu/tests/benchmark$ ./test_urcu 6 2 10
nr_reads   6065490002 nr_writes  237 nr_ops   6065490239
nr_reads   6476219475 nr_writes  186 nr_ops   6476219661
nr_reads   6474789528 nr_writes  183 nr_ops   6474789711
nr_reads   6476326433 nr_writes  188 nr_ops   6476326621
nr_reads   6479298142 nr_writes  179 nr_ops   6479298321
nr_reads   6476429569 nr_writes  186 nr_ops   6476429755
nr_reads   6478019994 nr_writes  191 nr_ops   6478020185
nr_reads   6479117595 nr_writes  183 nr_ops   6479117778
nr_reads   6478302181 nr_writes  185 nr_ops   6478302366
nr_reads   6481003399 nr_writes  191 nr_ops   6481003590

ubuntu@ubuntu:~/userspace-rcu/tests/benchmark$ ./test_urcu_bp 6 2 10
nr_reads644339902 nr_writes  485 nr_ops644340387
nr_reads644092800 nr_writes 1101 nr_ops644093901
nr_reads644676446 nr_writes  494 nr_ops644676940
nr_reads643845915 nr_writes  500 nr_ops643846415
nr_reads645156053 nr_writes  502 nr_ops645156555
nr_reads644626421 nr_writes  497 nr_ops644626918
nr_reads644710679 nr_writes  495 nr_ops644711174
nr_reads65530 nr_writes  503 nr_ops66033
nr_reads645150707 nr_writes  497 nr_ops645151204
nr_reads643681268 nr_writes  496 nr_ops643681764

Commits c0bb9f and 374530 patched in


ubuntu@ubuntu:~/userspace-rcu/tests/benchmark$ ./test_urcu 6 2 10
nr_reads   4097663510 nr_writes 6516 nr_ops   4097670026
nr_reads   4177088332 nr_writes 4183 nr_ops   4177092515
nr_reads   4153780077 nr_writes 1907 nr_ops   4153781984
nr_reads   4150954044 nr_writes 3942 nr_ops   4150957986
nr_reads   4267855073 nr_writes 2102 nr_ops   4267857175
nr_reads   4131310825 nr_writes 7119 nr_ops   4131317944
nr_reads   4183771431 nr_writes 1919 nr_ops   4183773350
nr_reads   4270944170 nr_writes 4958 nr_ops   4270949128
nr_reads   4123277225 nr_writes 4228 nr_ops   4123281453
nr_reads   4266997284 nr_writes 1723 nr_ops   4266999007


ubuntu@ubuntu:~/userspace-rcu/tests/benchmark$ ./test_urcu_bp 6 2 10
nr_reads   6530208343 nr_writes 8860 nr_ops   6530217203
nr_reads   6514357222 nr_writes10568 nr_ops   6514367790
nr_reads   6517420660 nr_writes 9534 nr_ops   6517430194
nr_reads   6510005433 nr_writes11799 nr_ops   6510017232
nr_reads   6492226563 nr_writes12517 nr_ops   6492239080
nr_reads   6532405460 nr_writes 6548 nr_ops   6532412008
nr_reads   6514205150 nr_writes 9686 nr_ops   6514214836
nr_reads   6481643486 nr_writes16167 nr_ops   6481659653
nr_reads   6509268022 nr_writes10582 nr_ops   6509278604
nr_reads   6523168701 nr_writes 9066 nr_ops   6523177767


Comparing and contrasting with 20.04:
=

Test Parameters:

Ubuntu 20.04 LTS
KVM, 2 vcpus
0.11.1 liburcu
5.4.0-29-generic

ubuntu@ubuntu:~/userspace-rcu/tests/benchmark$ ./test_urcu 6 2 10
nr_reads   4270089636 nr_writes 1638 nr_ops   4270091274
nr_reads   4281598850 nr_writes 3008 nr_ops   4281601858
nr_reads   4241230576 nr_writes 3612 nr_ops   4241234188
nr_reads   4230643208 nr_writes 5367 nr_ops   4230648575
nr_reads   4333495124 nr_writes 1354 nr_ops   4333496478
nr_reads   4291295097 nr_writes 3545 nr_ops   4291298642
nr_reads   4232582737 nr_writes 1983 nr_ops   4232584720
nr_reads   4268926719 nr_writes 3363 nr_ops   4268930082
nr_reads   4266736459 nr_writes 4881 nr_ops   4266741340
nr_reads   4313525276 nr_writes 4549 nr_ops   4313529825

ubuntu@ubuntu:~/userspace-rcu/tests/benchmark$ ./test_urcu_bp 6 2 10
nr_reads   6848011482 nr_writes 3171 nr_ops   6848014653
nr_reads   6842990129 nr_writes 4577 nr_ops   6842994706
nr_reads   6862298832 nr_writes 2875 nr_ops   6862301707
nr_reads   6849848255 nr_writes 4292 nr_ops   6849852547

[Touch-packages] [Bug 1876230] Re: liburcu: Enable MEMBARRIER_CMD_PRIVATE_EXPEDITED to address performance problems with MEMBARRIER_CMD_SHARED

2020-05-05 Thread Matthew Ruffell
To answer question 1, I went and checked every rdepends package:

gdnsd: dynamically links to -lurcu-qsbr
$ ldd /usr/sbin/gdnsd
liburcu-qsbr.so.6 => /usr/lib/x86_64-linux-gnu/liburcu-qsbr.so.6 
(0x7f33bfde5000)

glusterfs: The only package I am not entirely sure about. Only glusterd uses 
urcu:
ubuntu@ubuntu:~/glusterfs-3.13.2$ grep -Rin " header file. */
./config.h.in:203:/* Define to 1 if you have the  header file. */
./xlators/mgmt/glusterd/src/glusterd-rcu.h:14:#include 
./xlators/mgmt/glusterd/src/glusterd-rcu.h:15:#include 
./xlators/mgmt/glusterd/src/glusterd-rcu.h:16:#include 
./xlators/mgmt/glusterd/src/glusterd-rcu.h:17:#include 
./xlators/mgmt/glusterd/src/glusterd-rcu.h:18:#include 
./xlators/mgmt/glusterd/src/glusterd-conn-helper.c:15:#include 
$ ldd /usr/sbin/glusterd | grep urcu

No mention of static linking either:
ubuntu@ubuntu:~/glusterfs-3.13.2$ grep -Rin "\.a" . | grep urcu

The library linker settings are in ./configure.ac:
dnl Check for userspace-rcu
PKG_CHECK_MODULES([URCU], [liburcu-bp], [],
  [AC_CHECK_HEADERS([urcu-bp.h],
 [URCU_LIBS='-lurcu-bp'],
 AC_MSG_ERROR([liburcu-bp not found]))])
PKG_CHECK_MODULES([URCU_CDS], [liburcu-cds >= 0.8], [],
  [PKG_CHECK_MODULES([URCU_CDS], [liburcu-cds >= 0.7],
[AC_DEFINE(URCU_OLD, 1, [Define if liburcu 0.6 or 0.7 is found])],
[AC_CHECK_HEADERS([urcu/cds.h],
  [AC_DEFINE(URCU_OLD, 1, [Define if liburcu 0.6 or 0.7 is found])
   URCU_CDS_LIBS='-lurcu-cds'],
  [AC_MSG_ERROR([liburcu-cds not found])])])])
I ran ldd over all glusterfs binaries which are listed by dpkg -L, but they all 
came back negative for urcu. From what I can tell, glusterd from the xlators 
directory is either not built, or does not link against urcu in Ubuntu.

knot: dynamically links to -lurcu
ubuntu@ubuntu:~$ ldd /usr/sbin/knotd
liburcu.so.6 => /usr/lib/x86_64-linux-gnu/liburcu.so.6 (0x7fc9d1e8b000)

lttng: dynamically links to various urcu libraries
$ ldd /usr/bin/lttng
liburcu-common.so.6 => /usr/lib/x86_64-linux-gnu/liburcu-common.so.6 
(0x7f6711a2f000)
liburcu.so.6 => /usr/lib/x86_64-linux-gnu/liburcu.so.6 (0x7f6711827000)
liburcu-cds.so.6 => /usr/lib/x86_64-linux-gnu/liburcu-cds.so.6 
(0x7f671161d000)

multipath-tools: dynamically links to -lurcu
$ ldd /sbin/multipath
liburcu.so.6 => /usr/lib/x86_64-linux-gnu/liburcu.so.6 (0x7fe63ace1000)

netsniff-ng: dynamically links to -lurcu
~/netsniff-ng-0.6.4$ grep -Rin "urcu"
flowtop/Makefile:1:flowtop-libs =   -lurcu \
ui.h:7:#include 
flowtop.c:28:#include 
flowtop.c:29:#include 
flowtop.c:30:#include 
INSTALL:28: - liburcu:flowtop
$ ldd /usr/sbin/flowtop
liburcu.so.6 => /usr/lib/x86_64-linux-gnu/liburcu.so.6 (0x7f7561a5a000)

sheepdog: this program doesn't link against urcu at all! It only uses the 
 header file and not anything more.
search for urcu in sheepdog: https://paste.ubuntu.com/p/VPKr4pWtQg/
explanation found in d/changelog: https://paste.ubuntu.com/p/K8XwjK2czD/

ust: dynamically links to various urcu libraries
$ ldd /usr/lib/x86_64-linux-gnu/liblttng-ust-cyg-profile-fast.so.0.0.0
liburcu-bp.so.6 => /usr/lib/x86_64-linux-gnu/liburcu-bp.so.6 
(0x7f91b0c13000)
liburcu-cds.so.6 => /usr/lib/x86_64-linux-gnu/liburcu-cds.so.6 
(0x7f91b01f4000)

>From what I can tell, no rdepends package statically links to liburcu,
and they all use dynamic linking, meaning no packages will need to be
rebuilt for this SRU.

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to liburcu in Ubuntu.
https://bugs.launchpad.net/bugs/1876230

Title:
  liburcu: Enable MEMBARRIER_CMD_PRIVATE_EXPEDITED to address
  performance problems with MEMBARRIER_CMD_SHARED

Status in liburcu package in Ubuntu:
  Fix Released
Status in liburcu source package in Bionic:
  In Progress

Bug description:
  [Impact]

  In Linux 4.3, a new syscall was defined, called "membarrier". This
  systemcall was defined specifically for use in userspace-rcu (liburcu)
  to speed up the fast path / reader side of the library. The original
  implementation in Linux 4.3 only supported the MEMBARRIER_CMD_SHARED
  subcommand of the membarrier syscall.

  MEMBARRIER_CMD_SHARED executes a memory barrier on all threads from
  all processes running on the system. When it exits, the userspace
  thread which called it is guaranteed that all running threads share
  the same world view in regards to userspace addresses which are
  consumed by readers and writers.

  The problem with MEMBARRIER_CMD_SHARED is system calls made in this
  fashion can block, since it deploys a barrier across all threads in a
  system, and some other threads can be waiting on blocking operations,
  and take time to reach the barrier.

  In Linux 4.14, this was addressed by adding the
  MEMBARRIER_CMD_PRIVATE_EXPEDITED command to the membarrier syscall. It
  only targets threads which share the same mm as the thread calling the
  membarrier syscall, aka, threads in 

[Touch-packages] [Bug 1872863] Re: QEMU/KVM display is garbled when booting from kernel EFI stub due to missing bochs-drm module

2020-05-02 Thread Matthew Ruffell
For both tests I used the kmod package from my test ppa, since it is still
in the unapproved upload queue.

Firstly, I installed 4.15.0-99-generic from -updates, and booted my EFI based
KVM machine with VGA=std. The screen was garbled, as the framebuffer used was
efifb.

I then installed 4.15.0-100-generic from -proposed, and rebooted, keeping the
same settings. The screen was perfect and readable. The issue was fixed. I 
looked
at lsmod, and bochs-drm was loaded. 

$ lsmod | grep bochs_drm
bochs_drm  20480  1
ttm   106496  1 bochs_drm
drm_kms_helper172032  1 bochs_drm
drm   401408  4 drm_kms_helper,bochs_drm,ttm

Dmesg shows:

[1.082068] checking generic (c000 12c000) vs hw (c000 100)
[1.082069] fb: switching to bochsdrmfb from EFI VGA
[1.082468] Console: switching to colour dummy device 80x25
[1.083148] [drm] Found bochs VGA, ID 0xb0c5.
[1.083151] [drm] Framebuffer size 16384 kB @ 0xc000, mmio @ 0xc1c8c000.
[1.083386] [TTM] Zone  kernel: Available graphics memory: 4026194 kiB
[1.083389] [TTM] Zone   dma32: Available graphics memory: 2097152 kiB
[1.083390] [TTM] Initializing pool allocator
[1.083394] [TTM] Initializing DMA pool allocator
[1.085265] fbcon: bochsdrmfb (fb0) is primary device
[1.087421] Console: switching to colour frame buffer device 128x48
[1.091589] bochs-drm :00:01.0: fb0: bochsdrmfb frame buffer device
[1.108664] [drm] Initialized bochs-drm 1.0.0 20130925 for :00:01.0 on 
minor 0

Attached is a screenshot showing a good screen. I am happy to mark this as
verified for the linux package.

** Attachment added: "screenshot of vga=std with 4.15.0-100-generic"
   
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1872863/+attachment/5365529/+files/Screenshot%20from%202020-05-03%2013-25-59.png

** Tags removed: verification-needed-bionic
** Tags added: verification-done-bionic

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to kmod in Ubuntu.
https://bugs.launchpad.net/bugs/1872863

Title:
  QEMU/KVM display is garbled when booting from kernel EFI stub due to
  missing bochs-drm module

Status in kmod package in Ubuntu:
  Fix Released
Status in linux package in Ubuntu:
  Fix Released
Status in kmod source package in Bionic:
  In Progress
Status in linux source package in Bionic:
  Fix Committed

Bug description:
  BugLink: https://bugs.launchpad.net/bugs/1872863

  [Impact]

  A recent grub2 SRU, LP #1864533, now forces the kernel to boot via the
  kernel EFI stub whenever EFI is enabled. This causes problems for
  QEMU/KVM virtual machines which use the VGA=std video device, as the
  efifb driver yields an unreadable garbled screen. See the attached
  image.

  The correct framebuffer driver to use in this situation is bochs-drm,
  and modprobing it from a HWE kernel fixes the issues.

  bochs-drm is missing from Bionic since CONFIG_DRM_BOCHS was disabled
  in LP #1378648 due to bochs-drm causing problems in a PowerKVM
  machine. This problem appears to be fixed now, and bochs-drm has been
  re-enabled for Disco and up, in LP #1795857 and has been proven safe.

  This has also come up again in LP #1823152, as well as chatter on LP
  #1795857 to get this enabled on Bionic.

  The customer which is experiencing this issue cannot switch to VGA=qxl
  as a workaround, and must use VGA=std, hence I suggest we re-enable
  bochs-drm for Bionic.

  [Fix]

  I noticed on Focal, if you boot, the framebuffer is initially efifb:

  [ 0.603716] efifb: probing for efifb
  [ 0.603733] efifb: framebuffer at 0xc000, using 1876k, total 1875k
  [ 0.603735] efifb: mode is 800x600x32, linelength=3200, pages=1
  [ 0.603736] efifb: scrolling: redraw
  [ 0.603738] efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0
  [ 0.604462] Console: switching to colour frame buffer device 100x37
  [ 0.605829] fb0: EFI VGA frame buffer device

  This soon changes to bochs-drm about a second later:

  [ 0.935988] bochs-drm :00:01.0: remove_conflicting_pci_framebuffers: bar 
0: 0xc000 -> 0xc0ff
  [ 0.937023] bochs-drm :00:01.0: remove_conflicting_pci_framebuffers: bar 
2: 0xc1c8c000 -> 0xc1c8cfff
  [ 0.937776] checking generic (c000 1d5000) vs hw (c000 100)
  [ 0.937776] fb0: switching to bochsdrmfb from EFI VGA
  [ 0.939085] Console: switching to colour dummy device 80x25
  [ 0.939117] bochs-drm :00:01.0: vgaarb: deactivate vga console
  [ 0.939210] [drm] Found bochs VGA, ID 0xb0c5.
  [ 0.939212] [drm] Framebuffer size 16384 kB @ 0xc000, mmio @ 0xc1c8c000.
  [ 0.941955] lpc_ich :00:1f.0: I/O space for GPIO uninitialized
  [ 0.942069] [TTM] Zone kernel: Available graphics memory: 2006780 KiB
  [ 0.942081] [TTM] Initializing pool allocator
  [ 0.942089] [TTM] Initializing DMA pool allocator
  [ 0.943026] virtio_blk virtio2: [vda] 20971520 512-byte logical blocks (10.7 
GB/10.0 GiB)
  [ 0.944019] 

[Touch-packages] [Bug 1876230] Re: liburcu: Enable MEMBARRIER_CMD_PRIVATE_EXPEDITED to address performance problems with MEMBARRIER_CMD_SHARED

2020-04-30 Thread Matthew Ruffell
Attached is a debdiff for Bionic

** Patch added: "liburcu debdiff for Bionic"
   
https://bugs.launchpad.net/ubuntu/+source/liburcu/+bug/1876230/+attachment/5364290/+files/lp1876230_bionic.debdiff

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to liburcu in Ubuntu.
https://bugs.launchpad.net/bugs/1876230

Title:
  liburcu: Enable MEMBARRIER_CMD_PRIVATE_EXPEDITED to address
  performance problems with MEMBARRIER_CMD_SHARED

Status in liburcu package in Ubuntu:
  Fix Released
Status in liburcu source package in Bionic:
  In Progress

Bug description:
  [Impact]

  In Linux 4.3, a new syscall was defined, called "membarrier". This
  systemcall was defined specifically for use in userspace-rcu (liburcu)
  to speed up the fast path / reader side of the library. The original
  implementation in Linux 4.3 only supported the MEMBARRIER_CMD_SHARED
  subcommand of the membarrier syscall.

  MEMBARRIER_CMD_SHARED executes a memory barrier on all threads from
  all processes running on the system. When it exits, the userspace
  thread which called it is guaranteed that all running threads share
  the same world view in regards to userspace addresses which are
  consumed by readers and writers.

  The problem with MEMBARRIER_CMD_SHARED is system calls made in this
  fashion can block, since it deploys a barrier across all threads in a
  system, and some other threads can be waiting on blocking operations,
  and take time to reach the barrier.

  In Linux 4.14, this was addressed by adding the
  MEMBARRIER_CMD_PRIVATE_EXPEDITED command to the membarrier syscall. It
  only targets threads which share the same mm as the thread calling the
  membarrier syscall, aka, threads in the current process, and not all
  threads / processes in the system.

  Calls to membarrier with the MEMBARRIER_CMD_PRIVATE_EXPEDITED command
  are guaranteed non-blocking, due to using inter-processor interrupts
  to implement memory barriers.

  Because of this, membarrier calls that use
  MEMBARRIER_CMD_PRIVATE_EXPEDITED are much faster than those that use
  MEMBARRIER_CMD_SHARED.

  Since Bionic uses a 4.15 kernel, all kernel requirements are met, and
  this SRU is to enable support for MEMBARRIER_CMD_PRIVATE_EXPEDITED in
  the liburcu package.

  This brings the performance of the liburcu library back in line to
  where it was in Trusty, as this particular user has performance
  problems upon upgrading from Trusty to Bionic.

  [Test]

  Testing performance is heavily dependant on the application which
  links against liburcu, and the workload which it executes.

  A test package is available in the following ppa:
  https://launchpad.net/~mruffell/+archive/ubuntu/sf276198-test

  For the sake of testing, we can use the benchmarks provided in the
  liburcu source code. Download a copy of the source code for liburcu
  either from the repos or from github:

  $ pull-lp-source liburcu bionic
  # OR
  $ git clone https://github.com/urcu/userspace-rcu.git
  $ git checkout v0.10.1 # version in bionic

  Build the code:

  $ ./bootstrap
  $ ./configure
  $ make

  Go into the tests/benchmark directory

  $ cd tests/benchmark

  From there, you can run benchmarks for the four main usages of
  liburcu: urcu, urcu-bp, urcu-signal and urcu-mb.

  On a 8 core machine, 6 threads for readers and 2 threads for writers,
  with a 10 second runtime, execute:

  $ ./test_urcu 6 2 10
  $ ./test_urcu_bp 6 2 10
  $ ./test_urcu_signal 6 2 10
  $ ./test_urcu_mb 6 2 10

  Results:

  ./test_urcu 6 2 10
  0.10.1-1: 17612527667 reads, 268 writes, 17612527935 ops
  0.10.1-1ubuntu1: 14988437247 reads, 810069 writes, 14989247316 ops

  $ ./test_urcu_bp 6 2 10
  0.10.1-1: 1177891079 reads, 1699523 writes, 1179590602 ops
  0.10.1-1ubuntu1: 13230354737 reads, 575314 writes, 13230930051 ops

  $ ./test_urcu_signal 6 2 10
  0.10.1-1: 20128392417 reads, 6859 writes, 20128399276 ops
  0.10.1-1ubuntu1: 20501430707 reads, 6890 writes, 20501437597 ops

  $ ./test_urcu_mb 6 2 10
  0.10.1-1: 627996563 reads, 5409563 writes, 633406126 ops
  0.10.1-1ubuntu1: 653194752 reads, 4590020 writes, 657784772 ops

  The SRU only changes behaviour for urcu and urcu-bp, since they are
  the only "flavours" of liburcu which the patches change. From a pure
  ops standpoint:

  $ ./test_urcu 6 2 10
  17612527935 ops
  14989247316 ops

  $ ./test_urcu_bp 6 2 10
  1179590602 ops
  13230930051 ops

  We see that this particular benchmark workload, test_urcu sees extra
  performance overhead with MEMBARRIER_CMD_PRIVATE_EXPEDITED, which is
  explained by the extra impact that it has on the slowpath, and the
  extra amount of writes it did during my benchmark.

  The real winner in this benchmark workload is test_urcu_bp, which sees
  a 10x performance increase with MEMBARRIER_CMD_PRIVATE_EXPEDITED. Some
  of this may be down to the 3x less writes it did during my benchmark.

  Again, these benchmarks are indicative only are very 

[Touch-packages] [Bug 1876230] [NEW] liburcu: Enable MEMBARRIER_CMD_PRIVATE_EXPEDITED to address performance problems with MEMBARRIER_CMD_SHARED

2020-04-30 Thread Matthew Ruffell
the Bionic source code, and
"make regtest", "make short_bench" and "make long_bench" pass. You want
to run these on a cloud instance somewhere since they take multiple
hours.

If a regression were to occur, applications linked against -lurcu and
-lurcu-bp would be affected. The homepage: https://liburcu.org/ offers a
list of the major applications that use liburcu: Knot DNS, Netsniff-ng,
Sheepdog, GlusterFS, gdnsd and LTTng.

[Other]

The two commits which are being SRU'd are:

commit c0bb9f693f926595a7cb8b4ce712cef08d9f5d49
Author: Mathieu Desnoyers 
Date: Thu Dec 21 13:42:23 2017 -0500
Subject: liburcu: Use membarrier private expedited when available
Link: 
https://github.com/urcu/userspace-rcu/commit/c0bb9f693f926595a7cb8b4ce712cef08d9f5d49

commit 3745305bf09e7825e75ee5b5490347ee67c6efdd
Author: Mathieu Desnoyers 
Date: Fri Dec 22 10:57:59 2017 -0500
Subject: liburcu-bp: Use membarrier private expedited when available
Link: 
https://github.com/urcu/userspace-rcu/commit/3745305bf09e7825e75ee5b5490347ee67c6efdd

Both cherry pick directly onto 0.10.1 in Bionic, and are originally from
0.11.0, meaning that Eoan, Focal and Groovy already have the patch.

If you are interested in how the membarrier syscall works, you can read
their commits in the Linux kernel:

commit 5b25b13ab08f616efd566347d809b4ece54570d1
Author: Mathieu Desnoyers 
Date:   Fri Sep 11 13:07:39 2015 -0700
Subject: sys_membarrier(): system-wide memory barrier (generic, x86)
Link: 
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=5b25b13ab08f616efd566347d809b4ece54570d1

commit 22e4ebb975822833b083533035233d128b30e98f
Author: Mathieu Desnoyers 
Date:   Fri Jul 28 16:40:40 2017 -0400
Subject: membarrier: Provide expedited private command
Link: 
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=22e4ebb975822833b083533035233d128b30e98f

Additionally, blog posts from LTTng:
https://lttng.org/blog/2018/01/15/membarrier-system-call-performance-and-userspace-rcu/

And Phoronix:
https://www.phoronix.com/scan.php?page=news_item=URCU-Membarrier-Performance

** Affects: liburcu (Ubuntu)
 Importance: Undecided
 Status: Fix Released

** Affects: liburcu (Ubuntu Bionic)
 Importance: Medium
 Assignee: Matthew Ruffell (mruffell)
 Status: In Progress


** Tags: sts

** Also affects: liburcu (Ubuntu Bionic)
   Importance: Undecided
   Status: New

** Changed in: liburcu (Ubuntu)
   Status: New => Fix Released

** Changed in: liburcu (Ubuntu Bionic)
   Status: New => In Progress

** Changed in: liburcu (Ubuntu Bionic)
   Importance: Undecided => Medium

** Changed in: liburcu (Ubuntu Bionic)
 Assignee: (unassigned) => Matthew Ruffell (mruffell)

** Tags added: sts

** Description changed:

  [Impact]
  
  In Linux 4.3, a new syscall was defined, called "membarrier". This
  systemcall was defined specifically for use in userspace-rcu (liburcu)
  to speed up the fast path / reader side of the library. The original
  implementation in Linux 4.3 only supported the MEMBARRIER_CMD_SHARED
  subcommand of the membarrier syscall.
  
  MEMBARRIER_CMD_SHARED executes a memory barrier on all threads from all
  processes running on the system. When it exits, the userspace thread
  which called it is guaranteed that all running threads share the same
  world view in regards to userspace addresses which are consumed by
  readers and writers.
  
  The problem with MEMBARRIER_CMD_SHARED is system calls made in this
  fashion can block, since it deploys a barrier across all threads in a
  system, and some other threads can be waiting on blocking operations,
  and take time to reach the barrier.
  
  In Linux 4.14, this was addressed by adding the
  MEMBARRIER_CMD_PRIVATE_EXPEDITED command to the membarrier syscall. It
  only targets threads which share the same mm as the thread calling the
  membarrier syscall, aka, threads in the current process, and not all
  threads / processes in the system.
  
  Calls to membarrier with the MEMBARRIER_CMD_PRIVATE_EXPEDITED command
  are guaranteed non-blocking, due to using inter-processor interrupts to
  implement memory barriers.
  
  Because of this, membarrier calls that use
  MEMBARRIER_CMD_PRIVATE_EXPEDITED are much faster than those that use
  MEMBARRIER_CMD_SHARED.
  
  Since Bionic uses a 4.15 kernel, all kernel requirements are met, and
  this SRU is to enable support for MEMBARRIER_CMD_PRIVATE_EXPEDITED in
  the liburcu package.
  
  This brings the performance of the liburcu library back in line to where
  it was in Trusty, as this particular user has performance problems upon
  upgrading from Trusty to Bionic.
  
  [Test]
  
  Testing performance is heavily dependant on the application which links
  against liburcu, and the workload which it executes.
+ 
+ A test package is available in the following ppa:
+ https://launchpad.net/~mruffell/+archive/ubuntu/sf276198-test
  
  For

[Touch-packages] [Bug 1872863] Re: QEMU/KVM display is garbled when booting from kernel EFI stub due to missing bochs-drm module

2020-04-22 Thread Matthew Ruffell
** Description changed:

  BugLink: https://bugs.launchpad.net/bugs/1872863
  
  [Impact]
  
  A recent grub2 SRU, LP #1864533, now forces the kernel to boot via the
  kernel EFI stub whenever EFI is enabled. This causes problems for
  QEMU/KVM virtual machines which use the VGA=std video device, as the
  efifb driver yields an unreadable garbled screen. See the attached
  image.
  
  The correct framebuffer driver to use in this situation is bochs-drm,
  and modprobing it from a HWE kernel fixes the issues.
  
  bochs-drm is missing from Bionic since CONFIG_DRM_BOCHS was disabled in
  LP #1378648 due to bochs-drm causing problems in a PowerKVM machine.
  This problem appears to be fixed now, and bochs-drm has been re-enabled
  for Disco and up, in LP #1795857 and has been proven safe.
  
  This has also come up again in LP #1823152, as well as chatter on LP
  #1795857 to get this enabled on Bionic.
  
  The customer which is experiencing this issue cannot switch to VGA=qxl
  as a workaround, and must use VGA=std, hence I suggest we re-enable
  bochs-drm for Bionic.
  
  [Fix]
  
  I noticed on Focal, if you boot, the framebuffer is initially efifb:
  
  [ 0.603716] efifb: probing for efifb
  [ 0.603733] efifb: framebuffer at 0xc000, using 1876k, total 1875k
  [ 0.603735] efifb: mode is 800x600x32, linelength=3200, pages=1
  [ 0.603736] efifb: scrolling: redraw
  [ 0.603738] efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0
  [ 0.604462] Console: switching to colour frame buffer device 100x37
  [ 0.605829] fb0: EFI VGA frame buffer device
  
  This soon changes to bochs-drm about a second later:
  
  [ 0.935988] bochs-drm :00:01.0: remove_conflicting_pci_framebuffers: bar 
0: 0xc000 -> 0xc0ff
  [ 0.937023] bochs-drm :00:01.0: remove_conflicting_pci_framebuffers: bar 
2: 0xc1c8c000 -> 0xc1c8cfff
  [ 0.937776] checking generic (c000 1d5000) vs hw (c000 100)
  [ 0.937776] fb0: switching to bochsdrmfb from EFI VGA
  [ 0.939085] Console: switching to colour dummy device 80x25
  [ 0.939117] bochs-drm :00:01.0: vgaarb: deactivate vga console
  [ 0.939210] [drm] Found bochs VGA, ID 0xb0c5.
  [ 0.939212] [drm] Framebuffer size 16384 kB @ 0xc000, mmio @ 0xc1c8c000.
  [ 0.941955] lpc_ich :00:1f.0: I/O space for GPIO uninitialized
  [ 0.942069] [TTM] Zone kernel: Available graphics memory: 2006780 KiB
  [ 0.942081] [TTM] Initializing pool allocator
  [ 0.942089] [TTM] Initializing DMA pool allocator
  [ 0.943026] virtio_blk virtio2: [vda] 20971520 512-byte logical blocks (10.7 
GB/10.0 GiB)
  [ 0.944019] [drm] Found EDID data blob.
  [ 0.944162] [drm] Initialized bochs-drm 1.0.0 20130925 for :00:01.0 on 
minor 0
  [ 0.944979] fbcon: bochs-drmdrmfb (fb0) is primary device
  [ 0.947712] Console: switching to colour frame buffer device 128x48
  
  On bionic, the framebuffer never changes from efifb, since the bochs-drm
  kernel module is not built, and it is also present on the module banlist
  in /etc/modprobe.d/blacklist-framebuffer.conf
  
  bochs-drm needs to be enabled to be built in the kernel config, and
  removed from the module blacklist in kmod.
  
  [Testcase]
  
  Create a new QEMU/KVM virtual machine, I used virt-manager. Before you
  install the OS, check the box to modify settings before install. In the
  "Overview" tab, enable EFI by setting the firmware to "UEFI x86_64:
  /usr/share/OVMF/OVMF_CODE.secboot.fd".
  
  Set the video device to qxl while you install Bionic. Once the install
  is done, reboot, do a "apt update" and "apt upgrade", to ensure you have
  grub2 2.02-2ubuntu8.15 installed.
  
  Shut the VM down, and set the video device to VGA. Or VGA=std if you use
  the QEMU command line.
  
  Start the VM up, and the screen will be garbled. See attached picture.
  
  If you install my test packages, which are available here:
  
  https://launchpad.net/~mruffell/+archive/ubuntu/sf272653-test
  
  Instructions to install (on a bionic system):
  
  1) sudo add-apt-repository ppa:mruffell/sf272653-test
  2) sudo apt-get update
  3) sudo apt install linux-image-unsigned-4.15.0-96-generic 
linux-modules-4.15.0-96-generic linux-modules-extra-4.15.0-96-generic 
linux-headers-4.15.0-96-generic linux-headers-4.15.0-96 libkmod2 kmod
  4) sudo reboot
  5) uname -rv
  4.15.0-96-generic #97+TEST272653v20200409b1-Ubuntu SMP Thu Apr 9 04:09:18 UTC 
2020
  
  If you reboot, the screen will be perfectly readable, since the bochs-
  drm driver will be in use.
  
  [Regression Potential]
  
  We are enabling a display driver on Bionic which was previously
  disabled, so there will be some virtual machines making the jump to
  using bochs-drm since it would now be available.
  
  I don't think that this is a bad thing, and in many ways will probably
  end up improving some user's experience. Although there are two
  scenarios where a regression could happen:
  
  1 - frequently gamers use GPU passthrough to their kvm based virtual
  machines, and I have seen them expect, 

[Touch-packages] [Bug 1872863] Re: QEMU/KVM display is garbled when booting from kernel EFI stub due to missing bochs-drm module

2020-04-22 Thread Matthew Ruffell
** Description changed:

  BugLink: https://bugs.launchpad.net/bugs/1872863
  
  [Impact]
  
  A recent grub2 SRU, LP #1864533, now forces the kernel to boot via the
  kernel EFI stub whenever EFI is enabled. This causes problems for
  QEMU/KVM virtual machines which use the VGA=std video device, as the
  efifb driver yields an unreadable garbled screen. See the attached
  image.
  
  The correct framebuffer driver to use in this situation is bochs-drm,
  and modprobing it from a HWE kernel fixes the issues.
  
  bochs-drm is missing from Bionic since CONFIG_DRM_BOCHS was disabled in
  LP #1378648 due to bochs-drm causing problems in a PowerKVM machine.
  This problem appears to be fixed now, and bochs-drm has been re-enabled
  for Disco and up, in LP #1795857 and has been proven safe.
  
  This has also come up again in LP #1823152, as well as chatter on LP
  #1795857 to get this enabled on Bionic.
  
  The customer which is experiencing this issue cannot switch to VGA=qxl
  as a workaround, and must use VGA=std, hence I suggest we re-enable
  bochs-drm for Bionic.
  
  [Fix]
  
  I noticed on Focal, if you boot, the framebuffer is initially efifb:
  
  [ 0.603716] efifb: probing for efifb
  [ 0.603733] efifb: framebuffer at 0xc000, using 1876k, total 1875k
  [ 0.603735] efifb: mode is 800x600x32, linelength=3200, pages=1
  [ 0.603736] efifb: scrolling: redraw
  [ 0.603738] efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0
  [ 0.604462] Console: switching to colour frame buffer device 100x37
  [ 0.605829] fb0: EFI VGA frame buffer device
  
  This soon changes to bochs-drm about a second later:
  
  [ 0.935988] bochs-drm :00:01.0: remove_conflicting_pci_framebuffers: bar 
0: 0xc000 -> 0xc0ff
  [ 0.937023] bochs-drm :00:01.0: remove_conflicting_pci_framebuffers: bar 
2: 0xc1c8c000 -> 0xc1c8cfff
  [ 0.937776] checking generic (c000 1d5000) vs hw (c000 100)
  [ 0.937776] fb0: switching to bochsdrmfb from EFI VGA
  [ 0.939085] Console: switching to colour dummy device 80x25
  [ 0.939117] bochs-drm :00:01.0: vgaarb: deactivate vga console
  [ 0.939210] [drm] Found bochs VGA, ID 0xb0c5.
  [ 0.939212] [drm] Framebuffer size 16384 kB @ 0xc000, mmio @ 0xc1c8c000.
  [ 0.941955] lpc_ich :00:1f.0: I/O space for GPIO uninitialized
  [ 0.942069] [TTM] Zone kernel: Available graphics memory: 2006780 KiB
  [ 0.942081] [TTM] Initializing pool allocator
  [ 0.942089] [TTM] Initializing DMA pool allocator
  [ 0.943026] virtio_blk virtio2: [vda] 20971520 512-byte logical blocks (10.7 
GB/10.0 GiB)
  [ 0.944019] [drm] Found EDID data blob.
  [ 0.944162] [drm] Initialized bochs-drm 1.0.0 20130925 for :00:01.0 on 
minor 0
  [ 0.944979] fbcon: bochs-drmdrmfb (fb0) is primary device
  [ 0.947712] Console: switching to colour frame buffer device 128x48
  
  On bionic, the framebuffer never changes from efifb, since the bochs-drm
  kernel module is not built, and it is also present on the module banlist
  in /etc/modprobe.d/blacklist-framebuffer.conf
  
  bochs-drm needs to be enabled to be built in the kernel config, and
  removed from the module blacklist in kmod.
  
  [Testcase]
  
  Create a new QEMU/KVM virtual machine, I used virt-manager. Before you
  install the OS, check the box to modify settings before install. In the
  "Overview" tab, enable EFI by setting the firmware to "UEFI x86_64:
  /usr/share/OVMF/OVMF_CODE.secboot.fd".
  
  Set the video device to qxl while you install Bionic. Once the install
  is done, reboot, do a "apt update" and "apt upgrade", to ensure you have
  grub2 2.02-2ubuntu8.15 installed.
  
  Shut the VM down, and set the video device to VGA. Or VGA=std if you use
  the QEMU command line.
  
  Start the VM up, and the screen will be garbled. See attached picture.
  
  If you install my test packages, which are available here:
  
  https://launchpad.net/~mruffell/+archive/ubuntu/sf272653-test
  
  Instructions to install (on a bionic system):
  
  1) sudo add-apt-repository ppa:mruffell/sf272653-test
  2) sudo apt-get update
  3) sudo apt install linux-image-unsigned-4.15.0-96-generic 
linux-modules-4.15.0-96-generic linux-modules-extra-4.15.0-96-generic 
linux-headers-4.15.0-96-generic linux-headers-4.15.0-96 libkmod2 kmod
  4) sudo reboot
  5) uname -rv
  4.15.0-96-generic #97+TEST272653v20200409b1-Ubuntu SMP Thu Apr 9 04:09:18 UTC 
2020
  
  If you reboot, the screen will be perfectly readable, since the bochs-
  drm driver will be in use.
  
  [Regression Potential]
  
  We are enabling a display driver on Bionic which was previously
  disabled, so there will be some virtual machines making the jump to
  using bochs-drm since it would now be available.
  
  I don't think that this is a bad thing, and in many ways will probably
  end up improving some user's experience. Although there are two
  scenarios where a regression could happen:
  
  1 - frequently gamers use GPU passthrough to their kvm based virtual
  machines, and I have seen them expect, 

[Touch-packages] [Bug 1872863] Re: QEMU/KVM display is garbled when booting from kernel EFI stub due to missing bochs-drm module

2020-04-21 Thread Matthew Ruffell
I have verified that enabling the bochs-drm kernel module on bionic does
not cause any regressions for PowerKVM machines when the video device is
VGA=std.

TLDR: See Screenshot:
https://launchpadlibrarian.net/475658598/Screenshot%20from%202020-04-22%2015-14-13.png

This is in response to the main cause of regression concern, LP
#1378648, which got bochs-drm disabled in the first place, on yakkety.

On Christian's advice, I created a ppc64el instance on Canonistack, with
the m1.medium flavor and c8c120bd-3d55-4e2d-8cc5-93039a62524f image
(ubuntu/ubuntu-bionic-18.04-ppc64el-server-20200407-disk1.img).

>From there, I installed the KVM packages and a xfce4 desktop, and got
xrdp working. I then installed virt-manager.

I modprobed the kvm_pr kernel module on my instance to enable nested kvm
on ppc64el.

>From there, I installed Ubuntu 18.04.4 Server ppc64el, using virt-
manager. The "Machine Type" was pseries-2.12, and the Video Model was
set to "VGA" [1].

[1]:
https://launchpadlibrarian.net/475658622/Screenshot%20from%202020-04-22%2015-14-04.png

Upon booting, the system was using the Open Firmware frame buffer:

[0.888072] Using unsupported 800x600 vga at 20008100, depth=32, 
pitch=3200
[0.890480] Console: switching to colour frame buffer device 100x37
[0.892181] fb0: Open Firmware frame buffer device on 
/pci@8002000/vga@6

$ cat /proc/fb
0 OFfb vga

Which is what LP #1378648 wanted.

I then enabled my ppa where my test packages were built:

https://launchpad.net/~mruffell/+archive/ubuntu/sf272653-test

and installed new kmod and 4.15.0-96-generic test builds with bochs-drm
enabled in the kernel config, and removed from the kmod blacklist.

I then rebooted, and found that system boots successfully, and does not
get hung up at the bug in LP #1378648, indicating that it has been fixed
before 4.15.

I checked "lsmod" and bochs-drm has been loaded, and looking at dmesg,
initially the Open Firmware frame buffer device is used, and later in
the boot, bochs-drm is loaded, and takes over the frame buffer.

See screenshot [2] for proof that bochs-drm is running and functions
correctly in virt-manager.

[2]:
https://launchpadlibrarian.net/475658598/Screenshot%20from%202020-04-22%2015-14-13.png

A complete dmesg from this boot can be found here [3].

[3] https://paste.ubuntu.com/p/Q8V7fKnRzz/

Interesting parts:

[0.893046] Using unsupported 800x600 vga at 20008100, depth=32, 
pitch=3200
[0.895482] Console: switching to colour frame buffer device 100x37
[0.897270] fb0: Open Firmware frame buffer device on 
/pci@8002000/vga@6
...
[2.041835] checking generic (20008100 1d4c00) vs hw (20008100 
100)
[2.041836] fb: switching to bochsdrmfb from OFfb vga
[2.042568] Console: switching to colour dummy device 80x25
[2.045581] [drm] Found bochs VGA, ID 0xb0c5.
[2.046100] [drm] Framebuffer size 16384 kB @ 0x20008100, mmio @ 
0x20008200.
[2.058141] [TTM] Zone  kernel: Available graphics memory: 504512 kiB
[2.058883] [TTM] Initializing pool allocator
[2.059372] [TTM] Initializing DMA pool allocator
[2.074890] Console: switching to colour frame buffer device 128x48
[2.089805] bochs-drm :00:06.0: fb0: bochsdrmfb frame buffer device
[2.090549] [drm] Initialized bochs-drm 1.0.0 20130925 for :00:06.0 on 
minor 0

$ cat /proc/fb
0 bochsdrmfb

With good dmesg outputs, and positive test results in virt-manager, I am
happy to say that enabling bochs-drm will not cause any regressions on
ppc64el, and with that, I hope it satisfies your request for testing.

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to kmod in Ubuntu.
https://bugs.launchpad.net/bugs/1872863

Title:
  QEMU/KVM display is garbled when booting from kernel EFI stub due to
  missing bochs-drm module

Status in kmod package in Ubuntu:
  Fix Released
Status in linux package in Ubuntu:
  Fix Released
Status in kmod source package in Bionic:
  In Progress
Status in linux source package in Bionic:
  In Progress

Bug description:
  BugLink: https://bugs.launchpad.net/bugs/1872863

  [Impact]

  A recent grub2 SRU, LP #1864533, now forces the kernel to boot via the
  kernel EFI stub whenever EFI is enabled. This causes problems for
  QEMU/KVM virtual machines which use the VGA=std video device, as the
  efifb driver yields an unreadable garbled screen. See the attached
  image.

  The correct framebuffer driver to use in this situation is bochs-drm,
  and modprobing it from a HWE kernel fixes the issues.

  bochs-drm is missing from Bionic since CONFIG_DRM_BOCHS was disabled
  in LP #1378648 due to bochs-drm causing problems in a PowerKVM
  machine. This problem appears to be fixed now, and bochs-drm has been
  re-enabled for Disco and up, in LP #1795857 and has been proven safe.

  This has also come up again in LP #1823152, as well as chatter on LP
  #1795857 to get this enabled on Bionic.

  

[Touch-packages] [Bug 1872863] Re: QEMU/KVM display is garbled when booting from kernel EFI stub due to missing bochs-drm module

2020-04-21 Thread Matthew Ruffell
Attached is proof the VM is using VGA=std for the ppc64el test.

** Attachment added: "Proof VM is running VGA=std on ppc64el"
   
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1872863/+attachment/5357796/+files/Screenshot%20from%202020-04-22%2015-14-04.png

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to kmod in Ubuntu.
https://bugs.launchpad.net/bugs/1872863

Title:
  QEMU/KVM display is garbled when booting from kernel EFI stub due to
  missing bochs-drm module

Status in kmod package in Ubuntu:
  Fix Released
Status in linux package in Ubuntu:
  Fix Released
Status in kmod source package in Bionic:
  In Progress
Status in linux source package in Bionic:
  In Progress

Bug description:
  BugLink: https://bugs.launchpad.net/bugs/1872863

  [Impact]

  A recent grub2 SRU, LP #1864533, now forces the kernel to boot via the
  kernel EFI stub whenever EFI is enabled. This causes problems for
  QEMU/KVM virtual machines which use the VGA=std video device, as the
  efifb driver yields an unreadable garbled screen. See the attached
  image.

  The correct framebuffer driver to use in this situation is bochs-drm,
  and modprobing it from a HWE kernel fixes the issues.

  bochs-drm is missing from Bionic since CONFIG_DRM_BOCHS was disabled
  in LP #1378648 due to bochs-drm causing problems in a PowerKVM
  machine. This problem appears to be fixed now, and bochs-drm has been
  re-enabled for Disco and up, in LP #1795857 and has been proven safe.

  This has also come up again in LP #1823152, as well as chatter on LP
  #1795857 to get this enabled on Bionic.

  The customer which is experiencing this issue cannot switch to VGA=qxl
  as a workaround, and must use VGA=std, hence I suggest we re-enable
  bochs-drm for Bionic.

  [Fix]

  I noticed on Focal, if you boot, the framebuffer is initially efifb:

  [ 0.603716] efifb: probing for efifb
  [ 0.603733] efifb: framebuffer at 0xc000, using 1876k, total 1875k
  [ 0.603735] efifb: mode is 800x600x32, linelength=3200, pages=1
  [ 0.603736] efifb: scrolling: redraw
  [ 0.603738] efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0
  [ 0.604462] Console: switching to colour frame buffer device 100x37
  [ 0.605829] fb0: EFI VGA frame buffer device

  This soon changes to bochs-drm about a second later:

  [ 0.935988] bochs-drm :00:01.0: remove_conflicting_pci_framebuffers: bar 
0: 0xc000 -> 0xc0ff
  [ 0.937023] bochs-drm :00:01.0: remove_conflicting_pci_framebuffers: bar 
2: 0xc1c8c000 -> 0xc1c8cfff
  [ 0.937776] checking generic (c000 1d5000) vs hw (c000 100)
  [ 0.937776] fb0: switching to bochsdrmfb from EFI VGA
  [ 0.939085] Console: switching to colour dummy device 80x25
  [ 0.939117] bochs-drm :00:01.0: vgaarb: deactivate vga console
  [ 0.939210] [drm] Found bochs VGA, ID 0xb0c5.
  [ 0.939212] [drm] Framebuffer size 16384 kB @ 0xc000, mmio @ 0xc1c8c000.
  [ 0.941955] lpc_ich :00:1f.0: I/O space for GPIO uninitialized
  [ 0.942069] [TTM] Zone kernel: Available graphics memory: 2006780 KiB
  [ 0.942081] [TTM] Initializing pool allocator
  [ 0.942089] [TTM] Initializing DMA pool allocator
  [ 0.943026] virtio_blk virtio2: [vda] 20971520 512-byte logical blocks (10.7 
GB/10.0 GiB)
  [ 0.944019] [drm] Found EDID data blob.
  [ 0.944162] [drm] Initialized bochs-drm 1.0.0 20130925 for :00:01.0 on 
minor 0
  [ 0.944979] fbcon: bochs-drmdrmfb (fb0) is primary device
  [ 0.947712] Console: switching to colour frame buffer device 128x48

  On bionic, the framebuffer never changes from efifb, since the bochs-
  drm kernel module is not built, and it is also present on the module
  banlist in /etc/modprobe.d/blacklist-framebuffer.conf

  bochs-drm needs to be enabled to be built in the kernel config, and
  removed from the module blacklist in kmod.

  [Testcase]

  Create a new QEMU/KVM virtual machine, I used virt-manager. Before you
  install the OS, check the box to modify settings before install. In
  the "Overview" tab, enable EFI by setting the firmware to "UEFI
  x86_64: /usr/share/OVMF/OVMF_CODE.secboot.fd".

  Set the video device to qxl while you install Bionic. Once the install
  is done, reboot, do a "apt update" and "apt upgrade", to ensure you
  have grub2 2.02-2ubuntu8.15 installed.

  Shut the VM down, and set the video device to VGA. Or VGA=std if you
  use the QEMU command line.

  Start the VM up, and the screen will be garbled. See attached picture.

  If you install my test packages, which are available here:

  https://launchpad.net/~mruffell/+archive/ubuntu/sf272653-test

  Instructions to install (on a bionic system):

  1) sudo add-apt-repository ppa:mruffell/sf272653-test
  2) sudo apt-get update
  3) sudo apt install linux-image-unsigned-4.15.0-96-generic 
linux-modules-4.15.0-96-generic linux-modules-extra-4.15.0-96-generic 
linux-headers-4.15.0-96-generic linux-headers-4.15.0-96 libkmod2 kmod
  4) sudo reboot
  5) uname -rv
  

[Touch-packages] [Bug 1872863] Re: QEMU/KVM display is garbled when booting from kernel EFI stub due to missing bochs-drm module

2020-04-21 Thread Matthew Ruffell
Attached is screenshot proof that bochs-drm functions correctly on
4.15.0-96-generic ppc64el, and does not cause any regressions when
bochs-drm is enabled on ppc64el.

** Attachment added: "bochs-drm functions correctly on ppc64el"
   
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1872863/+attachment/5357795/+files/Screenshot%20from%202020-04-22%2015-14-13.png

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to kmod in Ubuntu.
https://bugs.launchpad.net/bugs/1872863

Title:
  QEMU/KVM display is garbled when booting from kernel EFI stub due to
  missing bochs-drm module

Status in kmod package in Ubuntu:
  Fix Released
Status in linux package in Ubuntu:
  Fix Released
Status in kmod source package in Bionic:
  In Progress
Status in linux source package in Bionic:
  In Progress

Bug description:
  BugLink: https://bugs.launchpad.net/bugs/1872863

  [Impact]

  A recent grub2 SRU, LP #1864533, now forces the kernel to boot via the
  kernel EFI stub whenever EFI is enabled. This causes problems for
  QEMU/KVM virtual machines which use the VGA=std video device, as the
  efifb driver yields an unreadable garbled screen. See the attached
  image.

  The correct framebuffer driver to use in this situation is bochs-drm,
  and modprobing it from a HWE kernel fixes the issues.

  bochs-drm is missing from Bionic since CONFIG_DRM_BOCHS was disabled
  in LP #1378648 due to bochs-drm causing problems in a PowerKVM
  machine. This problem appears to be fixed now, and bochs-drm has been
  re-enabled for Disco and up, in LP #1795857 and has been proven safe.

  This has also come up again in LP #1823152, as well as chatter on LP
  #1795857 to get this enabled on Bionic.

  The customer which is experiencing this issue cannot switch to VGA=qxl
  as a workaround, and must use VGA=std, hence I suggest we re-enable
  bochs-drm for Bionic.

  [Fix]

  I noticed on Focal, if you boot, the framebuffer is initially efifb:

  [ 0.603716] efifb: probing for efifb
  [ 0.603733] efifb: framebuffer at 0xc000, using 1876k, total 1875k
  [ 0.603735] efifb: mode is 800x600x32, linelength=3200, pages=1
  [ 0.603736] efifb: scrolling: redraw
  [ 0.603738] efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0
  [ 0.604462] Console: switching to colour frame buffer device 100x37
  [ 0.605829] fb0: EFI VGA frame buffer device

  This soon changes to bochs-drm about a second later:

  [ 0.935988] bochs-drm :00:01.0: remove_conflicting_pci_framebuffers: bar 
0: 0xc000 -> 0xc0ff
  [ 0.937023] bochs-drm :00:01.0: remove_conflicting_pci_framebuffers: bar 
2: 0xc1c8c000 -> 0xc1c8cfff
  [ 0.937776] checking generic (c000 1d5000) vs hw (c000 100)
  [ 0.937776] fb0: switching to bochsdrmfb from EFI VGA
  [ 0.939085] Console: switching to colour dummy device 80x25
  [ 0.939117] bochs-drm :00:01.0: vgaarb: deactivate vga console
  [ 0.939210] [drm] Found bochs VGA, ID 0xb0c5.
  [ 0.939212] [drm] Framebuffer size 16384 kB @ 0xc000, mmio @ 0xc1c8c000.
  [ 0.941955] lpc_ich :00:1f.0: I/O space for GPIO uninitialized
  [ 0.942069] [TTM] Zone kernel: Available graphics memory: 2006780 KiB
  [ 0.942081] [TTM] Initializing pool allocator
  [ 0.942089] [TTM] Initializing DMA pool allocator
  [ 0.943026] virtio_blk virtio2: [vda] 20971520 512-byte logical blocks (10.7 
GB/10.0 GiB)
  [ 0.944019] [drm] Found EDID data blob.
  [ 0.944162] [drm] Initialized bochs-drm 1.0.0 20130925 for :00:01.0 on 
minor 0
  [ 0.944979] fbcon: bochs-drmdrmfb (fb0) is primary device
  [ 0.947712] Console: switching to colour frame buffer device 128x48

  On bionic, the framebuffer never changes from efifb, since the bochs-
  drm kernel module is not built, and it is also present on the module
  banlist in /etc/modprobe.d/blacklist-framebuffer.conf

  bochs-drm needs to be enabled to be built in the kernel config, and
  removed from the module blacklist in kmod.

  [Testcase]

  Create a new QEMU/KVM virtual machine, I used virt-manager. Before you
  install the OS, check the box to modify settings before install. In
  the "Overview" tab, enable EFI by setting the firmware to "UEFI
  x86_64: /usr/share/OVMF/OVMF_CODE.secboot.fd".

  Set the video device to qxl while you install Bionic. Once the install
  is done, reboot, do a "apt update" and "apt upgrade", to ensure you
  have grub2 2.02-2ubuntu8.15 installed.

  Shut the VM down, and set the video device to VGA. Or VGA=std if you
  use the QEMU command line.

  Start the VM up, and the screen will be garbled. See attached picture.

  If you install my test packages, which are available here:

  https://launchpad.net/~mruffell/+archive/ubuntu/sf272653-test

  Instructions to install (on a bionic system):

  1) sudo add-apt-repository ppa:mruffell/sf272653-test
  2) sudo apt-get update
  3) sudo apt install linux-image-unsigned-4.15.0-96-generic 
linux-modules-4.15.0-96-generic linux-modules-extra-4.15.0-96-generic 

[Touch-packages] [Bug 1872863] Re: QEMU/KVM display is garbled when booting from kernel EFI stub due to missing bochs-drm module

2020-04-15 Thread Matthew Ruffell
I rebuilt the kmod and Linux packages to include the ppc64le and powerpc
architectures, so we can test against POWER to ensure that LP #1378648
is no more, and we don't cause any regressions.

I don't have access to any POWER machines though, so I need help
testing.

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to kmod in Ubuntu.
https://bugs.launchpad.net/bugs/1872863

Title:
  QEMU/KVM display is garbled when booting from kernel EFI stub due to
  missing bochs-drm module

Status in kmod package in Ubuntu:
  Fix Released
Status in linux package in Ubuntu:
  Fix Released
Status in kmod source package in Bionic:
  In Progress
Status in linux source package in Bionic:
  In Progress

Bug description:
  BugLink: https://bugs.launchpad.net/bugs/1872863

  [Impact]

  A recent grub2 SRU, LP #1864533, now forces the kernel to boot via the
  kernel EFI stub whenever EFI is enabled. This causes problems for
  QEMU/KVM virtual machines which use the VGA=std video device, as the
  efifb driver yields an unreadable garbled screen. See the attached
  image.

  The correct framebuffer driver to use in this situation is bochs-drm,
  and modprobing it from a HWE kernel fixes the issues.

  bochs-drm is missing from Bionic since CONFIG_DRM_BOCHS was disabled
  in LP #1378648 due to bochs-drm causing problems in a PowerKVM
  machine. This problem appears to be fixed now, and bochs-drm has been
  re-enabled for Disco and up, in LP #1795857 and has been proven safe.

  This has also come up again in LP #1823152, as well as chatter on LP
  #1795857 to get this enabled on Bionic.

  The customer which is experiencing this issue cannot switch to VGA=qxl
  as a workaround, and must use VGA=std, hence I suggest we re-enable
  bochs-drm for Bionic.

  [Fix]

  I noticed on Focal, if you boot, the framebuffer is initially efifb:

  [ 0.603716] efifb: probing for efifb
  [ 0.603733] efifb: framebuffer at 0xc000, using 1876k, total 1875k
  [ 0.603735] efifb: mode is 800x600x32, linelength=3200, pages=1
  [ 0.603736] efifb: scrolling: redraw
  [ 0.603738] efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0
  [ 0.604462] Console: switching to colour frame buffer device 100x37
  [ 0.605829] fb0: EFI VGA frame buffer device

  This soon changes to bochs-drm about a second later:

  [ 0.935988] bochs-drm :00:01.0: remove_conflicting_pci_framebuffers: bar 
0: 0xc000 -> 0xc0ff
  [ 0.937023] bochs-drm :00:01.0: remove_conflicting_pci_framebuffers: bar 
2: 0xc1c8c000 -> 0xc1c8cfff
  [ 0.937776] checking generic (c000 1d5000) vs hw (c000 100)
  [ 0.937776] fb0: switching to bochsdrmfb from EFI VGA
  [ 0.939085] Console: switching to colour dummy device 80x25
  [ 0.939117] bochs-drm :00:01.0: vgaarb: deactivate vga console
  [ 0.939210] [drm] Found bochs VGA, ID 0xb0c5.
  [ 0.939212] [drm] Framebuffer size 16384 kB @ 0xc000, mmio @ 0xc1c8c000.
  [ 0.941955] lpc_ich :00:1f.0: I/O space for GPIO uninitialized
  [ 0.942069] [TTM] Zone kernel: Available graphics memory: 2006780 KiB
  [ 0.942081] [TTM] Initializing pool allocator
  [ 0.942089] [TTM] Initializing DMA pool allocator
  [ 0.943026] virtio_blk virtio2: [vda] 20971520 512-byte logical blocks (10.7 
GB/10.0 GiB)
  [ 0.944019] [drm] Found EDID data blob.
  [ 0.944162] [drm] Initialized bochs-drm 1.0.0 20130925 for :00:01.0 on 
minor 0
  [ 0.944979] fbcon: bochs-drmdrmfb (fb0) is primary device
  [ 0.947712] Console: switching to colour frame buffer device 128x48

  On bionic, the framebuffer never changes from efifb, since the bochs-
  drm kernel module is not built, and it is also present on the module
  banlist in /etc/modprobe.d/blacklist-framebuffer.conf

  bochs-drm needs to be enabled to be built in the kernel config, and
  removed from the module blacklist in kmod.

  [Testcase]

  Create a new QEMU/KVM virtual machine, I used virt-manager. Before you
  install the OS, check the box to modify settings before install. In
  the "Overview" tab, enable EFI by setting the firmware to "UEFI
  x86_64: /usr/share/OVMF/OVMF_CODE.secboot.fd".

  Set the video device to qxl while you install Bionic. Once the install
  is done, reboot, do a "apt update" and "apt upgrade", to ensure you
  have grub2 2.02-2ubuntu8.15 installed.

  Shut the VM down, and set the video device to VGA. Or VGA=std if you
  use the QEMU command line.

  Start the VM up, and the screen will be garbled. See attached picture.

  If you install my test packages, which are available here:

  https://launchpad.net/~mruffell/+archive/ubuntu/sf272653-test

  Instructions to install (on a bionic system):

  1) sudo add-apt-repository ppa:mruffell/sf272653-test
  2) sudo apt-get update
  3) sudo apt install linux-image-unsigned-4.15.0-96-generic 
linux-modules-4.15.0-96-generic linux-modules-extra-4.15.0-96-generic 
linux-headers-4.15.0-96-generic linux-headers-4.15.0-96 libkmod2 kmod
  4) sudo reboot
  5) uname -rv
 

[Touch-packages] [Bug 1872863] Re: QEMU/KVM display is garbled when booting from kernel EFI stub due to missing bochs-drm module

2020-04-14 Thread Matthew Ruffell
Attached is the debdiff for kmod, which removes bochs-drm from the
framebuffer blacklist.

** Patch added: "kmod debdiff for bionic"
   
https://bugs.launchpad.net/ubuntu/+source/kmod/+bug/1872863/+attachment/5354280/+files/lp1872863_bionic.debdiff

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to kmod in Ubuntu.
https://bugs.launchpad.net/bugs/1872863

Title:
  QEMU/KVM display is garbled when booting from kernel EFI stub due to
  missing bochs-drm module

Status in kmod package in Ubuntu:
  Fix Released
Status in linux package in Ubuntu:
  Fix Released
Status in kmod source package in Bionic:
  In Progress
Status in linux source package in Bionic:
  In Progress

Bug description:
  BugLink: https://bugs.launchpad.net/bugs/1872863

  [Impact]

  A recent grub2 SRU, LP #1864533, now forces the kernel to boot via the
  kernel EFI stub whenever EFI is enabled. This causes problems for
  QEMU/KVM virtual machines which use the VGA=std video device, as the
  efifb driver yields an unreadable garbled screen. See the attached
  image.

  The correct framebuffer driver to use in this situation is bochs-drm,
  and modprobing it from a HWE kernel fixes the issues.

  bochs-drm is missing from Bionic since CONFIG_DRM_BOCHS was disabled
  in LP #1378648 due to bochs-drm causing problems in a PowerKVM
  machine. This problem appears to be fixed now, and bochs-drm has been
  re-enabled for Disco and up, in LP #1795857 and has been proven safe.

  This has also come up again in LP #1823152, as well as chatter on LP
  #1795857 to get this enabled on Bionic.

  The customer which is experiencing this issue cannot switch to VGA=qxl
  as a workaround, and must use VGA=std, hence I suggest we re-enable
  bochs-drm for Bionic.

  [Fix]

  I noticed on Focal, if you boot, the framebuffer is initially efifb:

  [ 0.603716] efifb: probing for efifb
  [ 0.603733] efifb: framebuffer at 0xc000, using 1876k, total 1875k
  [ 0.603735] efifb: mode is 800x600x32, linelength=3200, pages=1
  [ 0.603736] efifb: scrolling: redraw
  [ 0.603738] efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0
  [ 0.604462] Console: switching to colour frame buffer device 100x37
  [ 0.605829] fb0: EFI VGA frame buffer device

  This soon changes to bochs-drm about a second later:

  [ 0.935988] bochs-drm :00:01.0: remove_conflicting_pci_framebuffers: bar 
0: 0xc000 -> 0xc0ff
  [ 0.937023] bochs-drm :00:01.0: remove_conflicting_pci_framebuffers: bar 
2: 0xc1c8c000 -> 0xc1c8cfff
  [ 0.937776] checking generic (c000 1d5000) vs hw (c000 100)
  [ 0.937776] fb0: switching to bochsdrmfb from EFI VGA
  [ 0.939085] Console: switching to colour dummy device 80x25
  [ 0.939117] bochs-drm :00:01.0: vgaarb: deactivate vga console
  [ 0.939210] [drm] Found bochs VGA, ID 0xb0c5.
  [ 0.939212] [drm] Framebuffer size 16384 kB @ 0xc000, mmio @ 0xc1c8c000.
  [ 0.941955] lpc_ich :00:1f.0: I/O space for GPIO uninitialized
  [ 0.942069] [TTM] Zone kernel: Available graphics memory: 2006780 KiB
  [ 0.942081] [TTM] Initializing pool allocator
  [ 0.942089] [TTM] Initializing DMA pool allocator
  [ 0.943026] virtio_blk virtio2: [vda] 20971520 512-byte logical blocks (10.7 
GB/10.0 GiB)
  [ 0.944019] [drm] Found EDID data blob.
  [ 0.944162] [drm] Initialized bochs-drm 1.0.0 20130925 for :00:01.0 on 
minor 0
  [ 0.944979] fbcon: bochs-drmdrmfb (fb0) is primary device
  [ 0.947712] Console: switching to colour frame buffer device 128x48

  On bionic, the framebuffer never changes from efifb, since the bochs-
  drm kernel module is not built, and it is also present on the module
  banlist in /etc/modprobe.d/blacklist-framebuffer.conf

  bochs-drm needs to be enabled to be built in the kernel config, and
  removed from the module blacklist in kmod.

  [Testcase]

  Create a new QEMU/KVM virtual machine, I used virt-manager. Before you
  install the OS, check the box to modify settings before install. In
  the "Overview" tab, enable EFI by setting the firmware to "UEFI
  x86_64: /usr/share/OVMF/OVMF_CODE.secboot.fd".

  Set the video device to qxl while you install Bionic. Once the install
  is done, reboot, do a "apt update" and "apt upgrade", to ensure you
  have grub2 2.02-2ubuntu8.15 installed.

  Shut the VM down, and set the video device to VGA. Or VGA=std if you
  use the QEMU command line.

  Start the VM up, and the screen will be garbled. See attached picture.

  If you install my test packages, which are available here:

  https://launchpad.net/~mruffell/+archive/ubuntu/sf272653-test

  Instructions to install (on a bionic system):

  1) sudo add-apt-repository ppa:mruffell/sf272653-test
  2) sudo apt-get update
  3) sudo apt install linux-image-unsigned-4.15.0-96-generic 
linux-modules-4.15.0-96-generic linux-modules-extra-4.15.0-96-generic 
linux-headers-4.15.0-96-generic linux-headers-4.15.0-96 libkmod2 kmod
  4) sudo reboot
  5) uname -rv
  

[Touch-packages] [Bug 1872863] Re: QEMU/KVM display is garbled when booting from kernel EFI stub due to missing bochs-drm module

2020-04-14 Thread Matthew Ruffell
Submitted Kernel patches to mailing list:

https://lists.ubuntu.com/archives/kernel-team/2020-April/109027.html
https://lists.ubuntu.com/archives/kernel-team/2020-April/109028.html

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to kmod in Ubuntu.
https://bugs.launchpad.net/bugs/1872863

Title:
  QEMU/KVM display is garbled when booting from kernel EFI stub due to
  missing bochs-drm module

Status in kmod package in Ubuntu:
  Fix Released
Status in linux package in Ubuntu:
  Fix Released
Status in kmod source package in Bionic:
  In Progress
Status in linux source package in Bionic:
  In Progress

Bug description:
  BugLink: https://bugs.launchpad.net/bugs/1872863

  [Impact]

  A recent grub2 SRU, LP #1864533, now forces the kernel to boot via the
  kernel EFI stub whenever EFI is enabled. This causes problems for
  QEMU/KVM virtual machines which use the VGA=std video device, as the
  efifb driver yields an unreadable garbled screen. See the attached
  image.

  The correct framebuffer driver to use in this situation is bochs-drm,
  and modprobing it from a HWE kernel fixes the issues.

  bochs-drm is missing from Bionic since CONFIG_DRM_BOCHS was disabled
  in LP #1378648 due to bochs-drm causing problems in a PowerKVM
  machine. This problem appears to be fixed now, and bochs-drm has been
  re-enabled for Disco and up, in LP #1795857 and has been proven safe.

  This has also come up again in LP #1823152, as well as chatter on LP
  #1795857 to get this enabled on Bionic.

  The customer which is experiencing this issue cannot switch to VGA=qxl
  as a workaround, and must use VGA=std, hence I suggest we re-enable
  bochs-drm for Bionic.

  [Fix]

  I noticed on Focal, if you boot, the framebuffer is initially efifb:

  [ 0.603716] efifb: probing for efifb
  [ 0.603733] efifb: framebuffer at 0xc000, using 1876k, total 1875k
  [ 0.603735] efifb: mode is 800x600x32, linelength=3200, pages=1
  [ 0.603736] efifb: scrolling: redraw
  [ 0.603738] efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0
  [ 0.604462] Console: switching to colour frame buffer device 100x37
  [ 0.605829] fb0: EFI VGA frame buffer device

  This soon changes to bochs-drm about a second later:

  [ 0.935988] bochs-drm :00:01.0: remove_conflicting_pci_framebuffers: bar 
0: 0xc000 -> 0xc0ff
  [ 0.937023] bochs-drm :00:01.0: remove_conflicting_pci_framebuffers: bar 
2: 0xc1c8c000 -> 0xc1c8cfff
  [ 0.937776] checking generic (c000 1d5000) vs hw (c000 100)
  [ 0.937776] fb0: switching to bochsdrmfb from EFI VGA
  [ 0.939085] Console: switching to colour dummy device 80x25
  [ 0.939117] bochs-drm :00:01.0: vgaarb: deactivate vga console
  [ 0.939210] [drm] Found bochs VGA, ID 0xb0c5.
  [ 0.939212] [drm] Framebuffer size 16384 kB @ 0xc000, mmio @ 0xc1c8c000.
  [ 0.941955] lpc_ich :00:1f.0: I/O space for GPIO uninitialized
  [ 0.942069] [TTM] Zone kernel: Available graphics memory: 2006780 KiB
  [ 0.942081] [TTM] Initializing pool allocator
  [ 0.942089] [TTM] Initializing DMA pool allocator
  [ 0.943026] virtio_blk virtio2: [vda] 20971520 512-byte logical blocks (10.7 
GB/10.0 GiB)
  [ 0.944019] [drm] Found EDID data blob.
  [ 0.944162] [drm] Initialized bochs-drm 1.0.0 20130925 for :00:01.0 on 
minor 0
  [ 0.944979] fbcon: bochs-drmdrmfb (fb0) is primary device
  [ 0.947712] Console: switching to colour frame buffer device 128x48

  On bionic, the framebuffer never changes from efifb, since the bochs-
  drm kernel module is not built, and it is also present on the module
  banlist in /etc/modprobe.d/blacklist-framebuffer.conf

  bochs-drm needs to be enabled to be built in the kernel config, and
  removed from the module blacklist in kmod.

  [Testcase]

  Create a new QEMU/KVM virtual machine, I used virt-manager. Before you
  install the OS, check the box to modify settings before install. In
  the "Overview" tab, enable EFI by setting the firmware to "UEFI
  x86_64: /usr/share/OVMF/OVMF_CODE.secboot.fd".

  Set the video device to qxl while you install Bionic. Once the install
  is done, reboot, do a "apt update" and "apt upgrade", to ensure you
  have grub2 2.02-2ubuntu8.15 installed.

  Shut the VM down, and set the video device to VGA. Or VGA=std if you
  use the QEMU command line.

  Start the VM up, and the screen will be garbled. See attached picture.

  If you install my test packages, which are available here:

  https://launchpad.net/~mruffell/+archive/ubuntu/sf272653-test

  Instructions to install (on a bionic system):

  1) sudo add-apt-repository ppa:mruffell/sf272653-test
  2) sudo apt-get update
  3) sudo apt install linux-image-unsigned-4.15.0-96-generic 
linux-modules-4.15.0-96-generic linux-modules-extra-4.15.0-96-generic 
linux-headers-4.15.0-96-generic linux-headers-4.15.0-96 libkmod2 kmod
  4) sudo reboot
  5) uname -rv
  4.15.0-96-generic #97+TEST272653v20200409b1-Ubuntu SMP Thu Apr 9 04:09:18 UTC 
2020


[Touch-packages] [Bug 1872863] Re: QEMU/KVM display is garbled when booting from kernel EFI stub due to missing bochs-drm module

2020-04-14 Thread Matthew Ruffell
** Description changed:

  BugLink: https://bugs.launchpad.net/bugs/1872863
  
  [Impact]
  
  A recent grub2 SRU, LP #1864533, now forces the kernel to boot via the
  kernel EFI stub whenever EFI is enabled. This causes problems for
  QEMU/KVM virtual machines which use the VGA=std video device, as the
  efifb driver yields an unreadable garbled screen. See the attached
  image.
  
  The correct framebuffer driver to use in this situation is bochs-drm,
  and modprobing it from a HWE kernel fixes the issues.
  
  bochs-drm is missing from Bionic since CONFIG_DRM_BOCHS was disabled in
  LP #1378648 due to bochs-drm causing problems in a PowerKVM machine.
  This problem appears to be fixed now, and bochs-drm has been re-enabled
  for Disco and up, in LP #1795857 and has been proven safe.
  
  This has also come up again in LP #1823152, as well as chatter on LP
  #1795857 to get this enabled on Bionic.
  
  The customer which is experiencing this issue cannot switch to VGA=qxl
  as a workaround, and must use VGA=std, hence I suggest we re-enable
  bochs-drm for Bionic.
  
  [Fix]
  
  I noticed on Focal, if you boot, the framebuffer is initially efifb:
  
  [ 0.603716] efifb: probing for efifb
  [ 0.603733] efifb: framebuffer at 0xc000, using 1876k, total 1875k
  [ 0.603735] efifb: mode is 800x600x32, linelength=3200, pages=1
  [ 0.603736] efifb: scrolling: redraw
  [ 0.603738] efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0
  [ 0.604462] Console: switching to colour frame buffer device 100x37
  [ 0.605829] fb0: EFI VGA frame buffer device
  
  This soon changes to bochs-drm about a second later:
  
  [ 0.935988] bochs-drm :00:01.0: remove_conflicting_pci_framebuffers: bar 
0: 0xc000 -> 0xc0ff
  [ 0.937023] bochs-drm :00:01.0: remove_conflicting_pci_framebuffers: bar 
2: 0xc1c8c000 -> 0xc1c8cfff
  [ 0.937776] checking generic (c000 1d5000) vs hw (c000 100)
  [ 0.937776] fb0: switching to bochsdrmfb from EFI VGA
  [ 0.939085] Console: switching to colour dummy device 80x25
  [ 0.939117] bochs-drm :00:01.0: vgaarb: deactivate vga console
  [ 0.939210] [drm] Found bochs VGA, ID 0xb0c5.
  [ 0.939212] [drm] Framebuffer size 16384 kB @ 0xc000, mmio @ 0xc1c8c000.
  [ 0.941955] lpc_ich :00:1f.0: I/O space for GPIO uninitialized
  [ 0.942069] [TTM] Zone kernel: Available graphics memory: 2006780 KiB
  [ 0.942081] [TTM] Initializing pool allocator
  [ 0.942089] [TTM] Initializing DMA pool allocator
  [ 0.943026] virtio_blk virtio2: [vda] 20971520 512-byte logical blocks (10.7 
GB/10.0 GiB)
  [ 0.944019] [drm] Found EDID data blob.
  [ 0.944162] [drm] Initialized bochs-drm 1.0.0 20130925 for :00:01.0 on 
minor 0
  [ 0.944979] fbcon: bochs-drmdrmfb (fb0) is primary device
  [ 0.947712] Console: switching to colour frame buffer device 128x48
  
  On bionic, the framebuffer never changes from efifb, since the bochs-drm
  kernel module is not built, and it is also present on the module banlist
  in /etc/modprobe.d/blacklist-framebuffer.conf
  
  bochs-drm needs to be enabled to be built in the kernel config, and
  removed from the module blacklist in kmod.
  
  [Testcase]
  
  Create a new QEMU/KVM virtual machine, I used virt-manager. Before you
  install the OS, check the box to modify settings before install. In the
  "Overview" tab, enable EFI by setting the firmware to "UEFI x86_64:
  /usr/share/OVMF/OVMF_CODE.secboot.fd".
  
  Set the video device to qxl while you install Bionic. Once the install
  is done, reboot, do a "apt update" and "apt upgrade", to ensure you have
  grub2 2.02-2ubuntu8.15 installed.
  
  Shut the VM down, and set the video device to VGA. Or VGA=std if you use
  the QEMU command line.
  
  Start the VM up, and the screen will be garbled. See attached picture.
  
  If you install my test packages, which are available here:
  
  https://launchpad.net/~mruffell/+archive/ubuntu/sf272653-test
  
  Instructions to install (on a bionic system):
  
  1) sudo add-apt-repository ppa:mruffell/sf272653-test
  2) sudo apt-get update
  3) sudo apt install linux-image-unsigned-4.15.0-96-generic 
linux-modules-4.15.0-96-generic linux-modules-extra-4.15.0-96-generic 
linux-headers-4.15.0-96-generic linux-headers-4.15.0-96 libkmod2 kmod
  4) sudo reboot
  5) uname -rv
  4.15.0-96-generic #97+TEST272653v20200409b1-Ubuntu SMP Thu Apr 9 04:09:18 UTC 
2020
  
  If you reboot, the screen will be perfectly readable, since the bochs-
  drm driver will be in use.
  
  [Regression Potential]
  
  We are enabling a display driver on Bionic which was previously
  disabled, so there will be some virtual machines making the jump to
  using bochs-drm since it would now be available.
  
  I don't think that this is a bad thing, and in many ways will probably
  end up improving some user's experience. Although there are two
  scenarios where a regression could happen:
  
- 1 - frequently gamers use GPU passthrough to their Windows based virtual
+ 1 - frequently gamers use GPU 

[Touch-packages] [Bug 1872863] [NEW] QEMU/KVM display is garbled when booting from kernel EFI stub due to missing bochs-drm module

2020-04-14 Thread Matthew Ruffell
eant for Apple
Macintosh computers only:
https://www.kernel.org/doc/html/latest/fb/efifb.html Since most gamers
have to jump through a lot of hoops to get GPU passthrough working, and
most have moved on from Bionic, I don't expect this to be a problem.

2 - if the bug with PowerKVM machines isn't actually fixed in Bionic, it
will cause a regression on POWER. We just need to test this on POWER and
ensure that the bug in LP #1378648 was fixed. If not, then we could just
disable bochs-drm on POWER, leaving it enabled for all other archs.

** Affects: kmod (Ubuntu)
 Importance: Undecided
 Status: Fix Released

** Affects: linux (Ubuntu)
 Importance: Undecided
 Status: Fix Released

** Affects: kmod (Ubuntu Bionic)
 Importance: Medium
 Assignee: Matthew Ruffell (mruffell)
 Status: In Progress

** Affects: linux (Ubuntu Bionic)
 Importance: Medium
 Assignee: Matthew Ruffell (mruffell)
 Status: In Progress


** Tags: sts

** Description changed:

- BugLink:
+ BugLink: https://bugs.launchpad.net/bugs/1872863
  
  [Impact]
  
  A recent grub2 SRU, LP #1864533, now forces the kernel to boot via the
  kernel EFI stub whenever EFI is enabled. This causes problems for
  QEMU/KVM virtual machines which use the VGA=std video device, as the
  efifb driver yields an unreadable garbled screen. See the attached
  image.
  
  The correct framebuffer driver to use in this situation is bochs-drm,
  and modprobing it from a HWE kernel fixes the issues.
  
  bochs-drm is missing from Bionic since CONFIG_DRM_BOCHS was disabled in
  LP #1378648 due to bochs-drm causing problems in a PowerKVM machine.
  This problem appears to be fixed now, and bochs-drm has been re-enabled
  for Disco and up, in LP #1795857 and has been proven safe.
  
  This has also come up again in LP #1823152, as well as chatter on LP
  #1795857 to get this enabled on Bionic.
  
  The customer which is experiencing this issue cannot switch to VGA=qxl
  as a workaround, and must use VGA=std, hence I suggest we re-enable
  bochs-drm for Bionic.
  
  [Fix]
  
  I noticed on Focal, if you boot, the framebuffer is initially efifb:
  
  [ 0.603716] efifb: probing for efifb
  [ 0.603733] efifb: framebuffer at 0xc000, using 1876k, total 1875k
  [ 0.603735] efifb: mode is 800x600x32, linelength=3200, pages=1
  [ 0.603736] efifb: scrolling: redraw
  [ 0.603738] efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0
  [ 0.604462] Console: switching to colour frame buffer device 100x37
- [ 0.605829] fb0: EFI VGA frame buffer device 
+ [ 0.605829] fb0: EFI VGA frame buffer device
  
  This soon changes to bochs-drm about a second later:
  
  [ 0.935988] bochs-drm :00:01.0: remove_conflicting_pci_framebuffers: bar 
0: 0xc000 -> 0xc0ff
  [ 0.937023] bochs-drm :00:01.0: remove_conflicting_pci_framebuffers: bar 
2: 0xc1c8c000 -> 0xc1c8cfff
  [ 0.937776] checking generic (c000 1d5000) vs hw (c000 100)
  [ 0.937776] fb0: switching to bochsdrmfb from EFI VGA
  [ 0.939085] Console: switching to colour dummy device 80x25
  [ 0.939117] bochs-drm :00:01.0: vgaarb: deactivate vga console
  [ 0.939210] [drm] Found bochs VGA, ID 0xb0c5.
  [ 0.939212] [drm] Framebuffer size 16384 kB @ 0xc000, mmio @ 0xc1c8c000.
  [ 0.941955] lpc_ich :00:1f.0: I/O space for GPIO uninitialized
  [ 0.942069] [TTM] Zone kernel: Available graphics memory: 2006780 KiB
  [ 0.942081] [TTM] Initializing pool allocator
  [ 0.942089] [TTM] Initializing DMA pool allocator
  [ 0.943026] virtio_blk virtio2: [vda] 20971520 512-byte logical blocks (10.7 
GB/10.0 GiB)
  [ 0.944019] [drm] Found EDID data blob.
  [ 0.944162] [drm] Initialized bochs-drm 1.0.0 20130925 for :00:01.0 on 
minor 0
  [ 0.944979] fbcon: bochs-drmdrmfb (fb0) is primary device
- [ 0.947712] Console: switching to colour frame buffer device 128x48 
+ [ 0.947712] Console: switching to colour frame buffer device 128x48
  
  On bionic, the framebuffer never changes from efifb, since the bochs-drm
  kernel module is not built, and it is also present on the module banlist
  in /etc/modprobe.d/blacklist-framebuffer.conf
  
  bochs-drm needs to be enabled to be built in the kernel config, and
  removed from the module blacklist in kmod.
- 
  
  [Testcase]
  
  Create a new QEMU/KVM virtual machine, I used virt-manager. Before you
  install the OS, check the box to modify settings before install. In the
  "Overview" tab, enable EFI by setting the firmware to "UEFI x86_64:
  /usr/share/OVMF/OVMF_CODE.secboot.fd".
  
  Set the video device to qxl while you install Bionic. Once the install
  is done, reboot, do a "apt update" and "apt upgrade", to ensure you have
  grub2 2.02-2ubuntu8.15 installed.
  
  Shut the VM down, and set the video device to VGA. Or VGA=std if you use
  the QEMU command line.
  
  Start the VM up, and the screen will be garbled. See attached picture.
  
  If you install my test packages, which are availabl

[Touch-packages] [Bug 1872863] Re: QEMU/KVM display is garbled when booting from kernel EFI stub due to missing bochs-drm module

2020-04-14 Thread Matthew Ruffell
Attached is a picture of the garbled screen when VGA=std under booting
via kernel EFI stub.

** Attachment added: "Garbled screen"
   
https://bugs.launchpad.net/ubuntu/+source/kmod/+bug/1872863/+attachment/5354248/+files/Screenshot%20from%202020-04-15%2012-19-48.png

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to kmod in Ubuntu.
https://bugs.launchpad.net/bugs/1872863

Title:
  QEMU/KVM display is garbled when booting from kernel EFI stub due to
  missing bochs-drm module

Status in kmod package in Ubuntu:
  Fix Released
Status in linux package in Ubuntu:
  Fix Released
Status in kmod source package in Bionic:
  In Progress
Status in linux source package in Bionic:
  In Progress

Bug description:
  BugLink: https://bugs.launchpad.net/bugs/1872863

  [Impact]

  A recent grub2 SRU, LP #1864533, now forces the kernel to boot via the
  kernel EFI stub whenever EFI is enabled. This causes problems for
  QEMU/KVM virtual machines which use the VGA=std video device, as the
  efifb driver yields an unreadable garbled screen. See the attached
  image.

  The correct framebuffer driver to use in this situation is bochs-drm,
  and modprobing it from a HWE kernel fixes the issues.

  bochs-drm is missing from Bionic since CONFIG_DRM_BOCHS was disabled
  in LP #1378648 due to bochs-drm causing problems in a PowerKVM
  machine. This problem appears to be fixed now, and bochs-drm has been
  re-enabled for Disco and up, in LP #1795857 and has been proven safe.

  This has also come up again in LP #1823152, as well as chatter on LP
  #1795857 to get this enabled on Bionic.

  The customer which is experiencing this issue cannot switch to VGA=qxl
  as a workaround, and must use VGA=std, hence I suggest we re-enable
  bochs-drm for Bionic.

  [Fix]

  I noticed on Focal, if you boot, the framebuffer is initially efifb:

  [ 0.603716] efifb: probing for efifb
  [ 0.603733] efifb: framebuffer at 0xc000, using 1876k, total 1875k
  [ 0.603735] efifb: mode is 800x600x32, linelength=3200, pages=1
  [ 0.603736] efifb: scrolling: redraw
  [ 0.603738] efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0
  [ 0.604462] Console: switching to colour frame buffer device 100x37
  [ 0.605829] fb0: EFI VGA frame buffer device

  This soon changes to bochs-drm about a second later:

  [ 0.935988] bochs-drm :00:01.0: remove_conflicting_pci_framebuffers: bar 
0: 0xc000 -> 0xc0ff
  [ 0.937023] bochs-drm :00:01.0: remove_conflicting_pci_framebuffers: bar 
2: 0xc1c8c000 -> 0xc1c8cfff
  [ 0.937776] checking generic (c000 1d5000) vs hw (c000 100)
  [ 0.937776] fb0: switching to bochsdrmfb from EFI VGA
  [ 0.939085] Console: switching to colour dummy device 80x25
  [ 0.939117] bochs-drm :00:01.0: vgaarb: deactivate vga console
  [ 0.939210] [drm] Found bochs VGA, ID 0xb0c5.
  [ 0.939212] [drm] Framebuffer size 16384 kB @ 0xc000, mmio @ 0xc1c8c000.
  [ 0.941955] lpc_ich :00:1f.0: I/O space for GPIO uninitialized
  [ 0.942069] [TTM] Zone kernel: Available graphics memory: 2006780 KiB
  [ 0.942081] [TTM] Initializing pool allocator
  [ 0.942089] [TTM] Initializing DMA pool allocator
  [ 0.943026] virtio_blk virtio2: [vda] 20971520 512-byte logical blocks (10.7 
GB/10.0 GiB)
  [ 0.944019] [drm] Found EDID data blob.
  [ 0.944162] [drm] Initialized bochs-drm 1.0.0 20130925 for :00:01.0 on 
minor 0
  [ 0.944979] fbcon: bochs-drmdrmfb (fb0) is primary device
  [ 0.947712] Console: switching to colour frame buffer device 128x48

  On bionic, the framebuffer never changes from efifb, since the bochs-
  drm kernel module is not built, and it is also present on the module
  banlist in /etc/modprobe.d/blacklist-framebuffer.conf

  bochs-drm needs to be enabled to be built in the kernel config, and
  removed from the module blacklist in kmod.

  [Testcase]

  Create a new QEMU/KVM virtual machine, I used virt-manager. Before you
  install the OS, check the box to modify settings before install. In
  the "Overview" tab, enable EFI by setting the firmware to "UEFI
  x86_64: /usr/share/OVMF/OVMF_CODE.secboot.fd".

  Set the video device to qxl while you install Bionic. Once the install
  is done, reboot, do a "apt update" and "apt upgrade", to ensure you
  have grub2 2.02-2ubuntu8.15 installed.

  Shut the VM down, and set the video device to VGA. Or VGA=std if you
  use the QEMU command line.

  Start the VM up, and the screen will be garbled. See attached picture.

  If you install my test packages, which are available here:

  https://launchpad.net/~mruffell/+archive/ubuntu/sf272653-test

  Instructions to install (on a bionic system):

  1) sudo add-apt-repository ppa:mruffell/sf272653-test
  2) sudo apt-get update
  3) sudo apt install linux-image-unsigned-4.15.0-96-generic 
linux-modules-4.15.0-96-generic linux-modules-extra-4.15.0-96-generic 
linux-headers-4.15.0-96-generic linux-headers-4.15.0-96 libkmod2 kmod
  4) sudo reboot
  5) uname -rv

[Touch-packages] [Bug 1820584] Re: seahorse assert failure: seahorse: glib-watch.c:195: timeout_update: Assertion `!t->dead' failed.

2020-04-13 Thread Matthew Ruffell
This bug is triggered when I went to upgrade the avahi packages as part of a
daily upgrade in focal:

$ sudo apt list --upgradable
Listing... Done
avahi-autoipd/focal 0.7-4ubuntu7 amd64 [upgradable from: 0.7-4ubuntu6]
avahi-daemon/focal 0.7-4ubuntu7 amd64 [upgradable from: 0.7-4ubuntu6]
avahi-utils/focal 0.7-4ubuntu7 amd64 [upgradable from: 0.7-4ubuntu6]
libavahi-client3/focal 0.7-4ubuntu7 amd64 [upgradable from: 0.7-4ubuntu6]
libavahi-common-data/focal 0.7-4ubuntu7 amd64 [upgradable from: 0.7-4ubuntu6]
libavahi-common3/focal 0.7-4ubuntu7 amd64 [upgradable from: 0.7-4ubuntu6]
libavahi-core7/focal 0.7-4ubuntu7 amd64 [upgradable from: 0.7-4ubuntu6]
libavahi-glib1/focal 0.7-4ubuntu7 amd64 [upgradable from: 0.7-4ubuntu6]
libavahi-ui-gtk3-0/focal 0.7-4ubuntu7 amd64 [upgradable from: 0.7-4ubuntu6]

When the dpkg hook is run to restart the avahi service, seahorse looses
connection to the avahi-daemon:

Apr 14 16:55:10 ubuntu avahi-daemon[694]: Got SIGTERM, quitting.
Apr 14 16:55:10 ubuntu avahi-daemon[694]: Leaving mDNS multicast group on 
interface enp1s0.IPv6 with address fe80::daf8:4688:8169:19b5.
Apr 14 16:55:10 ubuntu avahi-daemon[694]: Leaving mDNS multicast group on 
interface enp1s0.IPv4 with address 192.168.122.34.
Apr 14 16:55:10 ubuntu avahi-daemon[694]: Leaving mDNS multicast group on 
interface lo.IPv6 with address ::1.
Apr 14 16:55:10 ubuntu avahi-daemon[694]: Leaving mDNS multicast group on 
interface lo.IPv4 with address 127.0.0.1.
Apr 14 16:55:10 ubuntu seahorse[2622]: failure communicating with to avahi: 
Daemon connection failed

>From there, seahorse hits the assert at:
https://github.com/lathiat/avahi/blob/master/avahi-common/simple-
watch.c#L273-L282

Apr 14 16:55:10 ubuntu org.gnome.seahorse.Application[2622]: seahorse:
glib-watch.c:195: timeout_update: Assertion `!t->dead' failed.

The avahi daemon continues to stop, and then restarts:

Apr 14 16:55:10 ubuntu dbus-daemon[713]: [system] Activating via systemd: 
service name='org.freedesktop.Avahi' unit='dbus-org.freedesktop.Avahi.service' 
requested by ':1.114' (uid=0 pid=789 comm="/usr/sbin/cups-browsed " 
label="/usr/sbin/cups-browsed (enforce)")
Apr 14 16:55:10 ubuntu systemd[1]: Stopping Avahi mDNS/DNS-SD Stack...
Apr 14 16:55:10 ubuntu avahi-daemon[694]: avahi-daemon 0.7 exiting.
Apr 14 16:55:10 ubuntu systemd[1]: avahi-daemon.service: Succeeded.
Apr 14 16:55:10 ubuntu systemd[1]: Stopped Avahi mDNS/DNS-SD Stack.
Apr 14 16:55:10 ubuntu systemd[1]: Starting Avahi mDNS/DNS-SD Stack...

Then apport goes and makes a crash report:

Apr 14 16:55:11 ubuntu avahi-daemon[4364]: Server startup complete. Host name 
is ubuntu.local. Local service cookie is 3169540412.
Apr 14 16:55:11 ubuntu systemd[1529]: Starting Notification regarding a crash 
report...
Apr 14 16:55:11 ubuntu update-notifier-crash[4516]: /usr/bin/whoopsie
Apr 14 16:55:11 ubuntu update-notifier-crash[4519]: seahorse

The next interesting thing is, if you launch seahorse, then restart
avahi-daemon.service, it prints an error but does not crash:

$ seahorse
$ sudo systemctl restart avahi-daemon.service
(seahorse:6477): seahorse-WARNING **: 17:09:53.344: failure communicating with 
to avahi: Daemon connection failed

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to avahi in Ubuntu.
https://bugs.launchpad.net/bugs/1820584

Title:
  seahorse assert failure: seahorse: glib-watch.c:195: timeout_update:
  Assertion `!t->dead' failed.

Status in avahi package in Ubuntu:
  Confirmed

Bug description:
  -
  Description:  Ubuntu Disco Dingo (development branch)
  Release:  19.04
  -
  seahorse:
Installed: 3.32-1
Candidate: 3.32-1
Version table:
   *** 3.32-1 500
  500 http://br.archive.ubuntu.com/ubuntu disco/main amd64 Packages
  100 /var/lib/dpkg/status
  -

  I have absolutely no idea why or how. Just collected this report from
  apport-cli.

  Some times receiving "internal error" reports, no details provided,
  just "Cancel" or "Send report" options. So, I decided to check apport
  and there was this report.

  ProblemType: Crash
  DistroRelease: Ubuntu 19.04
  Package: seahorse 3.32-1
  ProcVersionSignature: Ubuntu 5.0.0-7.8-generic 5.0.0
  Uname: Linux 5.0.0-7-generic x86_64
  NonfreeKernelModules: nvidia_modeset nvidia
  ApportVersion: 2.20.10-0ubuntu23
  Architecture: amd64
  AssertionMessage: seahorse: glib-watch.c:195: timeout_update: Assertion 
`!t->dead' failed.
  CurrentDesktop: ubuntu:GNOME
  Date: Sun Mar 17 18:09:01 2019
  ExecutablePath: /usr/bin/seahorse
  InstallationDate: Installed on 2019-03-17 (0 days ago)
  InstallationMedia: Ubuntu 19.04 "Disco Dingo" - Alpha amd64 (20190316)
  ProcCmdline: /usr/bin/seahorse --gapplication-service
  Signal: 6
  SourcePackage: seahorse
  StacktraceTop:
   

[Touch-packages] [Bug 1820584] Re: seahorse assert failure: seahorse: glib-watch.c:195: timeout_update: Assertion `!t->dead' failed.

2020-04-13 Thread Matthew Ruffell
Still happening under Focal to this day.

** Tags added: focal

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to avahi in Ubuntu.
https://bugs.launchpad.net/bugs/1820584

Title:
  seahorse assert failure: seahorse: glib-watch.c:195: timeout_update:
  Assertion `!t->dead' failed.

Status in avahi package in Ubuntu:
  Confirmed

Bug description:
  -
  Description:  Ubuntu Disco Dingo (development branch)
  Release:  19.04
  -
  seahorse:
Installed: 3.32-1
Candidate: 3.32-1
Version table:
   *** 3.32-1 500
  500 http://br.archive.ubuntu.com/ubuntu disco/main amd64 Packages
  100 /var/lib/dpkg/status
  -

  I have absolutely no idea why or how. Just collected this report from
  apport-cli.

  Some times receiving "internal error" reports, no details provided,
  just "Cancel" or "Send report" options. So, I decided to check apport
  and there was this report.

  ProblemType: Crash
  DistroRelease: Ubuntu 19.04
  Package: seahorse 3.32-1
  ProcVersionSignature: Ubuntu 5.0.0-7.8-generic 5.0.0
  Uname: Linux 5.0.0-7-generic x86_64
  NonfreeKernelModules: nvidia_modeset nvidia
  ApportVersion: 2.20.10-0ubuntu23
  Architecture: amd64
  AssertionMessage: seahorse: glib-watch.c:195: timeout_update: Assertion 
`!t->dead' failed.
  CurrentDesktop: ubuntu:GNOME
  Date: Sun Mar 17 18:09:01 2019
  ExecutablePath: /usr/bin/seahorse
  InstallationDate: Installed on 2019-03-17 (0 days ago)
  InstallationMedia: Ubuntu 19.04 "Disco Dingo" - Alpha amd64 (20190316)
  ProcCmdline: /usr/bin/seahorse --gapplication-service
  Signal: 6
  SourcePackage: seahorse
  StacktraceTop:
   __assert_fail_base (fmt=0x7f733ae49588 "%s%s%s:%u: %s%sAssertion `%s' 
failed.\n%n", assertion=0x7f733aecf02d "!t->dead", file=0x7f733aecf000 
"glib-watch.c", line=195, function=) at assert.c:92
   __GI___assert_fail (assertion=0x7f733aecf02d "!t->dead", file=0x7f733aecf000 
"glib-watch.c", line=195, function=0x7f733aecf1d8 "timeout_update") at 
assert.c:101
   ?? () from /lib/x86_64-linux-gnu/libavahi-glib.so.1
   ?? () from /lib/x86_64-linux-gnu/libavahi-client.so.3
   ?? () from /lib/x86_64-linux-gnu/libavahi-glib.so.1
  Title: seahorse assert failure: seahorse: glib-watch.c:195: timeout_update: 
Assertion `!t->dead' failed.
  UpgradeStatus: No upgrade log present (probably fresh install)
  UserGroups: adm cdrom dip lpadmin plugdev sambashare sudo
  separator:

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/avahi/+bug/1820584/+subscriptions

-- 
Mailing list: https://launchpad.net/~touch-packages
Post to : touch-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~touch-packages
More help   : https://help.launchpad.net/ListHelp


[Touch-packages] [Bug 1869831] [NEW] lvm2: VG swaps to most recent disk if underlying PV has duplicate UUID and VG name

2020-03-30 Thread Matthew Ruffell
Public bug reported:

[Impact]

On xenial with lvm2 2.02.133-1ubuntu10, if you say take a snapshot of a
disk with an active VG, and attach that disk to the host, the VG will
swap from the existing disk to the newly attached disk. This breaks
mountpoints and directs writes to the wrong storage device.

Note this only happens in the edge case where the PV and VGs are
absolute duplicates.

[Test]

Steps to reproduce:

1) Create a xenial instance.
2) Create a secondary disk and attach it.
3) Create a PV, VG and LV on the disk.
4) Run lsblk:

ubuntu@ubuntu:~$ lsblk
NAME MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
vdb  253:16   02G  0 disk 
└─test1-database 252:00  1.6G  0 lvm  /mnt/database

5) Create a "snapshot" of this disk (cp -rp disk.qcow2 disk2.qcow2) 
6) Attach disk2.qcow2. Check lsblk 

ubuntu@ubuntu:~$ lsblk
NAME MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
vdb  253:16   02G  0 disk 
vdc  253:32   02G  0 disk 
└─test1-database 252:00  1.6G  0 lvm  

We can see the VG jumps from /dev/vdb to /dev/vdc, even though the system was
running at the time, and the VG was mounted.

[Regression Potential]

The risk of regression if we fix this issue is extreme.

The commits themselves are large and modify core LVM code, and change the 
behaviour of how disks are scanned and processed.
The commits introduce new configuration options.

The commits do not apply cleanly, and all of them require significant
backporting, and quite a few dependencies.

Because of this, it would be far too risky to fix this for xenial, so I
will mark this as wontfix.

[Other Info]

This was fixed in 2.02.153 by the following commits:

commit 49d9582bed338dc555b616bcc9de20fafd00ce2d
Author: David Teigland 
Date: Tue Mar 29 13:29:39 2016 -0500
Subject: lvmcache: use active LVs and device sizes to choose between duplicates
Link: 
https://github.com/lvmteam/lvm2/commit/49d9582bed338dc555b616bcc9de20fafd00ce2d

commit d4e434d1e6082ff4d448f4fbc03962c6f0926da4
Author: David Teigland 
Date: Fri Apr 29 14:42:14 2016 -0500
Subject: pvs: new attr and field for unchosen duplicate device
Link: 
https://github.com/lvmteam/lvm2/commit/d4e434d1e6082ff4d448f4fbc03962c6f0926da4

commit d3d13e134af15611c3f12107aa1627b12110a974
Author: David Teigland 
Date: Thu Feb 11 12:37:36 2016 -0600
Subject: lvmcache: process duplicate PVs directly
Link: 
https://github.com/lvmteam/lvm2/commit/d3d13e134af15611c3f12107aa1627b12110a974

commit 8b7a78c728be3b9af698dae9344d01752c4cf615
Author: David Teigland 
Date: Tue Feb 9 13:06:27 2016 -0600
Subject: lvmcache: improve duplicate PV handling
Link: 
https://github.com/lvmteam/lvm2/commit/8b7a78c728be3b9af698dae9344d01752c4cf615

commit 3d2fbfe243550b7801165f393d27e35044c1a56d
Author: David Teigland 
Date: Fri Jan 29 13:31:54 2016 -0600
Subject: lvmetad: set disabled flag when duplicate PVs are seen
Link: 
https://github.com/lvmteam/lvm2/commit/3d2fbfe243550b7801165f393d27e35044c1a56d

commit 9539ee809827a870c88d02cfb2d0dc1e3eaac0d0
Author: David Teigland 
Date: Fri Jan 29 12:25:20 2016 -0600
Subject: lvmetad: set disabled flag in lvmetad if duplicate PVs are found
Link: 
https://github.com/lvmteam/lvm2/commit/9539ee809827a870c88d02cfb2d0dc1e3eaac0d0 

The fixes are present in Ubuntu 16.10 and later, meaning xenial is the
only impacted supported release.

** Affects: lvm2 (Ubuntu)
 Importance: Undecided
 Status: Fix Released

** Affects: lvm2 (Ubuntu Xenial)
 Importance: Medium
 Assignee: Matthew Ruffell (mruffell)
 Status: Won't Fix

** Affects: lvm2 (Ubuntu Bionic)
 Importance: Undecided
 Status: Fix Released

** Affects: lvm2 (Ubuntu Disco)
 Importance: Undecided
 Status: Fix Released

** Affects: lvm2 (Ubuntu Eoan)
 Importance: Undecided
 Status: Fix Released

** Affects: lvm2 (Ubuntu Focal)
 Importance: Undecided
 Status: Fix Released


** Tags: sts

** Also affects: lvm2 (Ubuntu Xenial)
   Importance: Undecided
   Status: New

** Also affects: lvm2 (Ubuntu Bionic)
   Importance: Undecided
   Status: New

** Also affects: lvm2 (Ubuntu Disco)
   Importance: Undecided
   Status: New

** Also affects: lvm2 (Ubuntu Focal)
   Importance: Undecided
   Status: New

** Also affects: lvm2 (Ubuntu Eoan)
   Importance: Undecided
   Status: New

** Changed in: lvm2 (Ubuntu Bionic)
   Status: New => Fix Released

** Changed in: lvm2 (Ubuntu Disco)
   Status: New => Fix Released

** Changed in: lvm2 (Ubuntu Eoan)
   Status: New => Fix Released

** Changed in: lvm2 (Ubuntu Focal)
   Status: New => Fix Released

** Changed in: lvm2 (Ubuntu Xenial)
   Status: New => Won't Fix

** Changed in: lvm2 (Ubuntu Xenial)
   Importance: Undecided => Medium

** Changed in: lvm2 (Ubuntu Xenial)
     Assignee: (unassigned) => Matthew Ruffell (mruffell)

** Tags added: sts

-- 
You received this bug notification because you are a member 

[Touch-packages] [Bug 1827172] Re: update-rc.d: enabling or disabling S runlevel services incorrectly modifies runlevel

2019-11-26 Thread Matthew Ruffell
The fix was released in sysv-rc 2.88dsf-41ubuntu6.3+esm1 for Trusty ESM.

** Changed in: sysvinit (Ubuntu Trusty)
   Status: In Progress => Fix Released

** Changed in: sysvinit (Ubuntu)
   Status: New => Won't Fix

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to sysvinit in Ubuntu.
https://bugs.launchpad.net/bugs/1827172

Title:
  update-rc.d: enabling or disabling S runlevel services incorrectly
  modifies runlevel

Status in sysvinit package in Ubuntu:
  Won't Fix
Status in sysvinit source package in Trusty:
  Fix Released

Bug description:
  [Impact]

   * update-rc.d, in sysv-rc-2.88dsf-41ubuntu6.3 is broken in trusty.

   * update-rc.d incorrectly modifies symlinks when enabling or disabling
     services which are started on the "S" runlevel.

   * This can lead to services being changed from S runlevel from where they
     would be started on boot, to "0" runlevel, and are run on halt, which is
     incorrect.

   * The bug is caused by trying to use the runlevel to index into an integer
     array of runlevels. When the runlevel in question is "S", an error is
     printed

     Argument "S" isn't numeric in array element at /usr/sbin/update-rc.d line
     232.

     Perl then sets the index to default to 0, which changes the
  runlevel.

   * The fix is to check if the runlevel is "S", and if it is, set the index to
     99 which conforms with other expected usages for the "S" runlevel in the
     script. See the "startstop" and "makelinks" subroutines.

  [Test Case]

  * You can reproduce this with any service that is started on the "S"
    runlevel. We will use open-iscsi for an example.

  1) Install open-iscsi

  $ sudo apt install open-iscsi

  2) Check to see symlinks for init.d scripts are set to defaults:

  root@trusty-openiscsi:/etc# ls -l /etc/rc[0123456S].d/*iscsi*
  /etc/rc0.d/K80umountiscsi.sh -> ../init.d/umountiscsi.sh
  /etc/rc0.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc1.d/K80umountiscsi.sh -> ../init.d/umountiscsi.sh
  /etc/rc1.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc6.d/K80umountiscsi.sh -> ../init.d/umountiscsi.sh
  /etc/rc6.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rcS.d/S45open-iscsi -> ../init.d/open-iscsi

  3) Use update-rc.d to enable open-iscsi service

  root@trusty-openiscsi:/etc# update-rc.d open-iscsi enable
  update-rc.d: warning: start runlevel arguments (none) do not match open-iscsi 
Default-Start values (S)
  update-rc.d: warning: stop runlevel arguments (none) do not match open-iscsi 
Default-Stop values (0 1 6)
  Argument "S" isn't numeric in array element at /usr/sbin/update-rc.d line 232.
  Enabling system startup links for /etc/init.d/open-iscsi ...
  Removing any system startup links for /etc/init.d/open-iscsi ...
  /etc/rc0.d/K81open-iscsi
  /etc/rc1.d/K81open-iscsi
  /etc/rc6.d/K81open-iscsi
  /etc/rcS.d/S45open-iscsi
  Adding system startup for /etc/init.d/open-iscsi ...
  /etc/rc0.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc1.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc6.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc0.d/S45open-iscsi -> ../init.d/open-iscsi

  * The problem is the "/etc/rcS.d/S45open-iscsi" symlink is incorrectly
    changed to "/etc/rc0.d/S45open-iscsi".

  * Instead, the correct behaviour is to keep the symlink in /etc/rcS.d/
    intact:

  root@trusty-openiscsi:/etc# update-rc.d open-iscsi enable
  update-rc.d: warning: start runlevel arguments (none) do not match open-iscsi 
Default-Start values (S)
  update-rc.d: warning: stop runlevel arguments (none) do not match open-iscsi 
Default-Stop values (0 1 6)
  Enabling system startup links for /etc/init.d/open-iscsi ...
  Removing any system startup links for /etc/init.d/open-iscsi ...
  /etc/rc0.d/K81open-iscsi
  /etc/rc1.d/K81open-iscsi
  /etc/rc6.d/K81open-iscsi
  /etc/rcS.d/S45open-iscsi
  Adding system startup for /etc/init.d/open-iscsi ...
  /etc/rc0.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc1.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc6.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rcS.d/S45open-iscsi -> ../init.d/open-iscsi

  [Regression Potential]

   * There is only one file modified, which is the update-rc.d perl script.
     Worst case scenario is that users cannot enable or disable their services,
     or a symlink is changed such that a service is started / stopped on an
     incorrect runlevel.

   * If a regression happens, any damage should be easily spotted by a
     sysadmin and can be manually repaired by making manual symlinks with
     "ln -s".

   * I have built and tested the change in a PPA, which you can find here:
 https://launchpad.net/~mruffell/+archive/ubuntu/sysvinit-testing

   * My only cause for concern is that if a regression does happen, it may 
 impact packages that run "update-rc.d  [en|dis]able" in a 
 postinstall module, although I haven't managed to find an example that
 does this, since most use 

[Touch-packages] [Bug 1847176] Re: [HP ENVY Laptop 13-ad1xx, Realtek ALC295, Speaker, Internal] Pulseaudio fails to detect card

2019-10-08 Thread Matthew Ruffell
@gunnarl, did your audio work correctly in kernel 5.0.0-29, and broke when you 
updated to 5.0.0-31?
If you boot 5.0.0-29 by selecting it in grub when you turn your computer on, 
does your audio work?

Might this be related to Bug 1846991?

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to alsa-driver in Ubuntu.
https://bugs.launchpad.net/bugs/1847176

Title:
  [HP ENVY Laptop 13-ad1xx, Realtek ALC295, Speaker, Internal]
  Pulseaudio fails to detect card

Status in alsa-driver package in Ubuntu:
  New
Status in linux package in Ubuntu:
  Confirmed

Bug description:
  No Sound on laptop speaker after upgrading ubuntu 19.04 a few days
  ago. Soundcard is found in alsamixer but only "dummy" is available in
  gnome settings.

  ProblemType: Bug
  DistroRelease: Ubuntu 19.04
  Package: pulseaudio 1:12.2-2ubuntu3
  ProcVersionSignature: Ubuntu 5.0.0-31.33-generic 5.0.21
  Uname: Linux 5.0.0-31-generic x86_64
  ApportVersion: 2.20.10-0ubuntu27.1
  Architecture: amd64
  AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/by-path', 
'/dev/snd/controlC0', '/dev/snd/hwC0D2', '/dev/snd/hwC0D0', 
'/dev/snd/pcmC0D10p', '/dev/snd/pcmC0D9p', '/dev/snd/pcmC0D8p', 
'/dev/snd/pcmC0D7p', '/dev/snd/pcmC0D3p', '/dev/snd/pcmC0D0c', 
'/dev/snd/pcmC0D0p', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
  CurrentDesktop: ubuntu:GNOME
  Date: Mon Oct  7 20:18:10 2019
  EcryptfsInUse: Yes
  InstallationDate: Installed on 2017-11-25 (681 days ago)
  InstallationMedia: Ubuntu 17.10 "Artful Aardvark" - Release amd64 (20171018)
  ProcEnviron:
   TERM=xterm-256color
   PATH=(custom, no user)
   XDG_RUNTIME_DIR=
   LANG=en_US.UTF-8
   SHELL=/bin/bash
  SourcePackage: pulseaudio
  Symptom: audio
  Symptom_Card: HDA-Intel - HDA Intel PCH
  Symptom_Jack: Speaker, Internal
  Title: [HP ENVY Laptop 13-ad1xx, Realtek ALC295, Speaker, Internal] 
Pulseaudio fails to detect card
  UpgradeStatus: Upgraded to disco on 2018-05-26 (499 days ago)
  dmi.bios.date: 08/07/2017
  dmi.bios.vendor: Insyde
  dmi.bios.version: F.13
  dmi.board.asset.tag: Type2 - Board Asset Tag
  dmi.board.name: 83A9
  dmi.board.vendor: HP
  dmi.board.version: KBC Version 39.32
  dmi.chassis.type: 10
  dmi.chassis.vendor: HP
  dmi.chassis.version: Chassis Version
  dmi.modalias: 
dmi:bvnInsyde:bvrF.13:bd08/07/2017:svnHP:pnHPENVYLaptop13-ad1xx:pvrType1ProductConfigId:rvnHP:rn83A9:rvrKBCVersion39.32:cvnHP:ct10:cvrChassisVersion:
  dmi.product.family: 103C_5335KV HP Envy
  dmi.product.name: HP ENVY Laptop 13-ad1xx
  dmi.product.sku: 1KT13UA#ABA
  dmi.product.version: Type1ProductConfigId
  dmi.sys.vendor: HP
  --- 
  ProblemType: Bug
  ApportVersion: 2.20.10-0ubuntu27.1
  Architecture: amd64
  AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/by-path', 
'/dev/snd/controlC0', '/dev/snd/hwC0D2', '/dev/snd/hwC0D0', 
'/dev/snd/pcmC0D10p', '/dev/snd/pcmC0D9p', '/dev/snd/pcmC0D8p', 
'/dev/snd/pcmC0D7p', '/dev/snd/pcmC0D3p', '/dev/snd/pcmC0D0c', 
'/dev/snd/pcmC0D0p', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
  CurrentDesktop: ubuntu:GNOME
  DistroRelease: Ubuntu 19.04
  EcryptfsInUse: Yes
  HibernationDevice: RESUME=UUID=ec963f21-6d46-4fd7-a5b5-286e29729b6b
  InstallationDate: Installed on 2017-11-25 (681 days ago)
  InstallationMedia: Ubuntu 17.10 "Artful Aardvark" - Release amd64 (20171018)
  Lsusb:
   Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
   Bus 001 Device 003: ID 8087:0a2a Intel Corp. 
   Bus 001 Device 002: ID 0bda:5620 Realtek Semiconductor Corp. 
   Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
  MachineType: HP HP ENVY Laptop 13-ad1xx
  Package: linux (not installed)
  ProcEnviron:
   TERM=xterm-256color
   PATH=(custom, no user)
   XDG_RUNTIME_DIR=
   LANG=en_US.UTF-8
   SHELL=/bin/bash
  ProcFB: 0 inteldrmfb
  ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-5.0.0-31-generic 
root=UUID=d203e9e9-3d32-4223-b420-e5ce61567401 ro quiet splash vt.handoff=1
  ProcVersionSignature: Ubuntu 5.0.0-31.33-generic 5.0.21
  RelatedPackageVersions:
   linux-restricted-modules-5.0.0-31-generic N/A
   linux-backports-modules-5.0.0-31-generic  N/A
   linux-firmware1.178.3
  Tags:  disco
  Uname: Linux 5.0.0-31-generic x86_64
  UpgradeStatus: Upgraded to disco on 2018-05-26 (499 days ago)
  UserGroups: adm cdrom dip lpadmin plugdev sambashare sudo vboxusers
  _MarkForUpload: True
  dmi.bios.date: 08/07/2017
  dmi.bios.vendor: Insyde
  dmi.bios.version: F.13
  dmi.board.asset.tag: Type2 - Board Asset Tag
  dmi.board.name: 83A9
  dmi.board.vendor: HP
  dmi.board.version: KBC Version 39.32
  dmi.chassis.type: 10
  dmi.chassis.vendor: HP
  dmi.chassis.version: Chassis Version
  dmi.modalias: 
dmi:bvnInsyde:bvrF.13:bd08/07/2017:svnHP:pnHPENVYLaptop13-ad1xx:pvrType1ProductConfigId:rvnHP:rn83A9:rvrKBCVersion39.32:cvnHP:ct10:cvrChassisVersion:
  dmi.product.family: 103C_5335KV HP Envy
  dmi.product.name: HP ENVY Laptop 

[Touch-packages] [Bug 1838358] Re: Ibus causes gnome-shell to freeze when password fields are selected in Firefox

2019-08-08 Thread Matthew Ruffell
I enabled bionic-proposed and installed ibus, ibus-gtk, ibus-gtk3
versions 1.5.17-3ubuntu5.

I ran the test in a bionic VM which had VMware Horizon Agent for Linux
installed, version 7.9.0-13916467.

Running firefox with no environment variables caused long input delays
and small lockups with the affected gnome-shell library shipped by
VMware Horizon.

When enabling the following environment variables for ibus which are
implemented as part of this SRU:

$ env IBUS_DISCARD_PASSWORD=1 firefox
and
$ export IBUS_DISCARD_PASSWORD_APPS="firefox"
$ firefox

I could enter text into password fields in firefox with no problems at
all, and the problem is fixed.

The customer has also validated that the fix works in their VMware
Horizon environment, and the problem is solved.

I am happy to mark this as verified.

** Tags removed: verification-needed verification-needed-bionic
** Tags added: verification-done-bionic

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to ibus in Ubuntu.
https://bugs.launchpad.net/bugs/1838358

Title:
  Ibus causes gnome-shell to freeze when password fields are selected in
  Firefox

Status in ibus package in Ubuntu:
  Fix Released
Status in ibus source package in Bionic:
  Fix Committed

Bug description:
  [Impact]

  The following has been seen in a VMware Horizon VDI. I cannot reproduce this
  issue myself.

  When a user interacts with any password field in Firefox, gnome-shell
  and Firefox both freeze and the system becomes unusable. If you ssh
  into the system and terminate Firefox, gnome-shell unfreezes.

  This only happens when the environment variable GTK_IM_MODULE is set to 
"ibus". If you unset the variable, or change it to
  GTK_IM_MODULE=gtk-im-context-simple and then start Firefox, everything works 
as intended.

  This has been seen before with gnome-shell 3.28.4-0ubuntu18.04.1, ibus
  1.5.17-3ubuntu4 and Firefox versions starting with 
68.0+build3-0ubuntu0.18.04.1

  Note: Chrome[ium] and other applications do not trigger it, and it cannot be
  reproduced in other desktop environments.

  This seems to be an interaction issue between ibus and gnome-shell.

  
  [Test Case]

  Launch firefox from within a gnome-session, making sure the
  GTK_IM_MODULE is set to "ibus". Note, this is the default value.

  $ env GTK_IM_MODULE="ibus" firefox

  Navigate to any website which has a password field. Wikipedia or
  Reddit will do.

  Click a password field and attempt to enter text. Firefox and gnome-
  shell both lock up and stay frozen for an extended period of time.

  Now, try it with the fix by enabling:

  $ env IBUS_DISCARD_PASSWORD=1 firefox

  When you enter text into a password field, ibus should directly pass through
  the text and the problem will be solved.

  We can also ask it to always apply for a specific application with:

  $ export IBUS_DISCARD_PASSWORD_APPS="firefox"
  $ firefox

  Again, when you enter text into a password input field, the problem will be
  solved.

  Test package is available here:

  https://launchpad.net/~mruffell/+archive/ubuntu/sf235370-test

  Please test with the revised version,
  1.5.17-3ubuntu4+sf235370v20190731b1.

  [Regression Potential]

  This change has a low risk of regression, because the default behaviour is
  unchanged. To be able to use the password input field discard functionality, 
a user has to explicitly set an environment variable for the specific process, 
or set a regex that matches a process name.

  This means the fix is not enabled by default on any machines, and will
  only be utilised by those suffering problems and go and manually set
  environment variables or have their system administrator enable the
  environment variables permanently.

  This commit is present in upstream ibus from version ibus-1.5.19
  onward, and is currently present in cosmic, disco and eoan.

  If a regression occurs, users can ensure that the environment
  variables are unset and continue working.

  [Other info]

  * This patch is functionally the same as ibus-xx-f19-password.patch,
  but just hides the features behind environment variables.

  * When ibus is built with the patch ibus-xx-f19-password.patch which
  was dropped in ibus-1.5.17-2, the problem is solved.

  Instead of using ibus-xx-f19-password.patch, we will instead fix it with
  upstream commit f328fd67f479faa46ca87bf3c85eed7080ec5ec0:

  https://github.com/ibus/ibus/commit/f328fd67f479faa46ca87bf3c85eed7080ec5ec0

  Subject: client/gtk2: Add IBUS_DISCARD_PASSWORD for firefox and chrome
  Author: fujiwarat 

  This implements the password discard functionality found in
  ibus-xx-f19-password.patch and places it behind two environment variables,
  IBUS_DISCARD_PASSWORD and IBUS_DISCARD_PASSWORD_APPS.

  IBUS_DISCARD_PASSWORD is for a single process, and IBUS_DISCARD_PASSWORD_APPS
  lets you set a regex of process names to filter and enable the fix for.

  If IBUS_DISCARD_PASSWORD is set or 

[Touch-packages] [Bug 1838358] Re: Ibus causes gnome-shell to freeze when password fields are selected in Firefox

2019-08-05 Thread Matthew Ruffell
Hi Lukasz,

We are in luck, after some very helpful debugging done by the customer, I have
been able to reliably reproduce this issue myself.

Firstly, this issue only affects users of Ubuntu inside a VMware Horizon VDI, 
and only occurs after you install the VMware Horizon Agent for Linux. 

The issue can be replicated reliably in the customer environment, and several 
hundred users have been affected.

I downloaded and installed the Horizon Agent 
VMware-horizonagent-linux-x86_64-7.9.0-13916467.tar.gz in my test 18.04 VM.

Download location [1] (Requires registration)
Installation Instructions [2]

I now see gnome-shell and firefox act strange, with a large delay in input to 
the password field and having the box clearing randomly.

The customer also sees this behaviour. If the customer enables NVIDIA GRID
graphics acceleration, then gnome-shell freezes instead of having the large 
input delay. Disabling graphics acceleration shows the behaviour I can 
reproduce, of seeing large input delay and intermittent gnome-shell lockups.

I did some digging, and it seems the Horizon agent replaces some gnome-shell
libraries with its own.

# openssl sha256 ./usr/lib/gnome-shell/libgnome-shell.so
bd86f21646db0be70139fa75879c6e56dd28b849bfc4a8ed4fae390c8bac
# openssl sha256 
./home/ubuntu/VMware-horizonagent-linux-x86_64-7.9.0-13916467/sso/ubuntu/1804/libgnome-shell.so
bd86f21646db0be70139fa75879c6e56dd28b849bfc4a8ed4fae390c8bac

# openssl sha256 ./usr/lib/vmware/viewagent/sso/backup/libgnome-shell.orig.so
381698f0554e53512c3627afd170dcd1567fa833d9a0e19fb36c0054296ead00
# openssl sha256 
./home/ubuntu/VMware-horizonagent-linux-x86_64-7.9.0-13916467/sso/ubuntu/1804/libgnome-shell.orig.so
381698f0554e53512c3627afd170dcd1567fa833d9a0e19fb36c0054296ead00

VMware Horizon has a feature called "True SSO" [3] which seems to enable users
to authenticate once to the VMware Identity Manager, and then use a SSO token
afterwards to log into supported websites.

>From what I can see, VMware have built their own custom gnome-shell libraries
and modified their library to implement the True SSO feature, which probably
uses interactions between gnome-shell and ibus to detect password input fields
and automate password entry.

When I revert the custom gnome-shell library to the one Canonical provides in
the Ubuntu 18.04 main archive, the issue is fixed, and I can enter text in
password fields normally.

>From our side, Canonical cannot support these custom gnome-shell libraries,
since we do not have source code for them and we don't know whats in them or
how they are built.

What we can do is work around them, and the proposed change to ibus does
exactly that. With those environment variables in place, input is "discarded"
by ibus and passed directly into the application, and likely skips the 
"True SSO" functionality which is breaking Firefox.

A case has now been opened with VMware so they can fix their custom gnome-shell
libraries, but for now, the ibus workaround is a agreeable solution for this
customer, mostly since they do not use the "True SSO" functionality.

I can now replicate the problem, and customer is also on board to test the
-proposed package in their environment.

I can now verify myself that the test package in my ppa works around the problem
successfully.

Lukasz, I hope that answers some of your questions.

Matthew

[1] https://my.vmware.com/group/vmware/evalcenter?p=horizon-7#tab_download
[2] 
https://docs.vmware.com/en/VMware-Horizon-7/7.1/com.vmware.horizon-view.linuxdesktops.doc/GUID-F06FF1A7-BDEF-4269-B2AB-C62819D4FCCD.html
[3] 
https://blogs.vmware.com/euc/2016/03/true-sso-single-sign-on-view-identity-manager-authenticate.html

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to ibus in Ubuntu.
https://bugs.launchpad.net/bugs/1838358

Title:
  Ibus causes gnome-shell to freeze when password fields are selected in
  Firefox

Status in ibus package in Ubuntu:
  Fix Released
Status in ibus source package in Bionic:
  In Progress

Bug description:
  [Impact]

  The following has been seen in a VMware Horizon VDI. I cannot reproduce this
  issue myself.

  When a user interacts with any password field in Firefox, gnome-shell
  and Firefox both freeze and the system becomes unusable. If you ssh
  into the system and terminate Firefox, gnome-shell unfreezes.

  This only happens when the environment variable GTK_IM_MODULE is set to 
"ibus". If you unset the variable, or change it to
  GTK_IM_MODULE=gtk-im-context-simple and then start Firefox, everything works 
as intended.

  This has been seen before with gnome-shell 3.28.4-0ubuntu18.04.1, ibus
  1.5.17-3ubuntu4 and Firefox versions starting with 
68.0+build3-0ubuntu0.18.04.1

  Note: Chrome[ium] and other applications do not trigger it, and it cannot be
  reproduced in other desktop environments.

  This seems to be an interaction issue between ibus and gnome-shell.

  
  [Test Case]

  

[Touch-packages] [Bug 1838358] Re: Ibus causes gnome-shell to freeze when password fields are selected in Firefox

2019-07-31 Thread Matthew Ruffell
Revised debdiff for bionic which enables the environment variables
IBUS_DISCARD_PASSWORD and IBUS_DISCARD_PASSWORD_APPS

** Patch added: "revised debdiff for bionic"
   
https://bugs.launchpad.net/ubuntu/+source/ibus/+bug/1838358/+attachment/5280261/+files/lp1838358_bionic_revised.debdiff

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to ibus in Ubuntu.
https://bugs.launchpad.net/bugs/1838358

Title:
  Ibus causes gnome-shell to freeze when password fields are selected in
  Firefox

Status in ibus package in Ubuntu:
  Confirmed
Status in ibus source package in Bionic:
  In Progress

Bug description:
  [Impact]

  The following has been seen in a VMware Horizon VDI. I cannot reproduce this
  issue myself.

  When a user interacts with any password field in Firefox, gnome-shell
  and Firefox both freeze and the system becomes unusable. If you ssh
  into the system and terminate Firefox, gnome-shell unfreezes.

  This only happens when the environment variable GTK_IM_MODULE is set to 
"ibus". If you unset the variable, or change it to
  GTK_IM_MODULE=gtk-im-context-simple and then start Firefox, everything works 
as intended.

  This has been seen before with gnome-shell 3.28.4-0ubuntu18.04.1, ibus
  1.5.17-3ubuntu4 and Firefox versions starting with 
68.0+build3-0ubuntu0.18.04.1

  Note: Chrome[ium] and other applications do not trigger it, and it cannot be
  reproduced in other desktop environments.

  This seems to be an interaction issue between ibus and gnome-shell.

  [Fix]

  When ibus is built with the patch ibus-xx-f19-password.patch which was
  dropped in ibus-1.5.17-2, the problem is solved.

  Instead of using ibus-xx-f19-password.patch, we will instead fix it with
  upstream commit f328fd67f479faa46ca87bf3c85eed7080ec5ec0:

  https://github.com/ibus/ibus/commit/f328fd67f479faa46ca87bf3c85eed7080ec5ec0

  Subject: client/gtk2: Add IBUS_DISCARD_PASSWORD for firefox and chrome 
  Author: fujiwarat 

  This implements the password discard functionality found in 
  ibus-xx-f19-password.patch and places it behind two environment variables,
  IBUS_DISCARD_PASSWORD and IBUS_DISCARD_PASSWORD_APPS.

  IBUS_DISCARD_PASSWORD is for a single process, and IBUS_DISCARD_PASSWORD_APPS
  lets you set a regex of process names to filter and enable the fix for.

  If IBUS_DISCARD_PASSWORD is set or IBUS_DISCARD_PASSWORD_APPS is set with the
  process name which input is being placed into password fields, ibus will pass
  through the input to the application without any processing.

  [Testcase]

  Launch firefox from within a gnome-session, making sure the
  GTK_IM_MODULE is set to "ibus". Note, this is the default value.

  $ env GTK_IM_MODULE="ibus" firefox

  Navigate to any website which has a password field. Wikipedia or
  Reddit will do.

  Click a password field and attempt to enter text. Firefox and gnome-
  shell both lock up and stay frozen for an extended period of time.

  Now, try it with the fix by enabling:

  $ env IBUS_DISCARD_PASSWORD=1 firefox

  When you enter text into a password field, ibus should directly pass through
  the text and the problem will be solved.

  We can also ask it to always apply for a specific application with:

  $ export IBUS_DISCARD_PASSWORD_APPS="firefox"
  $ firefox 

  Again, when you enter text into a password input field, the problem will be
  solved.

  Test package is available here:

  https://launchpad.net/~mruffell/+archive/ubuntu/sf235370-test

  Please test with the revised version,
  1.5.17-3ubuntu4+sf235370v20190731b1.

  [Regression Potential]

  This change has a low risk of regression, because the default behaviour is
  unchanged. To be able to use the password input field discard functionality,
  a user has to explicitly set an environment variable for the specific 
process, or set a regex that matches a process name.

  This means the fix is not enabled by default on any machines, and will
  only be utilised by those suffering problems and go and manually set
  environment variables or have their system administrator enable the
  environment variables permanently.

  This commit is present in upstream ibus from version ibus-1.5.19 onward, and
  is currently present in cosmic, disco and eoan. 

  If a regression occurs, users can ensure that the environment variables are
  unset and continue working.

  [Notes]

  This patch is functionally the same as ibus-xx-f19-password.patch, but just
  hides the features behind environment variables.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/ibus/+bug/1838358/+subscriptions

-- 
Mailing list: https://launchpad.net/~touch-packages
Post to : touch-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~touch-packages
More help   : https://help.launchpad.net/ListHelp


[Touch-packages] [Bug 1838358] Re: Ibus causes gnome-shell to freeze when password fields are selected in Firefox

2019-07-31 Thread Matthew Ruffell
Hi Oliver,

Full details from the environment which has the problem:

Affected machines are all running Bionic and are fully patched.
firefox: 68.0.1+build1-0ubuntu0.18.04.1
gnome-shell: 3.28.4-0ubuntu18.04.1
ibus: 1.5.17-3ubuntu4

Machines are accessed as a VDI through VMware Horizon 7.9.0, and users who have
problems are using Gnome 3.

After discussing this with Eric D, we agreed that it might be a better idea to
not restore the old ibus-xx-f19-password.patch and instead go with the upstream
solution of 
https://github.com/ibus/ibus/commit/f328fd67f479faa46ca87bf3c85eed7080ec5ec0
which implements the same functionality, but hides it behind two environment
variables, IBUS_DISCARD_PASSWORD and IBUS_DISCARD_PASSWORD_APPS.

To be honest, I think this approach is better since it doesn't change any
default behaviour and the fix is opt in for environments which need it. Plus
this is present in cosmic, disco and eoan, and will be available in the future 
20.04 LTS.

I have tried my best to reproduce this, but I have not had any luck. I think
this solution is probably the best for all parties, and this has been tested
successfully in the affected environment, so I have made a new debdiff with the
above patch.

I would like to determine and fix the root cause, but that might be a little
challenging, and this is a sufficient workaround.

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to ibus in Ubuntu.
https://bugs.launchpad.net/bugs/1838358

Title:
  Ibus causes gnome-shell to freeze when password fields are selected in
  Firefox

Status in ibus package in Ubuntu:
  Confirmed
Status in ibus source package in Bionic:
  In Progress

Bug description:
  [Impact]

  The following has been seen in a VMware Horizon VDI. I cannot reproduce this
  issue myself.

  When a user interacts with any password field in Firefox, gnome-shell
  and Firefox both freeze and the system becomes unusable. If you ssh
  into the system and terminate Firefox, gnome-shell unfreezes.

  This only happens when the environment variable GTK_IM_MODULE is set to 
"ibus". If you unset the variable, or change it to
  GTK_IM_MODULE=gtk-im-context-simple and then start Firefox, everything works 
as intended.

  This has been seen before with gnome-shell 3.28.4-0ubuntu18.04.1, ibus
  1.5.17-3ubuntu4 and Firefox versions starting with 
68.0+build3-0ubuntu0.18.04.1

  Note: Chrome[ium] and other applications do not trigger it, and it cannot be
  reproduced in other desktop environments.

  This seems to be an interaction issue between ibus and gnome-shell.

  [Fix]

  When ibus is built with the patch ibus-xx-f19-password.patch which was
  dropped in ibus-1.5.17-2, the problem is solved.

  Instead of using ibus-xx-f19-password.patch, we will instead fix it with
  upstream commit f328fd67f479faa46ca87bf3c85eed7080ec5ec0:

  https://github.com/ibus/ibus/commit/f328fd67f479faa46ca87bf3c85eed7080ec5ec0

  Subject: client/gtk2: Add IBUS_DISCARD_PASSWORD for firefox and chrome 
  Author: fujiwarat 

  This implements the password discard functionality found in 
  ibus-xx-f19-password.patch and places it behind two environment variables,
  IBUS_DISCARD_PASSWORD and IBUS_DISCARD_PASSWORD_APPS.

  IBUS_DISCARD_PASSWORD is for a single process, and IBUS_DISCARD_PASSWORD_APPS
  lets you set a regex of process names to filter and enable the fix for.

  If IBUS_DISCARD_PASSWORD is set or IBUS_DISCARD_PASSWORD_APPS is set with the
  process name which input is being placed into password fields, ibus will pass
  through the input to the application without any processing.

  [Testcase]

  Launch firefox from within a gnome-session, making sure the
  GTK_IM_MODULE is set to "ibus". Note, this is the default value.

  $ env GTK_IM_MODULE="ibus" firefox

  Navigate to any website which has a password field. Wikipedia or
  Reddit will do.

  Click a password field and attempt to enter text. Firefox and gnome-
  shell both lock up and stay frozen for an extended period of time.

  Now, try it with the fix by enabling:

  $ env IBUS_DISCARD_PASSWORD=1 firefox

  When you enter text into a password field, ibus should directly pass through
  the text and the problem will be solved.

  We can also ask it to always apply for a specific application with:

  $ export IBUS_DISCARD_PASSWORD_APPS="firefox"
  $ firefox 

  Again, when you enter text into a password input field, the problem will be
  solved.

  Test package is available here:

  https://launchpad.net/~mruffell/+archive/ubuntu/sf235370-test

  Please test with the revised version,
  1.5.17-3ubuntu4+sf235370v20190731b1.

  [Regression Potential]

  This change has a low risk of regression, because the default behaviour is
  unchanged. To be able to use the password input field discard functionality,
  a user has to explicitly set an environment variable for the specific 
process, or set a regex that matches a process name.

  This means the fix is 

[Touch-packages] [Bug 1838358] Re: Ibus causes gnome-shell to freeze when password fields are selected in Firefox

2019-07-31 Thread Matthew Ruffell
** Description changed:

  [Impact]
  
  The following has been seen in a VMware Horizon VDI. I cannot reproduce this
  issue myself.
  
  When a user interacts with any password field in Firefox, gnome-shell
  and Firefox both freeze and the system becomes unusable. If you ssh into
  the system and terminate Firefox, gnome-shell unfreezes.
  
- This only happens when the environment variable GTK_IM_MODULE is set to 
"ibus". If you unset the variable, or change it to 
+ This only happens when the environment variable GTK_IM_MODULE is set to 
"ibus". If you unset the variable, or change it to
  GTK_IM_MODULE=gtk-im-context-simple and then start Firefox, everything works 
as intended.
  
  This has been seen before with gnome-shell 3.28.4-0ubuntu18.04.1, ibus
  1.5.17-3ubuntu4 and Firefox versions starting with 
68.0+build3-0ubuntu0.18.04.1
  
  Note: Chrome[ium] and other applications do not trigger it, and it cannot be
  reproduced in other desktop environments.
  
  This seems to be an interaction issue between ibus and gnome-shell.
  
  [Fix]
  
  When ibus is built with the patch ibus-xx-f19-password.patch which was
  dropped in ibus-1.5.17-2, the problem is solved.
  
- ibus-xx-f19-password.patch checks to see if the GTK version is at or
- above 3.6, and if it is, checks to see if the input purpose is for a
- password field. If it is, then no further action is taken by ibus.
+ Instead of using ibus-xx-f19-password.patch, we will instead fix it with
+ upstream commit f328fd67f479faa46ca87bf3c85eed7080ec5ec0:
+ 
+ https://github.com/ibus/ibus/commit/f328fd67f479faa46ca87bf3c85eed7080ec5ec0
+ 
+ Subject: client/gtk2: Add IBUS_DISCARD_PASSWORD for firefox and chrome 
+ Author: fujiwarat 
+ 
+ This implements the password discard functionality found in 
+ ibus-xx-f19-password.patch and places it behind two environment variables,
+ IBUS_DISCARD_PASSWORD and IBUS_DISCARD_PASSWORD_APPS.
+ 
+ IBUS_DISCARD_PASSWORD is for a single process, and IBUS_DISCARD_PASSWORD_APPS
+ lets you set a regex of process names to filter and enable the fix for.
+ 
+ If IBUS_DISCARD_PASSWORD is set or IBUS_DISCARD_PASSWORD_APPS is set with the
+ process name which input is being placed into password fields, ibus will pass
+ through the input to the application without any processing.
  
  [Testcase]
  
  Launch firefox from within a gnome-session, making sure the
  GTK_IM_MODULE is set to "ibus". Note, this is the default value.
  
  $ env GTK_IM_MODULE="ibus" firefox
  
  Navigate to any website which has a password field. Wikipedia or Reddit
  will do.
  
  Click a password field and attempt to enter text. Firefox and gnome-
  shell both lock up and stay frozen for an extended period of time.
  
- With the test package which includes ibus-xx-f19-password.patch, gnome-shell
- and Firefox do not lock up and everything works as intended.
+ Now, try it with the fix by enabling:
+ 
+ $ env IBUS_DISCARD_PASSWORD=1 firefox
+ 
+ When you enter text into a password field, ibus should directly pass through
+ the text and the problem will be solved.
+ 
+ We can also ask it to always apply for a specific application with:
+ 
+ $ export IBUS_DISCARD_PASSWORD_APPS="firefox"
+ $ firefox 
+ 
+ Again, when you enter text into a password input field, the problem will be
+ solved.
  
  Test package is available here:
  
  https://launchpad.net/~mruffell/+archive/ubuntu/sf235370-test
  
+ Please test with the revised version,
+ 1.5.17-3ubuntu4+sf235370v20190731b1.
+ 
  [Regression Potential]
  
- This has a low to medium risk of regression, and is limited to inputs to
- password fields in all applications, including the gnome-shell lock screen.
+ This change has a low risk of regression, because the default behaviour is
+ unchanged. To be able to use the password input field discard functionality,
+ a user has to explicitly set an environment variable for the specific 
process, or set a regex that matches a process name.
  
- This patch has been extensively tested and it is present in the
- following Ubuntu releases:
+ This means the fix is not enabled by default on any machines, and will
+ only be utilised by those suffering problems and go and manually set
+ environment variables or have their system administrator enable the
+ environment variables permanently.
  
- artful: 1.5.14-2ubuntu1
- zesty: 1.5.14-2ubuntu1
- yakkety: 1.5.11-1ubuntu3
- xenial: 1.5.11-1ubuntu2
- wily: 1.5.10-1ubuntu1
- vivid: 1.5.9-1ubuntu3
- utopic: 1.5.8-2ubuntu2
- trusty: 1.5.5-1ubuntu3.2
+ This commit is present in upstream ibus from version ibus-1.5.19 onward, and
+ is currently present in cosmic, disco and eoan. 
  
- The patch was introduced in trusty, and removed in bionic. I feel that the 
risk of reintroducing the patch is low, since the use case of the software is 
the same as previous releases in regards to input software and input language
- selection.
- 
- I still acknowledge a potential risk of regression when users change their
- input engines to non 

[Touch-packages] [Bug 1838358] Re: Ibus causes gnome-shell to freeze when password fields are selected in Firefox

2019-07-29 Thread Matthew Ruffell
Attached is the debdiff for bionic which restores ibus-
xx-f19-password.patch

** Patch added: "ibus debdiff for bionic"
   
https://bugs.launchpad.net/ubuntu/+source/ibus/+bug/1838358/+attachment/5279953/+files/lp1838358_bionic.debdiff

** Tags added: sts-sponsor

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to ibus in Ubuntu.
https://bugs.launchpad.net/bugs/1838358

Title:
  Ibus causes gnome-shell to freeze when password fields are selected in
  Firefox

Status in ibus package in Ubuntu:
  New
Status in ibus source package in Bionic:
  In Progress

Bug description:
  [Impact]

  The following has been seen in a VMware Horizon VDI. I cannot reproduce this
  issue myself.

  When a user interacts with any password field in Firefox, gnome-shell
  and Firefox both freeze and the system becomes unusable. If you ssh
  into the system and terminate Firefox, gnome-shell unfreezes.

  This only happens when the environment variable GTK_IM_MODULE is set to 
"ibus". If you unset the variable, or change it to 
  GTK_IM_MODULE=gtk-im-context-simple and then start Firefox, everything works 
as intended.

  This has been seen before with gnome-shell 3.28.4-0ubuntu18.04.1, ibus
  1.5.17-3ubuntu4 and Firefox versions starting with 
68.0+build3-0ubuntu0.18.04.1

  Note: Chrome[ium] and other applications do not trigger it, and it cannot be
  reproduced in other desktop environments.

  This seems to be an interaction issue between ibus and gnome-shell.

  [Fix]

  When ibus is built with the patch ibus-xx-f19-password.patch which was
  dropped in ibus-1.5.17-2, the problem is solved.

  ibus-xx-f19-password.patch checks to see if the GTK version is at or
  above 3.6, and if it is, checks to see if the input purpose is for a
  password field. If it is, then no further action is taken by ibus.

  [Testcase]

  Launch firefox from within a gnome-session, making sure the
  GTK_IM_MODULE is set to "ibus". Note, this is the default value.

  $ env GTK_IM_MODULE="ibus" firefox

  Navigate to any website which has a password field. Wikipedia or
  Reddit will do.

  Click a password field and attempt to enter text. Firefox and gnome-
  shell both lock up and stay frozen for an extended period of time.

  With the test package which includes ibus-xx-f19-password.patch, gnome-shell
  and Firefox do not lock up and everything works as intended.

  Test package is available here:

  https://launchpad.net/~mruffell/+archive/ubuntu/sf235370-test

  [Regression Potential]

  This has a low to medium risk of regression, and is limited to inputs to
  password fields in all applications, including the gnome-shell lock screen.

  This patch has been extensively tested and it is present in the
  following Ubuntu releases:

  artful: 1.5.14-2ubuntu1
  zesty: 1.5.14-2ubuntu1
  yakkety: 1.5.11-1ubuntu3
  xenial: 1.5.11-1ubuntu2
  wily: 1.5.10-1ubuntu1
  vivid: 1.5.9-1ubuntu3
  utopic: 1.5.8-2ubuntu2
  trusty: 1.5.5-1ubuntu3.2

  The patch was introduced in trusty, and removed in bionic. I feel that the 
risk of reintroducing the patch is low, since the use case of the software is 
the same as previous releases in regards to input software and input language
  selection.

  I still acknowledge a potential risk of regression when users change their
  input engines to non defaults and pair it with non default input languages.

  In the event of regression ibus can be temporarily be disabled via an
  environment variable and the patch dropped in any subsequent packages.

  [Notes]

  This patch will not be needed in newer versions of ibus as an official
  workaround has been implemented as of ibus-1.5.19.

  https://github.com/ibus/ibus/commit/f328fd67f479faa46ca87bf3c85eed7080ec5ec0

  If problems arise with password input fields in ibus versions 1.5.19
  or later, which are found in cosmic onward, environment variables can
  be set that have the same effect as ibus-xx-f19-password.patch.

  env IBUS_DISCARD_PASSWORD=1 firefox
  or
  export IBUS_DISCARD_PASSWORD_APPS='firefox,.*chrome.*'

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/ibus/+bug/1838358/+subscriptions

-- 
Mailing list: https://launchpad.net/~touch-packages
Post to : touch-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~touch-packages
More help   : https://help.launchpad.net/ListHelp


[Touch-packages] [Bug 1838358] [NEW] Ibus causes gnome-shell to freeze when password fields are selected in Firefox

2019-07-29 Thread Matthew Ruffell
Public bug reported:

[Impact]

The following has been seen in a VMware Horizon VDI. I cannot reproduce this
issue myself.

When a user interacts with any password field in Firefox, gnome-shell
and Firefox both freeze and the system becomes unusable. If you ssh into
the system and terminate Firefox, gnome-shell unfreezes.

This only happens when the environment variable GTK_IM_MODULE is set to "ibus". 
If you unset the variable, or change it to 
GTK_IM_MODULE=gtk-im-context-simple and then start Firefox, everything works as 
intended.

This has been seen before with gnome-shell 3.28.4-0ubuntu18.04.1, ibus
1.5.17-3ubuntu4 and Firefox versions starting with 68.0+build3-0ubuntu0.18.04.1

Note: Chrome[ium] and other applications do not trigger it, and it cannot be
reproduced in other desktop environments.

This seems to be an interaction issue between ibus and gnome-shell.

[Fix]

When ibus is built with the patch ibus-xx-f19-password.patch which was
dropped in ibus-1.5.17-2, the problem is solved.

ibus-xx-f19-password.patch checks to see if the GTK version is at or
above 3.6, and if it is, checks to see if the input purpose is for a
password field. If it is, then no further action is taken by ibus.

[Testcase]

Launch firefox from within a gnome-session, making sure the
GTK_IM_MODULE is set to "ibus". Note, this is the default value.

$ env GTK_IM_MODULE="ibus" firefox

Navigate to any website which has a password field. Wikipedia or Reddit
will do.

Click a password field and attempt to enter text. Firefox and gnome-
shell both lock up and stay frozen for an extended period of time.

With the test package which includes ibus-xx-f19-password.patch, gnome-shell
and Firefox do not lock up and everything works as intended.

Test package is available here:

https://launchpad.net/~mruffell/+archive/ubuntu/sf235370-test

[Regression Potential]

This has a low to medium risk of regression, and is limited to inputs to
password fields in all applications, including the gnome-shell lock screen.

This patch has been extensively tested and it is present in the
following Ubuntu releases:

artful: 1.5.14-2ubuntu1
zesty: 1.5.14-2ubuntu1
yakkety: 1.5.11-1ubuntu3
xenial: 1.5.11-1ubuntu2
wily: 1.5.10-1ubuntu1
vivid: 1.5.9-1ubuntu3
utopic: 1.5.8-2ubuntu2
trusty: 1.5.5-1ubuntu3.2

The patch was introduced in trusty, and removed in bionic. I feel that the risk 
of reintroducing the patch is low, since the use case of the software is the 
same as previous releases in regards to input software and input language
selection.

I still acknowledge a potential risk of regression when users change their
input engines to non defaults and pair it with non default input languages.

In the event of regression ibus can be temporarily be disabled via an
environment variable and the patch dropped in any subsequent packages.

[Notes]

This patch will not be needed in newer versions of ibus as an official
workaround has been implemented as of ibus-1.5.19.

https://github.com/ibus/ibus/commit/f328fd67f479faa46ca87bf3c85eed7080ec5ec0

If problems arise with password input fields in ibus versions 1.5.19 or
later, which are found in cosmic onward, environment variables can be
set that have the same effect as ibus-xx-f19-password.patch.

env IBUS_DISCARD_PASSWORD=1 firefox
or
export IBUS_DISCARD_PASSWORD_APPS='firefox,.*chrome.*'

** Affects: ibus (Ubuntu)
 Importance: Undecided
 Status: New

** Affects: ibus (Ubuntu Bionic)
 Importance: Medium
 Assignee: Matthew Ruffell (mruffell)
 Status: In Progress


** Tags: sts

** Also affects: ibus (Ubuntu Bionic)
   Importance: Undecided
   Status: New

** Changed in: ibus (Ubuntu Bionic)
   Status: New => In Progress

** Changed in: ibus (Ubuntu Bionic)
 Assignee: (unassigned) => Matthew Ruffell (mruffell)

** Changed in: ibus (Ubuntu Bionic)
   Importance: Undecided => Medium

** Description changed:

  [Impact]
  
  The following has been seen in a VMware Horizon VDI. I cannot reproduce this
  issue myself.
  
- When a user interacts with any password field in Firefox, gnome-shell and 
Firefox
- both freeze and the system becomes unusable. If you ssh into the system and
- terminate Firefox, gnome-shell unfreezes.
+ When a user interacts with any password field in Firefox, gnome-shell
+ and Firefox both freeze and the system becomes unusable. If you ssh into
+ the system and terminate Firefox, gnome-shell unfreezes.
  
- This only happens when the environment variable GTK_IM_MODULE is set to 
"ibus".
- If you unset the variable, or change it to GTK_IM_MODULE=gtk-im-context-simple
- and then start Firefox, everything works as intended.
+ This only happens when the environment variable GTK_IM_MODULE is set to 
"ibus". If you unset the variable, or change it to 
+ GTK_IM_MODULE=gtk-im-context-simple and then start Firefox, everything works 
as intended.
  
- This has been seen before with gnome-shell 3.28.4-0

[Touch-packages] [Bug 1827172] Re: update-rc.d: enabling or disabling S runlevel services incorrectly modifies runlevel

2019-06-06 Thread Matthew Ruffell
Attached is the final debdiff for release, mostly for having a record of
it.

I have built it in a personal ppa, tested it and verified it works.

** Patch added: "sysvinit debdiff for trusty esm"
   
https://bugs.launchpad.net/ubuntu/+source/sysvinit/+bug/1827172/+attachment/5269216/+files/lp1827172_trusty_esm.debdiff

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to sysvinit in Ubuntu.
https://bugs.launchpad.net/bugs/1827172

Title:
  update-rc.d: enabling or disabling S runlevel services incorrectly
  modifies runlevel

Status in sysvinit package in Ubuntu:
  New
Status in sysvinit source package in Trusty:
  In Progress

Bug description:
  [Impact]

   * update-rc.d, in sysv-rc-2.88dsf-41ubuntu6.3 is broken in trusty.

   * update-rc.d incorrectly modifies symlinks when enabling or disabling
     services which are started on the "S" runlevel.

   * This can lead to services being changed from S runlevel from where they
     would be started on boot, to "0" runlevel, and are run on halt, which is
     incorrect.

   * The bug is caused by trying to use the runlevel to index into an integer
     array of runlevels. When the runlevel in question is "S", an error is
     printed

     Argument "S" isn't numeric in array element at /usr/sbin/update-rc.d line
     232.

     Perl then sets the index to default to 0, which changes the
  runlevel.

   * The fix is to check if the runlevel is "S", and if it is, set the index to
     99 which conforms with other expected usages for the "S" runlevel in the
     script. See the "startstop" and "makelinks" subroutines.

  [Test Case]

  * You can reproduce this with any service that is started on the "S"
    runlevel. We will use open-iscsi for an example.

  1) Install open-iscsi

  $ sudo apt install open-iscsi

  2) Check to see symlinks for init.d scripts are set to defaults:

  root@trusty-openiscsi:/etc# ls -l /etc/rc[0123456S].d/*iscsi*
  /etc/rc0.d/K80umountiscsi.sh -> ../init.d/umountiscsi.sh
  /etc/rc0.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc1.d/K80umountiscsi.sh -> ../init.d/umountiscsi.sh
  /etc/rc1.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc6.d/K80umountiscsi.sh -> ../init.d/umountiscsi.sh
  /etc/rc6.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rcS.d/S45open-iscsi -> ../init.d/open-iscsi

  3) Use update-rc.d to enable open-iscsi service

  root@trusty-openiscsi:/etc# update-rc.d open-iscsi enable
  update-rc.d: warning: start runlevel arguments (none) do not match open-iscsi 
Default-Start values (S)
  update-rc.d: warning: stop runlevel arguments (none) do not match open-iscsi 
Default-Stop values (0 1 6)
  Argument "S" isn't numeric in array element at /usr/sbin/update-rc.d line 232.
  Enabling system startup links for /etc/init.d/open-iscsi ...
  Removing any system startup links for /etc/init.d/open-iscsi ...
  /etc/rc0.d/K81open-iscsi
  /etc/rc1.d/K81open-iscsi
  /etc/rc6.d/K81open-iscsi
  /etc/rcS.d/S45open-iscsi
  Adding system startup for /etc/init.d/open-iscsi ...
  /etc/rc0.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc1.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc6.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc0.d/S45open-iscsi -> ../init.d/open-iscsi

  * The problem is the "/etc/rcS.d/S45open-iscsi" symlink is incorrectly
    changed to "/etc/rc0.d/S45open-iscsi".

  * Instead, the correct behaviour is to keep the symlink in /etc/rcS.d/
    intact:

  root@trusty-openiscsi:/etc# update-rc.d open-iscsi enable
  update-rc.d: warning: start runlevel arguments (none) do not match open-iscsi 
Default-Start values (S)
  update-rc.d: warning: stop runlevel arguments (none) do not match open-iscsi 
Default-Stop values (0 1 6)
  Enabling system startup links for /etc/init.d/open-iscsi ...
  Removing any system startup links for /etc/init.d/open-iscsi ...
  /etc/rc0.d/K81open-iscsi
  /etc/rc1.d/K81open-iscsi
  /etc/rc6.d/K81open-iscsi
  /etc/rcS.d/S45open-iscsi
  Adding system startup for /etc/init.d/open-iscsi ...
  /etc/rc0.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc1.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc6.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rcS.d/S45open-iscsi -> ../init.d/open-iscsi

  [Regression Potential]

   * There is only one file modified, which is the update-rc.d perl script.
     Worst case scenario is that users cannot enable or disable their services,
     or a symlink is changed such that a service is started / stopped on an
     incorrect runlevel.

   * If a regression happens, any damage should be easily spotted by a
     sysadmin and can be manually repaired by making manual symlinks with
     "ln -s".

   * I have built and tested the change in a PPA, which you can find here:
 https://launchpad.net/~mruffell/+archive/ubuntu/sysvinit-testing

   * My only cause for concern is that if a regression does happen, it may 
 impact packages that run "update-rc.d  [en|dis]able" in a 
 postinstall 

[Touch-packages] [Bug 1827172] Re: update-rc.d: enabling or disabling S runlevel services incorrectly modifies runlevel

2019-05-02 Thread Matthew Ruffell
** Tags added: sts-sponsor

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to sysvinit in Ubuntu.
https://bugs.launchpad.net/bugs/1827172

Title:
  update-rc.d: enabling or disabling S runlevel services incorrectly
  modifies runlevel

Status in sysvinit package in Ubuntu:
  New
Status in sysvinit source package in Trusty:
  In Progress

Bug description:
  [Impact]

   * update-rc.d, in sysv-rc-2.88dsf-41ubuntu6.3 is broken in trusty.

   * update-rc.d incorrectly modifies symlinks when enabling or disabling
     services which are started on the "S" runlevel.

   * This can lead to services being changed from S runlevel from where they
     would be started on boot, to "0" runlevel, and are run on halt, which is
     incorrect.

   * The bug is caused by trying to use the runlevel to index into an integer
     array of runlevels. When the runlevel in question is "S", an error is
     printed

     Argument "S" isn't numeric in array element at /usr/sbin/update-rc.d line
     232.

     Perl then sets the index to default to 0, which changes the
  runlevel.

   * The fix is to check if the runlevel is "S", and if it is, set the index to
     99 which conforms with other expected usages for the "S" runlevel in the
     script. See the "startstop" and "makelinks" subroutines.

  [Test Case]

  * You can reproduce this with any service that is started on the "S"
    runlevel. We will use open-iscsi for an example.

  1) Install open-iscsi

  $ sudo apt install open-iscsi

  2) Check to see symlinks for init.d scripts are set to defaults:

  root@trusty-openiscsi:/etc# ls -l /etc/rc[0123456S].d/*iscsi*
  /etc/rc0.d/K80umountiscsi.sh -> ../init.d/umountiscsi.sh
  /etc/rc0.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc1.d/K80umountiscsi.sh -> ../init.d/umountiscsi.sh
  /etc/rc1.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc6.d/K80umountiscsi.sh -> ../init.d/umountiscsi.sh
  /etc/rc6.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rcS.d/S45open-iscsi -> ../init.d/open-iscsi

  3) Use update-rc.d to enable open-iscsi service

  root@trusty-openiscsi:/etc# update-rc.d open-iscsi enable
  update-rc.d: warning: start runlevel arguments (none) do not match open-iscsi 
Default-Start values (S)
  update-rc.d: warning: stop runlevel arguments (none) do not match open-iscsi 
Default-Stop values (0 1 6)
  Argument "S" isn't numeric in array element at /usr/sbin/update-rc.d line 232.
  Enabling system startup links for /etc/init.d/open-iscsi ...
  Removing any system startup links for /etc/init.d/open-iscsi ...
  /etc/rc0.d/K81open-iscsi
  /etc/rc1.d/K81open-iscsi
  /etc/rc6.d/K81open-iscsi
  /etc/rcS.d/S45open-iscsi
  Adding system startup for /etc/init.d/open-iscsi ...
  /etc/rc0.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc1.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc6.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc0.d/S45open-iscsi -> ../init.d/open-iscsi

  * The problem is the "/etc/rcS.d/S45open-iscsi" symlink is incorrectly
    changed to "/etc/rc0.d/S45open-iscsi".

  * Instead, the correct behaviour is to keep the symlink in /etc/rcS.d/
    intact:

  root@trusty-openiscsi:/etc# update-rc.d open-iscsi enable
  update-rc.d: warning: start runlevel arguments (none) do not match open-iscsi 
Default-Start values (S)
  update-rc.d: warning: stop runlevel arguments (none) do not match open-iscsi 
Default-Stop values (0 1 6)
  Enabling system startup links for /etc/init.d/open-iscsi ...
  Removing any system startup links for /etc/init.d/open-iscsi ...
  /etc/rc0.d/K81open-iscsi
  /etc/rc1.d/K81open-iscsi
  /etc/rc6.d/K81open-iscsi
  /etc/rcS.d/S45open-iscsi
  Adding system startup for /etc/init.d/open-iscsi ...
  /etc/rc0.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc1.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc6.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rcS.d/S45open-iscsi -> ../init.d/open-iscsi

  [Regression Potential]

   * There is only one file modified, which is the update-rc.d perl script.
     Worst case scenario is that users cannot enable or disable their services,
     or a symlink is changed such that a service is started / stopped on an
     incorrect runlevel.

   * If a regression happens, any damage should be easily spotted by a
     sysadmin and can be manually repaired by making manual symlinks with
     "ln -s".

   * I have built and tested the change in a PPA, which you can find here:
 https://launchpad.net/~mruffell/+archive/ubuntu/sysvinit-testing

   * My only cause for concern is that if a regression does happen, it may 
 impact packages that run "update-rc.d  [en|dis]able" in a 
 postinstall module, although I haven't managed to find an example that
 does this, since most use the "defaults" command instead.

  [Other Info]

   * trusty is the last Ubuntu release to use sysvinit, and this bug is not
     present in newer versions since they use systemd, and the code in question

[Touch-packages] [Bug 1827172] Re: update-rc.d: enabling or disabling S runlevel services incorrectly modifies runlevel

2019-05-01 Thread Matthew Ruffell
This debdiff contains the patch for trusty. I have tested it, built it
in a ppa and run autopkgtest, and things look fine to me.

** Patch added: "sysvinit debdiff for trusty"
   
https://bugs.launchpad.net/ubuntu/+source/sysvinit/+bug/1827172/+attachment/5260712/+files/lp1827172_trusty.debdiff

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to sysvinit in Ubuntu.
https://bugs.launchpad.net/bugs/1827172

Title:
  update-rc.d: enabling or disabling S runlevel services incorrectly
  modifies runlevel

Status in sysvinit package in Ubuntu:
  New
Status in sysvinit source package in Trusty:
  In Progress

Bug description:
  [Impact]

   * update-rc.d, in sysv-rc-2.88dsf-41ubuntu6.3 is broken in trusty.

   * update-rc.d incorrectly modifies symlinks when enabling or disabling
     services which are started on the "S" runlevel.

   * This can lead to services being changed from S runlevel from where they
     would be started on boot, to "0" runlevel, and are run on halt, which is
     incorrect.

   * The bug is caused by trying to use the runlevel to index into an integer
     array of runlevels. When the runlevel in question is "S", an error is
     printed

     Argument "S" isn't numeric in array element at /usr/sbin/update-rc.d line
     232.

     Perl then sets the index to default to 0, which changes the
  runlevel.

   * The fix is to check if the runlevel is "S", and if it is, set the index to
     99 which conforms with other expected usages for the "S" runlevel in the
     script. See the "startstop" and "makelinks" subroutines.

  [Test Case]

  * You can reproduce this with any service that is started on the "S"
    runlevel. We will use open-iscsi for an example.

  1) Install open-iscsi

  $ sudo apt install open-iscsi

  2) Check to see symlinks for init.d scripts are set to defaults:

  root@trusty-openiscsi:/etc# ls -l /etc/rc[0123456S].d/*iscsi*
  /etc/rc0.d/K80umountiscsi.sh -> ../init.d/umountiscsi.sh
  /etc/rc0.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc1.d/K80umountiscsi.sh -> ../init.d/umountiscsi.sh
  /etc/rc1.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc6.d/K80umountiscsi.sh -> ../init.d/umountiscsi.sh
  /etc/rc6.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rcS.d/S45open-iscsi -> ../init.d/open-iscsi

  3) Use update-rc.d to enable open-iscsi service

  root@trusty-openiscsi:/etc# update-rc.d open-iscsi enable
  update-rc.d: warning: start runlevel arguments (none) do not match open-iscsi 
Default-Start values (S)
  update-rc.d: warning: stop runlevel arguments (none) do not match open-iscsi 
Default-Stop values (0 1 6)
  Argument "S" isn't numeric in array element at /usr/sbin/update-rc.d line 232.
  Enabling system startup links for /etc/init.d/open-iscsi ...
  Removing any system startup links for /etc/init.d/open-iscsi ...
  /etc/rc0.d/K81open-iscsi
  /etc/rc1.d/K81open-iscsi
  /etc/rc6.d/K81open-iscsi
  /etc/rcS.d/S45open-iscsi
  Adding system startup for /etc/init.d/open-iscsi ...
  /etc/rc0.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc1.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc6.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc0.d/S45open-iscsi -> ../init.d/open-iscsi

  * The problem is the "/etc/rcS.d/S45open-iscsi" symlink is incorrectly
    changed to "/etc/rc0.d/S45open-iscsi".

  * Instead, the correct behaviour is to keep the symlink in /etc/rcS.d/
    intact:

  root@trusty-openiscsi:/etc# update-rc.d open-iscsi enable
  update-rc.d: warning: start runlevel arguments (none) do not match open-iscsi 
Default-Start values (S)
  update-rc.d: warning: stop runlevel arguments (none) do not match open-iscsi 
Default-Stop values (0 1 6)
  Enabling system startup links for /etc/init.d/open-iscsi ...
  Removing any system startup links for /etc/init.d/open-iscsi ...
  /etc/rc0.d/K81open-iscsi
  /etc/rc1.d/K81open-iscsi
  /etc/rc6.d/K81open-iscsi
  /etc/rcS.d/S45open-iscsi
  Adding system startup for /etc/init.d/open-iscsi ...
  /etc/rc0.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc1.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc6.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rcS.d/S45open-iscsi -> ../init.d/open-iscsi

  [Regression Potential]

   * There is only one file modified, which is the update-rc.d perl script.
     Worst case scenario is that users cannot enable or disable their services,
     or a symlink is changed such that a service is started / stopped on an
     incorrect runlevel.

   * If a regression happens, any damage should be easily spotted by a
     sysadmin and can be manually repaired by making manual symlinks with
     "ln -s".

   * I have built and tested the change in a PPA, which you can find here:
 https://launchpad.net/~mruffell/+archive/ubuntu/sysvinit-testing

   * My only cause for concern is that if a regression does happen, it may 
 impact packages that run "update-rc.d  [en|dis]able" in a 
 postinstall module, although I 

[Touch-packages] [Bug 1827172] Re: update-rc.d: enabling or disabling S runlevel services incorrectly modifies runlevel

2019-05-01 Thread Matthew Ruffell
I have developed a patch which I believe fixes the problem.

https://paste.ubuntu.com/p/QRw4XTkXkV/

** Description changed:

  [Impact]
  
   * update-rc.d, in sysv-rc-2.88dsf-41ubuntu6.3 is broken in trusty.
  
-  * update-rc.d incorrectly modifies symlinks when enabling or disabling  
-services which are started on the "S" runlevel.
+  * update-rc.d incorrectly modifies symlinks when enabling or disabling
+    services which are started on the "S" runlevel.
  
-  * This can lead to services being changed from S runlevel from where they 
-would be started on boot, to "0" runlevel, and are run on halt, which is 
-incorrect.
+  * This can lead to services being changed from S runlevel from where they
+    would be started on boot, to "0" runlevel, and are run on halt, which is
+    incorrect.
  
   * The bug is caused by trying to use the runlevel to index into an integer
-    array of runlevels. When the runlevel in question is "S", an error is 
-printed
+    array of runlevels. When the runlevel in question is "S", an error is
+    printed
  
-Argument "S" isn't numeric in array element at /usr/sbin/update-rc.d line 
-232.
+    Argument "S" isn't numeric in array element at /usr/sbin/update-rc.d line
+    232.
  
     Perl then sets the index to default to 0, which changes the runlevel.
  
-  * The fix is to check if the runlevel is "S", and if it is, set the index to 
 
-99 which conforms with other expected usages for the "S" runlevel in the 
-script. See the "startstop" and "makelinks" subroutines.
+  * The fix is to check if the runlevel is "S", and if it is, set the index to
+    99 which conforms with other expected usages for the "S" runlevel in the
+    script. See the "startstop" and "makelinks" subroutines.
  
  [Test Case]
  
- * You can reproduce this with any service that is started on the "S" 
-   runlevel. We will use open-iscsi for an example.
+ * You can reproduce this with any service that is started on the "S"
+   runlevel. We will use open-iscsi for an example.
  
  1) Install open-iscsi
  
  $ sudo apt install open-iscsi
  
  2) Check to see symlinks for init.d scripts are set to defaults:
  
  root@trusty-openiscsi:/etc# ls -l /etc/rc[0123456S].d/*iscsi*
  /etc/rc0.d/K80umountiscsi.sh -> ../init.d/umountiscsi.sh
  /etc/rc0.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc1.d/K80umountiscsi.sh -> ../init.d/umountiscsi.sh
  /etc/rc1.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc6.d/K80umountiscsi.sh -> ../init.d/umountiscsi.sh
  /etc/rc6.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rcS.d/S45open-iscsi -> ../init.d/open-iscsi
  
  3) Use update-rc.d to enable open-iscsi service
  
  root@trusty-openiscsi:/etc# update-rc.d open-iscsi enable
  update-rc.d: warning: start runlevel arguments (none) do not match open-iscsi 
Default-Start values (S)
  update-rc.d: warning: stop runlevel arguments (none) do not match open-iscsi 
Default-Stop values (0 1 6)
  Argument "S" isn't numeric in array element at /usr/sbin/update-rc.d line 232.
  Enabling system startup links for /etc/init.d/open-iscsi ...
  Removing any system startup links for /etc/init.d/open-iscsi ...
  /etc/rc0.d/K81open-iscsi
  /etc/rc1.d/K81open-iscsi
  /etc/rc6.d/K81open-iscsi
  /etc/rcS.d/S45open-iscsi
  Adding system startup for /etc/init.d/open-iscsi ...
  /etc/rc0.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc1.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc6.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc0.d/S45open-iscsi -> ../init.d/open-iscsi
  
- * The problem is the "/etc/rcS.d/S45open-iscsi" symlink is incorrectly 
-   changed to "/etc/rc0.d/S45open-iscsi".
+ * The problem is the "/etc/rcS.d/S45open-iscsi" symlink is incorrectly
+   changed to "/etc/rc0.d/S45open-iscsi".
  
- * Instead, the correct behaviour is to keep the symlink in /etc/rcS.d/ 
-   intact:
+ * Instead, the correct behaviour is to keep the symlink in /etc/rcS.d/
+   intact:
  
  root@trusty-openiscsi:/etc# update-rc.d open-iscsi enable
  update-rc.d: warning: start runlevel arguments (none) do not match open-iscsi 
Default-Start values (S)
  update-rc.d: warning: stop runlevel arguments (none) do not match open-iscsi 
Default-Stop values (0 1 6)
  Enabling system startup links for /etc/init.d/open-iscsi ...
  Removing any system startup links for /etc/init.d/open-iscsi ...
  /etc/rc0.d/K81open-iscsi
  /etc/rc1.d/K81open-iscsi
  /etc/rc6.d/K81open-iscsi
  /etc/rcS.d/S45open-iscsi
  Adding system startup for /etc/init.d/open-iscsi ...
  /etc/rc0.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc1.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc6.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rcS.d/S45open-iscsi -> ../init.d/open-iscsi
  
  [Regression Potential]
  
-  * There is only one file modified, which is the update-rc.d perl script. 
-Worst case scenario is that users cannot enable or disable their services, 
-or a symlink is changed such that a service is started / stopped on an 
-

[Touch-packages] [Bug 1827172] Re: update-rc.d: enabling or disabling S runlevel services incorrectly modifies runlevel

2019-04-30 Thread Matthew Ruffell
** Description changed:

  [Impact]
  
-  * update-rc.d, in sysv-rc-2.88dsf-41ubuntu6.3 is broken in trusty.
+  * update-rc.d, in sysv-rc-2.88dsf-41ubuntu6.3 is broken in trusty.
  
-  * update-rc.d incorrectly modifies symlinks when enabling or disabling 
services
-which are started on the "S" runlevel.
+  * update-rc.d incorrectly modifies symlinks when enabling or disabling  
+services which are started on the "S" runlevel.
  
-  * This can lead to services being changed from S runlevel from where they 
would
-be started on boot, to "0" runlevel, and are run on halt, which is 
incorrect.
+  * This can lead to services being changed from S runlevel from where they 
+would be started on boot, to "0" runlevel, and are run on halt, which is 
+incorrect.
  
-  * The bug is caused by trying to use the runlevel to index into an integer
-array of runlevels. When the runlevel in question is "S", an error is 
printed
-
-Argument "S" isn't numeric in array element at /usr/sbin/update-rc.d line 
232.
+  * The bug is caused by trying to use the runlevel to index into an integer
+    array of runlevels. When the runlevel in question is "S", an error is 
+printed
  
-Perl then sets the index to default to 0, which changes the runlevel.
+Argument "S" isn't numeric in array element at /usr/sbin/update-rc.d line 
+232.
  
-  * The fix is to check if the runlevel is "S", and if it is, set the index to 
99
-which conforms with other expected usages for the "S" runlevel in the 
script.
-See the "startstop" and "makelinks" subroutines.
+    Perl then sets the index to default to 0, which changes the runlevel.
+ 
+  * The fix is to check if the runlevel is "S", and if it is, set the index to 
 
+99 which conforms with other expected usages for the "S" runlevel in the 
+script. See the "startstop" and "makelinks" subroutines.
  
  [Test Case]
  
- * You can reproduce this with any service that is started on the "S" runlevel.
-   We will use open-iscsi for an example.
+ * You can reproduce this with any service that is started on the "S" 
+   runlevel. We will use open-iscsi for an example.
  
  1) Install open-iscsi
  
  $ sudo apt install open-iscsi
  
  2) Check to see symlinks for init.d scripts are set to defaults:
  
  root@trusty-openiscsi:/etc# ls -l /etc/rc[0123456S].d/*iscsi*
- lrwxrwxrwx 1 root root 24 Apr 29 20:56 /etc/rc0.d/K80umountiscsi.sh -> 
../init.d/umountiscsi.sh
- lrwxrwxrwx 1 root root 20 Apr 29 20:56 /etc/rc0.d/K81open-iscsi -> 
../init.d/open-iscsi
- lrwxrwxrwx 1 root root 24 Apr 29 20:56 /etc/rc1.d/K80umountiscsi.sh -> 
../init.d/umountiscsi.sh
- lrwxrwxrwx 1 root root 20 Apr 29 20:56 /etc/rc1.d/K81open-iscsi -> 
../init.d/open-iscsi
- lrwxrwxrwx 1 root root 24 Apr 29 20:56 /etc/rc6.d/K80umountiscsi.sh -> 
../init.d/umountiscsi.sh
- lrwxrwxrwx 1 root root 20 Apr 29 20:56 /etc/rc6.d/K81open-iscsi -> 
../init.d/open-iscsi
- lrwxrwxrwx 1 root root 20 Apr 29 20:56 /etc/rcS.d/S45open-iscsi -> 
../init.d/open-iscsi
+ /etc/rc0.d/K80umountiscsi.sh -> ../init.d/umountiscsi.sh
+ /etc/rc0.d/K81open-iscsi -> ../init.d/open-iscsi
+ /etc/rc1.d/K80umountiscsi.sh -> ../init.d/umountiscsi.sh
+ /etc/rc1.d/K81open-iscsi -> ../init.d/open-iscsi
+ /etc/rc6.d/K80umountiscsi.sh -> ../init.d/umountiscsi.sh
+ /etc/rc6.d/K81open-iscsi -> ../init.d/open-iscsi
+ /etc/rcS.d/S45open-iscsi -> ../init.d/open-iscsi
  
  3) Use update-rc.d to enable open-iscsi service
  
  root@trusty-openiscsi:/etc# update-rc.d open-iscsi enable
  update-rc.d: warning: start runlevel arguments (none) do not match open-iscsi 
Default-Start values (S)
  update-rc.d: warning: stop runlevel arguments (none) do not match open-iscsi 
Default-Stop values (0 1 6)
  Argument "S" isn't numeric in array element at /usr/sbin/update-rc.d line 232.
  Enabling system startup links for /etc/init.d/open-iscsi ...
  Removing any system startup links for /etc/init.d/open-iscsi ...
  /etc/rc0.d/K81open-iscsi
  /etc/rc1.d/K81open-iscsi
  /etc/rc6.d/K81open-iscsi
  /etc/rcS.d/S45open-iscsi
  Adding system startup for /etc/init.d/open-iscsi ...
  /etc/rc0.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc1.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc6.d/K81open-iscsi -> ../init.d/open-iscsi
  /etc/rc0.d/S45open-iscsi -> ../init.d/open-iscsi
  
- * The problem is the "/etc/rcS.d/S45open-iscsi" symlink is incorrectly 
changed to
-   "/etc/rc0.d/S45open-iscsi".
+ * The problem is the "/etc/rcS.d/S45open-iscsi" symlink is incorrectly 
+   changed to "/etc/rc0.d/S45open-iscsi".
  
- * Instead, the correct behaviour is to keep the symlink in /etc/rcS.d/
- intact:
+ * Instead, the correct behaviour is to keep the symlink in /etc/rcS.d/ 
+   intact:
  
  root@trusty-openiscsi:/etc# update-rc.d open-iscsi enable
  update-rc.d: warning: start runlevel arguments (none) do not match open-iscsi 
Default-Start values (S)
  update-rc.d: warning: stop runlevel arguments (none) do not match open-iscsi 
Default-Stop values 

[Touch-packages] [Bug 1827172] [NEW] update-rc.d: enabling or disabling S runlevel services incorrectly modifies runlevel

2019-04-30 Thread Matthew Ruffell
Public bug reported:

[Impact]

 * update-rc.d, in sysv-rc-2.88dsf-41ubuntu6.3 is broken in trusty.

 * update-rc.d incorrectly modifies symlinks when enabling or disabling  
   services which are started on the "S" runlevel.

 * This can lead to services being changed from S runlevel from where they 
   would be started on boot, to "0" runlevel, and are run on halt, which is 
   incorrect.

 * The bug is caused by trying to use the runlevel to index into an integer
   array of runlevels. When the runlevel in question is "S", an error is 
   printed

   Argument "S" isn't numeric in array element at /usr/sbin/update-rc.d line 
   232.

   Perl then sets the index to default to 0, which changes the runlevel.

 * The fix is to check if the runlevel is "S", and if it is, set the index to  
   99 which conforms with other expected usages for the "S" runlevel in the 
   script. See the "startstop" and "makelinks" subroutines.

[Test Case]

* You can reproduce this with any service that is started on the "S" 
  runlevel. We will use open-iscsi for an example.

1) Install open-iscsi

$ sudo apt install open-iscsi

2) Check to see symlinks for init.d scripts are set to defaults:

root@trusty-openiscsi:/etc# ls -l /etc/rc[0123456S].d/*iscsi*
/etc/rc0.d/K80umountiscsi.sh -> ../init.d/umountiscsi.sh
/etc/rc0.d/K81open-iscsi -> ../init.d/open-iscsi
/etc/rc1.d/K80umountiscsi.sh -> ../init.d/umountiscsi.sh
/etc/rc1.d/K81open-iscsi -> ../init.d/open-iscsi
/etc/rc6.d/K80umountiscsi.sh -> ../init.d/umountiscsi.sh
/etc/rc6.d/K81open-iscsi -> ../init.d/open-iscsi
/etc/rcS.d/S45open-iscsi -> ../init.d/open-iscsi

3) Use update-rc.d to enable open-iscsi service

root@trusty-openiscsi:/etc# update-rc.d open-iscsi enable
update-rc.d: warning: start runlevel arguments (none) do not match open-iscsi 
Default-Start values (S)
update-rc.d: warning: stop runlevel arguments (none) do not match open-iscsi 
Default-Stop values (0 1 6)
Argument "S" isn't numeric in array element at /usr/sbin/update-rc.d line 232.
Enabling system startup links for /etc/init.d/open-iscsi ...
Removing any system startup links for /etc/init.d/open-iscsi ...
/etc/rc0.d/K81open-iscsi
/etc/rc1.d/K81open-iscsi
/etc/rc6.d/K81open-iscsi
/etc/rcS.d/S45open-iscsi
Adding system startup for /etc/init.d/open-iscsi ...
/etc/rc0.d/K81open-iscsi -> ../init.d/open-iscsi
/etc/rc1.d/K81open-iscsi -> ../init.d/open-iscsi
/etc/rc6.d/K81open-iscsi -> ../init.d/open-iscsi
/etc/rc0.d/S45open-iscsi -> ../init.d/open-iscsi

* The problem is the "/etc/rcS.d/S45open-iscsi" symlink is incorrectly 
  changed to "/etc/rc0.d/S45open-iscsi".

* Instead, the correct behaviour is to keep the symlink in /etc/rcS.d/ 
  intact:

root@trusty-openiscsi:/etc# update-rc.d open-iscsi enable
update-rc.d: warning: start runlevel arguments (none) do not match open-iscsi 
Default-Start values (S)
update-rc.d: warning: stop runlevel arguments (none) do not match open-iscsi 
Default-Stop values (0 1 6)
Enabling system startup links for /etc/init.d/open-iscsi ...
Removing any system startup links for /etc/init.d/open-iscsi ...
/etc/rc0.d/K81open-iscsi
/etc/rc1.d/K81open-iscsi
/etc/rc6.d/K81open-iscsi
/etc/rcS.d/S45open-iscsi
Adding system startup for /etc/init.d/open-iscsi ...
/etc/rc0.d/K81open-iscsi -> ../init.d/open-iscsi
/etc/rc1.d/K81open-iscsi -> ../init.d/open-iscsi
/etc/rc6.d/K81open-iscsi -> ../init.d/open-iscsi
/etc/rcS.d/S45open-iscsi -> ../init.d/open-iscsi

[Regression Potential]

 * There is only one file modified, which is the update-rc.d perl script. 
   Worst case scenario is that users cannot enable or disable their services, 
   or a symlink is changed such that a service is started / stopped on an 
   incorrect runlevel.

 * If a regression happens, any damage should be easily spotted by a 
   sysadmin and can be manually repaired by making manual symlinks with 
   "ln -s".

[Other Info]

 * trusty is the last Ubuntu release to use sysvinit, and this bug is not 
   present in newer versions since they use systemd, and the code in question 
   is removed from update-rc.d.

 * The bug exists in debian squeeze, which is now unsupported, and the code 
   in question is not in any newer versions.

 * The bug was introduced in 2009-06-29 and somehow evaded anyone noticing 
   it.

** Affects: sysvinit (Ubuntu)
 Importance: Medium
 Assignee: Matthew Ruffell (mruffell)
 Status: New

** Affects: sysvinit (Ubuntu Trusty)
 Importance: Medium
 Assignee: Matthew Ruffell (mruffell)
 Status: New


** Tags: sts

** Also affects: sysvinit (Ubuntu Trusty)
   Importance: Undecided
   Status: New

** Changed in: sysvinit (Ubuntu Trusty)
   Importance: Undecided => Medium

** Changed in: sysvinit (Ubuntu Trusty)
 Assignee: (unassigned) => Matthew Ruffell (mruffell)

-- 
You received thi