Re: DOS by unprivileged user

[vcaputo]

On Sun, Apr 22, 2018 at 12:16:54PM +0200, Pavel Machek wrote:
> On Thu 2018-04-19 21:13:35, Ferry Toth wrote:
> > It appears any ordinary user can easily create a DOS on linux.
> > 
> > One sure way to reproduce this is to open gitk on the linux kernel repo 
> > (SIC) on a machine with 8GB RAM 16 GB swap on a HDD with btrfs and quad 
> > core 
> > + hyperthreading. But I will be easy enough to get the same effect with 
> > more 
> > RAM, other fs etc.
> 
> You may want to disable swap.
> 

I run without swap on my laptops, and still observe long periods of
thrashing on the road towards OOM.  What seems to occur is the active
file-backed mappings of executables/libraries become a sort of swap
area, repeatedly being discarded and faulted back in as the context
switches occur.

If there's any good way to prevent this, I'd like to know.

Regards,
Vito Caputo

Re: DOS by unprivileged user

[vcaputo]

On Sun, Apr 22, 2018 at 12:16:54PM +0200, Pavel Machek wrote:
> On Thu 2018-04-19 21:13:35, Ferry Toth wrote:
> > It appears any ordinary user can easily create a DOS on linux.
> > 
> > One sure way to reproduce this is to open gitk on the linux kernel repo 
> > (SIC) on a machine with 8GB RAM 16 GB swap on a HDD with btrfs and quad 
> > core 
> > + hyperthreading. But I will be easy enough to get the same effect with 
> > more 
> > RAM, other fs etc.
> 
> You may want to disable swap.
> 

I run without swap on my laptops, and still observe long periods of
thrashing on the road towards OOM.  What seems to occur is the active
file-backed mappings of executables/libraries become a sort of swap
area, repeatedly being discarded and faulted back in as the context
switches occur.

If there's any good way to prevent this, I'd like to know.

Regards,
Vito Caputo

Re: [REGRESSION] (>= v4.12) IO w/dmcrypt causing audio underruns

[vcaputo]

On Fri, Jan 19, 2018 at 11:57:32AM +0100, Enric Balletbo Serra wrote:
> Hi Vito,
> 
> 2018-01-17 23:48 GMT+01:00  :
> > On Mon, Dec 18, 2017 at 10:25:33AM +0100, Enric Balletbo Serra wrote:
> >> Hi Vito,
> >>
> >> 2017-12-01 22:33 GMT+01:00  :
> >> > On Wed, Nov 29, 2017 at 10:39:19AM -0800, vcap...@pengaru.com wrote:
> >> >> Hello,
> >> >>
> >> >> Recently I noticed substantial audio dropouts when listening to MP3s in
> >> >> `cmus` while doing big and churny `git checkout` commands in my linux 
> >> >> git
> >> >> tree.
> >> >>
> >> >> It's not something I've done much of over the last couple months so I
> >> >> hadn't noticed until yesterday, but didn't remember this being a 
> >> >> problem in
> >> >> recent history.
> >> >>
> >> >> As there's quite an accumulation of similarly configured and built 
> >> >> kernels
> >> >> in my grub menu, it was trivial to determine approximately when this 
> >> >> began:
> >> >>
> >> >> 4.11.0: no dropouts
> >> >> 4.12.0-rc7: dropouts
> >> >> 4.14.0-rc6: dropouts (seem more substantial as well, didn't investigate)
> >> >>
> >> >> Watching top while this is going on in the various kernel versions, it's
> >> >> apparent that the kworker behavior changed.  Both the priority and 
> >> >> quantity
> >> >> of running kworker threads is elevated in kernels experiencing dropouts.
> >> >>
> >> >> Searching through the commit history for v4.11..v4.12 uncovered:
> >> >>
> >> >> commit a1b89132dc4f61071bdeaab92ea958e0953380a1
> >> >> Author: Tim Murray 
> >> >> Date:   Fri Apr 21 11:11:36 2017 +0200
> >> >>
> >> >> dm crypt: use WQ_HIGHPRI for the IO and crypt workqueues
> >> >>
> >> >> Running dm-crypt with workqueues at the standard priority results 
> >> >> in IO
> >> >> competing for CPU time with standard user apps, which can lead to
> >> >> pipeline bubbles and seriously degraded performance.  Move to using
> >> >> WQ_HIGHPRI workqueues to protect against that.
> >> >>
> >> >> Signed-off-by: Tim Murray 
> >> >> Signed-off-by: Enric Balletbo i Serra 
> >> >> Signed-off-by: Mike Snitzer 
> >> >>
> >> >> ---
> >> >>
> >> >> Reverting a1b8913 from 4.14.0-rc6, my current kernel, eliminates the
> >> >> problem completely.
> >> >>
> >> >> Looking at the diff in that commit, it looks like the commit message 
> >> >> isn't
> >> >> even accurate; not only is the priority of the dmcrypt workqueues being
> >> >> changed - they're also being made "CPU intensive" workqueues as well.
> >> >>
> >> >> This combination appears to result in both elevated scheduling priority 
> >> >> and
> >> >> greater quantity of participant worker threads effectively starving any
> >> >> normal priority user task under periods of heavy IO on dmcrypt volumes.
> >> >>
> >> >> I don't know what the right solution is here.  It seems to me we're 
> >> >> lacking
> >> >> the appropriate mechanism for charging CPU resources consumed on behalf 
> >> >> of
> >> >> user processes in kworker threads to the work-causing process.
> >> >>
> >> >> What effectively happens is my normal `git` user process is able to
> >> >> greatly amplify what share of CPU it takes from the system by 
> >> >> generating IO
> >> >> on what happens to be a high-priority CPU-intensive storage volume.
> >> >>
> >> >> It looks potentially complicated to fix properly, but I suspect at its 
> >> >> core
> >> >> this may be a fairly longstanding shortcoming of the page cache and its
> >> >> asynchronous design.  Something that has been exacerbated substantially 
> >> >> by
> >> >> the introduction of CPU-intensive storage subsystems like dmcrypt.
> >> >>
> >> >> If we imagine the whole stack simplified, where all the IO was being 
> >> >> done
> >> >> synchronously in-band, and the dmcrypt kernel code simply ran in the
> >> >> IO-causing process context, it would be getting charged to the calling
> >> >> process and scheduled accordingly.  The resource accounting and 
> >> >> scheduling
> >> >> problems all emerge with the page cache, buffered IO, and async 
> >> >> background
> >> >> writeback in a pool of unrelated worker threads, etc.  That's how it
> >> >> appears to me anyways...
> >> >>
> >> >> The system used is a X61s Thinkpad 1.8Ghz with 840 EVO SSD, lvm on 
> >> >> dmcrypt.
> >> >> The kernel .config is attached in case it's of interest.
> >> >>
> >> >> Thanks,
> >> >> Vito Caputo
> >> >
> >> >
> >> >
> >> > Ping...
> >> >
> >> > Could somebody please at least ACK receiving this so I'm not left 
> >> > wondering
> >> > if my mails to lkml are somehow winding up flagged as spam, thanks!
> >>
> >> Sorry I did not notice your email before you ping me directly. It's
> >> interesting that issue, though we didn't notice this problem. It's a
> >> bit far since I tested this patch but I'll setup the environment again
> >> and do more tests to understand better what is happening.

Re: [REGRESSION] (>= v4.12) IO w/dmcrypt causing audio underruns

[vcaputo]

On Fri, Jan 19, 2018 at 11:57:32AM +0100, Enric Balletbo Serra wrote:
> Hi Vito,
> 
> 2018-01-17 23:48 GMT+01:00  :
> > On Mon, Dec 18, 2017 at 10:25:33AM +0100, Enric Balletbo Serra wrote:
> >> Hi Vito,
> >>
> >> 2017-12-01 22:33 GMT+01:00  :
> >> > On Wed, Nov 29, 2017 at 10:39:19AM -0800, vcap...@pengaru.com wrote:
> >> >> Hello,
> >> >>
> >> >> Recently I noticed substantial audio dropouts when listening to MP3s in
> >> >> `cmus` while doing big and churny `git checkout` commands in my linux 
> >> >> git
> >> >> tree.
> >> >>
> >> >> It's not something I've done much of over the last couple months so I
> >> >> hadn't noticed until yesterday, but didn't remember this being a 
> >> >> problem in
> >> >> recent history.
> >> >>
> >> >> As there's quite an accumulation of similarly configured and built 
> >> >> kernels
> >> >> in my grub menu, it was trivial to determine approximately when this 
> >> >> began:
> >> >>
> >> >> 4.11.0: no dropouts
> >> >> 4.12.0-rc7: dropouts
> >> >> 4.14.0-rc6: dropouts (seem more substantial as well, didn't investigate)
> >> >>
> >> >> Watching top while this is going on in the various kernel versions, it's
> >> >> apparent that the kworker behavior changed.  Both the priority and 
> >> >> quantity
> >> >> of running kworker threads is elevated in kernels experiencing dropouts.
> >> >>
> >> >> Searching through the commit history for v4.11..v4.12 uncovered:
> >> >>
> >> >> commit a1b89132dc4f61071bdeaab92ea958e0953380a1
> >> >> Author: Tim Murray 
> >> >> Date:   Fri Apr 21 11:11:36 2017 +0200
> >> >>
> >> >> dm crypt: use WQ_HIGHPRI for the IO and crypt workqueues
> >> >>
> >> >> Running dm-crypt with workqueues at the standard priority results 
> >> >> in IO
> >> >> competing for CPU time with standard user apps, which can lead to
> >> >> pipeline bubbles and seriously degraded performance.  Move to using
> >> >> WQ_HIGHPRI workqueues to protect against that.
> >> >>
> >> >> Signed-off-by: Tim Murray 
> >> >> Signed-off-by: Enric Balletbo i Serra 
> >> >> Signed-off-by: Mike Snitzer 
> >> >>
> >> >> ---
> >> >>
> >> >> Reverting a1b8913 from 4.14.0-rc6, my current kernel, eliminates the
> >> >> problem completely.
> >> >>
> >> >> Looking at the diff in that commit, it looks like the commit message 
> >> >> isn't
> >> >> even accurate; not only is the priority of the dmcrypt workqueues being
> >> >> changed - they're also being made "CPU intensive" workqueues as well.
> >> >>
> >> >> This combination appears to result in both elevated scheduling priority 
> >> >> and
> >> >> greater quantity of participant worker threads effectively starving any
> >> >> normal priority user task under periods of heavy IO on dmcrypt volumes.
> >> >>
> >> >> I don't know what the right solution is here.  It seems to me we're 
> >> >> lacking
> >> >> the appropriate mechanism for charging CPU resources consumed on behalf 
> >> >> of
> >> >> user processes in kworker threads to the work-causing process.
> >> >>
> >> >> What effectively happens is my normal `git` user process is able to
> >> >> greatly amplify what share of CPU it takes from the system by 
> >> >> generating IO
> >> >> on what happens to be a high-priority CPU-intensive storage volume.
> >> >>
> >> >> It looks potentially complicated to fix properly, but I suspect at its 
> >> >> core
> >> >> this may be a fairly longstanding shortcoming of the page cache and its
> >> >> asynchronous design.  Something that has been exacerbated substantially 
> >> >> by
> >> >> the introduction of CPU-intensive storage subsystems like dmcrypt.
> >> >>
> >> >> If we imagine the whole stack simplified, where all the IO was being 
> >> >> done
> >> >> synchronously in-band, and the dmcrypt kernel code simply ran in the
> >> >> IO-causing process context, it would be getting charged to the calling
> >> >> process and scheduled accordingly.  The resource accounting and 
> >> >> scheduling
> >> >> problems all emerge with the page cache, buffered IO, and async 
> >> >> background
> >> >> writeback in a pool of unrelated worker threads, etc.  That's how it
> >> >> appears to me anyways...
> >> >>
> >> >> The system used is a X61s Thinkpad 1.8Ghz with 840 EVO SSD, lvm on 
> >> >> dmcrypt.
> >> >> The kernel .config is attached in case it's of interest.
> >> >>
> >> >> Thanks,
> >> >> Vito Caputo
> >> >
> >> >
> >> >
> >> > Ping...
> >> >
> >> > Could somebody please at least ACK receiving this so I'm not left 
> >> > wondering
> >> > if my mails to lkml are somehow winding up flagged as spam, thanks!
> >>
> >> Sorry I did not notice your email before you ping me directly. It's
> >> interesting that issue, though we didn't notice this problem. It's a
> >> bit far since I tested this patch but I'll setup the environment again
> >> and do more tests to understand better what is happening.
> >>
> >
> > Any update on this?
> >
> 
> I did not reproduce the issue for now. Can you try what happens if you
> remove the 

Re: [REGRESSION] (>= v4.12) IO w/dmcrypt causing audio underruns

[vcaputo]

On Fri, Jan 19, 2018 at 11:57:32AM +0100, Enric Balletbo Serra wrote:
> Hi Vito,
> 
> 2018-01-17 23:48 GMT+01:00  :
> > On Mon, Dec 18, 2017 at 10:25:33AM +0100, Enric Balletbo Serra wrote:
> >> Hi Vito,
> >>
> >> 2017-12-01 22:33 GMT+01:00  :
> >> > On Wed, Nov 29, 2017 at 10:39:19AM -0800, vcap...@pengaru.com wrote:
> >> >> Hello,
> >> >>
> >> >> Recently I noticed substantial audio dropouts when listening to MP3s in
> >> >> `cmus` while doing big and churny `git checkout` commands in my linux 
> >> >> git
> >> >> tree.
> >> >>
> >> >> It's not something I've done much of over the last couple months so I
> >> >> hadn't noticed until yesterday, but didn't remember this being a 
> >> >> problem in
> >> >> recent history.
> >> >>
> >> >> As there's quite an accumulation of similarly configured and built 
> >> >> kernels
> >> >> in my grub menu, it was trivial to determine approximately when this 
> >> >> began:
> >> >>
> >> >> 4.11.0: no dropouts
> >> >> 4.12.0-rc7: dropouts
> >> >> 4.14.0-rc6: dropouts (seem more substantial as well, didn't investigate)
> >> >>
> >> >> Watching top while this is going on in the various kernel versions, it's
> >> >> apparent that the kworker behavior changed.  Both the priority and 
> >> >> quantity
> >> >> of running kworker threads is elevated in kernels experiencing dropouts.
> >> >>
> >> >> Searching through the commit history for v4.11..v4.12 uncovered:
> >> >>
> >> >> commit a1b89132dc4f61071bdeaab92ea958e0953380a1
> >> >> Author: Tim Murray 
> >> >> Date:   Fri Apr 21 11:11:36 2017 +0200
> >> >>
> >> >> dm crypt: use WQ_HIGHPRI for the IO and crypt workqueues
> >> >>
> >> >> Running dm-crypt with workqueues at the standard priority results 
> >> >> in IO
> >> >> competing for CPU time with standard user apps, which can lead to
> >> >> pipeline bubbles and seriously degraded performance.  Move to using
> >> >> WQ_HIGHPRI workqueues to protect against that.
> >> >>
> >> >> Signed-off-by: Tim Murray 
> >> >> Signed-off-by: Enric Balletbo i Serra 
> >> >> Signed-off-by: Mike Snitzer 
> >> >>
> >> >> ---
> >> >>
> >> >> Reverting a1b8913 from 4.14.0-rc6, my current kernel, eliminates the
> >> >> problem completely.
> >> >>
> >> >> Looking at the diff in that commit, it looks like the commit message 
> >> >> isn't
> >> >> even accurate; not only is the priority of the dmcrypt workqueues being
> >> >> changed - they're also being made "CPU intensive" workqueues as well.
> >> >>
> >> >> This combination appears to result in both elevated scheduling priority 
> >> >> and
> >> >> greater quantity of participant worker threads effectively starving any
> >> >> normal priority user task under periods of heavy IO on dmcrypt volumes.
> >> >>
> >> >> I don't know what the right solution is here.  It seems to me we're 
> >> >> lacking
> >> >> the appropriate mechanism for charging CPU resources consumed on behalf 
> >> >> of
> >> >> user processes in kworker threads to the work-causing process.
> >> >>
> >> >> What effectively happens is my normal `git` user process is able to
> >> >> greatly amplify what share of CPU it takes from the system by 
> >> >> generating IO
> >> >> on what happens to be a high-priority CPU-intensive storage volume.
> >> >>
> >> >> It looks potentially complicated to fix properly, but I suspect at its 
> >> >> core
> >> >> this may be a fairly longstanding shortcoming of the page cache and its
> >> >> asynchronous design.  Something that has been exacerbated substantially 
> >> >> by
> >> >> the introduction of CPU-intensive storage subsystems like dmcrypt.
> >> >>
> >> >> If we imagine the whole stack simplified, where all the IO was being 
> >> >> done
> >> >> synchronously in-band, and the dmcrypt kernel code simply ran in the
> >> >> IO-causing process context, it would be getting charged to the calling
> >> >> process and scheduled accordingly.  The resource accounting and 
> >> >> scheduling
> >> >> problems all emerge with the page cache, buffered IO, and async 
> >> >> background
> >> >> writeback in a pool of unrelated worker threads, etc.  That's how it
> >> >> appears to me anyways...
> >> >>
> >> >> The system used is a X61s Thinkpad 1.8Ghz with 840 EVO SSD, lvm on 
> >> >> dmcrypt.
> >> >> The kernel .config is attached in case it's of interest.
> >> >>
> >> >> Thanks,
> >> >> Vito Caputo
> >> >
> >> >
> >> >
> >> > Ping...
> >> >
> >> > Could somebody please at least ACK receiving this so I'm not left 
> >> > wondering
> >> > if my mails to lkml are somehow winding up flagged as spam, thanks!
> >>
> >> Sorry I did not notice your email before you ping me directly. It's
> >> interesting that issue, though we didn't notice this problem. It's a
> >> bit far since I tested this patch but I'll setup the environment again
> >> and do more tests to understand better what is happening.

Re: [REGRESSION] (>= v4.12) IO w/dmcrypt causing audio underruns

[vcaputo]

On Fri, Jan 19, 2018 at 11:57:32AM +0100, Enric Balletbo Serra wrote:
> Hi Vito,
> 
> 2018-01-17 23:48 GMT+01:00  :
> > On Mon, Dec 18, 2017 at 10:25:33AM +0100, Enric Balletbo Serra wrote:
> >> Hi Vito,
> >>
> >> 2017-12-01 22:33 GMT+01:00  :
> >> > On Wed, Nov 29, 2017 at 10:39:19AM -0800, vcap...@pengaru.com wrote:
> >> >> Hello,
> >> >>
> >> >> Recently I noticed substantial audio dropouts when listening to MP3s in
> >> >> `cmus` while doing big and churny `git checkout` commands in my linux 
> >> >> git
> >> >> tree.
> >> >>
> >> >> It's not something I've done much of over the last couple months so I
> >> >> hadn't noticed until yesterday, but didn't remember this being a 
> >> >> problem in
> >> >> recent history.
> >> >>
> >> >> As there's quite an accumulation of similarly configured and built 
> >> >> kernels
> >> >> in my grub menu, it was trivial to determine approximately when this 
> >> >> began:
> >> >>
> >> >> 4.11.0: no dropouts
> >> >> 4.12.0-rc7: dropouts
> >> >> 4.14.0-rc6: dropouts (seem more substantial as well, didn't investigate)
> >> >>
> >> >> Watching top while this is going on in the various kernel versions, it's
> >> >> apparent that the kworker behavior changed.  Both the priority and 
> >> >> quantity
> >> >> of running kworker threads is elevated in kernels experiencing dropouts.
> >> >>
> >> >> Searching through the commit history for v4.11..v4.12 uncovered:
> >> >>
> >> >> commit a1b89132dc4f61071bdeaab92ea958e0953380a1
> >> >> Author: Tim Murray 
> >> >> Date:   Fri Apr 21 11:11:36 2017 +0200
> >> >>
> >> >> dm crypt: use WQ_HIGHPRI for the IO and crypt workqueues
> >> >>
> >> >> Running dm-crypt with workqueues at the standard priority results 
> >> >> in IO
> >> >> competing for CPU time with standard user apps, which can lead to
> >> >> pipeline bubbles and seriously degraded performance.  Move to using
> >> >> WQ_HIGHPRI workqueues to protect against that.
> >> >>
> >> >> Signed-off-by: Tim Murray 
> >> >> Signed-off-by: Enric Balletbo i Serra 
> >> >> Signed-off-by: Mike Snitzer 
> >> >>
> >> >> ---
> >> >>
> >> >> Reverting a1b8913 from 4.14.0-rc6, my current kernel, eliminates the
> >> >> problem completely.
> >> >>
> >> >> Looking at the diff in that commit, it looks like the commit message 
> >> >> isn't
> >> >> even accurate; not only is the priority of the dmcrypt workqueues being
> >> >> changed - they're also being made "CPU intensive" workqueues as well.
> >> >>
> >> >> This combination appears to result in both elevated scheduling priority 
> >> >> and
> >> >> greater quantity of participant worker threads effectively starving any
> >> >> normal priority user task under periods of heavy IO on dmcrypt volumes.
> >> >>
> >> >> I don't know what the right solution is here.  It seems to me we're 
> >> >> lacking
> >> >> the appropriate mechanism for charging CPU resources consumed on behalf 
> >> >> of
> >> >> user processes in kworker threads to the work-causing process.
> >> >>
> >> >> What effectively happens is my normal `git` user process is able to
> >> >> greatly amplify what share of CPU it takes from the system by 
> >> >> generating IO
> >> >> on what happens to be a high-priority CPU-intensive storage volume.
> >> >>
> >> >> It looks potentially complicated to fix properly, but I suspect at its 
> >> >> core
> >> >> this may be a fairly longstanding shortcoming of the page cache and its
> >> >> asynchronous design.  Something that has been exacerbated substantially 
> >> >> by
> >> >> the introduction of CPU-intensive storage subsystems like dmcrypt.
> >> >>
> >> >> If we imagine the whole stack simplified, where all the IO was being 
> >> >> done
> >> >> synchronously in-band, and the dmcrypt kernel code simply ran in the
> >> >> IO-causing process context, it would be getting charged to the calling
> >> >> process and scheduled accordingly.  The resource accounting and 
> >> >> scheduling
> >> >> problems all emerge with the page cache, buffered IO, and async 
> >> >> background
> >> >> writeback in a pool of unrelated worker threads, etc.  That's how it
> >> >> appears to me anyways...
> >> >>
> >> >> The system used is a X61s Thinkpad 1.8Ghz with 840 EVO SSD, lvm on 
> >> >> dmcrypt.
> >> >> The kernel .config is attached in case it's of interest.
> >> >>
> >> >> Thanks,
> >> >> Vito Caputo
> >> >
> >> >
> >> >
> >> > Ping...
> >> >
> >> > Could somebody please at least ACK receiving this so I'm not left 
> >> > wondering
> >> > if my mails to lkml are somehow winding up flagged as spam, thanks!
> >>
> >> Sorry I did not notice your email before you ping me directly. It's
> >> interesting that issue, though we didn't notice this problem. It's a
> >> bit far since I tested this patch but I'll setup the environment again
> >> and do more tests to understand better what is happening.
> >>
> >
> > Any update on this?
> >
> 
> I did not reproduce the issue for now. Can you try what happens if you
> remove the 

Re: [REGRESSION] (>= v4.12) IO w/dmcrypt causing audio underruns

[vcaputo]

On Fri, Jan 19, 2018 at 11:57:32AM +0100, Enric Balletbo Serra wrote:
> Hi Vito,
> 
> 2018-01-17 23:48 GMT+01:00  :
> > On Mon, Dec 18, 2017 at 10:25:33AM +0100, Enric Balletbo Serra wrote:
> >> Hi Vito,
> >>
> >> 2017-12-01 22:33 GMT+01:00  :
> >> > On Wed, Nov 29, 2017 at 10:39:19AM -0800, vcap...@pengaru.com wrote:
> >> >> Hello,
> >> >>
> >> >> Recently I noticed substantial audio dropouts when listening to MP3s in
> >> >> `cmus` while doing big and churny `git checkout` commands in my linux 
> >> >> git
> >> >> tree.
> >> >>
> >> >> It's not something I've done much of over the last couple months so I
> >> >> hadn't noticed until yesterday, but didn't remember this being a 
> >> >> problem in
> >> >> recent history.
> >> >>
> >> >> As there's quite an accumulation of similarly configured and built 
> >> >> kernels
> >> >> in my grub menu, it was trivial to determine approximately when this 
> >> >> began:
> >> >>
> >> >> 4.11.0: no dropouts
> >> >> 4.12.0-rc7: dropouts
> >> >> 4.14.0-rc6: dropouts (seem more substantial as well, didn't investigate)
> >> >>
> >> >> Watching top while this is going on in the various kernel versions, it's
> >> >> apparent that the kworker behavior changed.  Both the priority and 
> >> >> quantity
> >> >> of running kworker threads is elevated in kernels experiencing dropouts.
> >> >>
> >> >> Searching through the commit history for v4.11..v4.12 uncovered:
> >> >>
> >> >> commit a1b89132dc4f61071bdeaab92ea958e0953380a1
> >> >> Author: Tim Murray 
> >> >> Date:   Fri Apr 21 11:11:36 2017 +0200
> >> >>
> >> >> dm crypt: use WQ_HIGHPRI for the IO and crypt workqueues
> >> >>
> >> >> Running dm-crypt with workqueues at the standard priority results 
> >> >> in IO
> >> >> competing for CPU time with standard user apps, which can lead to
> >> >> pipeline bubbles and seriously degraded performance.  Move to using
> >> >> WQ_HIGHPRI workqueues to protect against that.
> >> >>
> >> >> Signed-off-by: Tim Murray 
> >> >> Signed-off-by: Enric Balletbo i Serra 
> >> >> Signed-off-by: Mike Snitzer 
> >> >>
> >> >> ---
> >> >>
> >> >> Reverting a1b8913 from 4.14.0-rc6, my current kernel, eliminates the
> >> >> problem completely.
> >> >>
> >> >> Looking at the diff in that commit, it looks like the commit message 
> >> >> isn't
> >> >> even accurate; not only is the priority of the dmcrypt workqueues being
> >> >> changed - they're also being made "CPU intensive" workqueues as well.
> >> >>
> >> >> This combination appears to result in both elevated scheduling priority 
> >> >> and
> >> >> greater quantity of participant worker threads effectively starving any
> >> >> normal priority user task under periods of heavy IO on dmcrypt volumes.
> >> >>
> >> >> I don't know what the right solution is here.  It seems to me we're 
> >> >> lacking
> >> >> the appropriate mechanism for charging CPU resources consumed on behalf 
> >> >> of
> >> >> user processes in kworker threads to the work-causing process.
> >> >>
> >> >> What effectively happens is my normal `git` user process is able to
> >> >> greatly amplify what share of CPU it takes from the system by 
> >> >> generating IO
> >> >> on what happens to be a high-priority CPU-intensive storage volume.
> >> >>
> >> >> It looks potentially complicated to fix properly, but I suspect at its 
> >> >> core
> >> >> this may be a fairly longstanding shortcoming of the page cache and its
> >> >> asynchronous design.  Something that has been exacerbated substantially 
> >> >> by
> >> >> the introduction of CPU-intensive storage subsystems like dmcrypt.
> >> >>
> >> >> If we imagine the whole stack simplified, where all the IO was being 
> >> >> done
> >> >> synchronously in-band, and the dmcrypt kernel code simply ran in the
> >> >> IO-causing process context, it would be getting charged to the calling
> >> >> process and scheduled accordingly.  The resource accounting and 
> >> >> scheduling
> >> >> problems all emerge with the page cache, buffered IO, and async 
> >> >> background
> >> >> writeback in a pool of unrelated worker threads, etc.  That's how it
> >> >> appears to me anyways...
> >> >>
> >> >> The system used is a X61s Thinkpad 1.8Ghz with 840 EVO SSD, lvm on 
> >> >> dmcrypt.
> >> >> The kernel .config is attached in case it's of interest.
> >> >>
> >> >> Thanks,
> >> >> Vito Caputo
> >> >
> >> >
> >> >
> >> > Ping...
> >> >
> >> > Could somebody please at least ACK receiving this so I'm not left 
> >> > wondering
> >> > if my mails to lkml are somehow winding up flagged as spam, thanks!
> >>
> >> Sorry I did not notice your email before you ping me directly. It's
> >> interesting that issue, though we didn't notice this problem. It's a
> >> bit far since I tested this patch but I'll setup the environment again
> >> and do more tests to understand better what is happening.

Re: [REGRESSION] (>= v4.12) IO w/dmcrypt causing audio underruns

[vcaputo]

On Fri, Jan 19, 2018 at 11:57:32AM +0100, Enric Balletbo Serra wrote:
> Hi Vito,
> 
> 2018-01-17 23:48 GMT+01:00  :
> > On Mon, Dec 18, 2017 at 10:25:33AM +0100, Enric Balletbo Serra wrote:
> >> Hi Vito,
> >>
> >> 2017-12-01 22:33 GMT+01:00  :
> >> > On Wed, Nov 29, 2017 at 10:39:19AM -0800, vcap...@pengaru.com wrote:
> >> >> Hello,
> >> >>
> >> >> Recently I noticed substantial audio dropouts when listening to MP3s in
> >> >> `cmus` while doing big and churny `git checkout` commands in my linux 
> >> >> git
> >> >> tree.
> >> >>
> >> >> It's not something I've done much of over the last couple months so I
> >> >> hadn't noticed until yesterday, but didn't remember this being a 
> >> >> problem in
> >> >> recent history.
> >> >>
> >> >> As there's quite an accumulation of similarly configured and built 
> >> >> kernels
> >> >> in my grub menu, it was trivial to determine approximately when this 
> >> >> began:
> >> >>
> >> >> 4.11.0: no dropouts
> >> >> 4.12.0-rc7: dropouts
> >> >> 4.14.0-rc6: dropouts (seem more substantial as well, didn't investigate)
> >> >>
> >> >> Watching top while this is going on in the various kernel versions, it's
> >> >> apparent that the kworker behavior changed.  Both the priority and 
> >> >> quantity
> >> >> of running kworker threads is elevated in kernels experiencing dropouts.
> >> >>
> >> >> Searching through the commit history for v4.11..v4.12 uncovered:
> >> >>
> >> >> commit a1b89132dc4f61071bdeaab92ea958e0953380a1
> >> >> Author: Tim Murray 
> >> >> Date:   Fri Apr 21 11:11:36 2017 +0200
> >> >>
> >> >> dm crypt: use WQ_HIGHPRI for the IO and crypt workqueues
> >> >>
> >> >> Running dm-crypt with workqueues at the standard priority results 
> >> >> in IO
> >> >> competing for CPU time with standard user apps, which can lead to
> >> >> pipeline bubbles and seriously degraded performance.  Move to using
> >> >> WQ_HIGHPRI workqueues to protect against that.
> >> >>
> >> >> Signed-off-by: Tim Murray 
> >> >> Signed-off-by: Enric Balletbo i Serra 
> >> >> Signed-off-by: Mike Snitzer 
> >> >>
> >> >> ---
> >> >>
> >> >> Reverting a1b8913 from 4.14.0-rc6, my current kernel, eliminates the
> >> >> problem completely.
> >> >>
> >> >> Looking at the diff in that commit, it looks like the commit message 
> >> >> isn't
> >> >> even accurate; not only is the priority of the dmcrypt workqueues being
> >> >> changed - they're also being made "CPU intensive" workqueues as well.
> >> >>
> >> >> This combination appears to result in both elevated scheduling priority 
> >> >> and
> >> >> greater quantity of participant worker threads effectively starving any
> >> >> normal priority user task under periods of heavy IO on dmcrypt volumes.
> >> >>
> >> >> I don't know what the right solution is here.  It seems to me we're 
> >> >> lacking
> >> >> the appropriate mechanism for charging CPU resources consumed on behalf 
> >> >> of
> >> >> user processes in kworker threads to the work-causing process.
> >> >>
> >> >> What effectively happens is my normal `git` user process is able to
> >> >> greatly amplify what share of CPU it takes from the system by 
> >> >> generating IO
> >> >> on what happens to be a high-priority CPU-intensive storage volume.
> >> >>
> >> >> It looks potentially complicated to fix properly, but I suspect at its 
> >> >> core
> >> >> this may be a fairly longstanding shortcoming of the page cache and its
> >> >> asynchronous design.  Something that has been exacerbated substantially 
> >> >> by
> >> >> the introduction of CPU-intensive storage subsystems like dmcrypt.
> >> >>
> >> >> If we imagine the whole stack simplified, where all the IO was being 
> >> >> done
> >> >> synchronously in-band, and the dmcrypt kernel code simply ran in the
> >> >> IO-causing process context, it would be getting charged to the calling
> >> >> process and scheduled accordingly.  The resource accounting and 
> >> >> scheduling
> >> >> problems all emerge with the page cache, buffered IO, and async 
> >> >> background
> >> >> writeback in a pool of unrelated worker threads, etc.  That's how it
> >> >> appears to me anyways...
> >> >>
> >> >> The system used is a X61s Thinkpad 1.8Ghz with 840 EVO SSD, lvm on 
> >> >> dmcrypt.
> >> >> The kernel .config is attached in case it's of interest.
> >> >>
> >> >> Thanks,
> >> >> Vito Caputo
> >> >
> >> >
> >> >
> >> > Ping...
> >> >
> >> > Could somebody please at least ACK receiving this so I'm not left 
> >> > wondering
> >> > if my mails to lkml are somehow winding up flagged as spam, thanks!
> >>
> >> Sorry I did not notice your email before you ping me directly. It's
> >> interesting that issue, though we didn't notice this problem. It's a
> >> bit far since I tested this patch but I'll setup the environment again
> >> and do more tests to understand better what is happening.
> >>
> >
> > Any update on this?
> >
> 
> I did not reproduce the issue for now. Can you try what happens if you
> remove the 

Re: [REGRESSION] (>= v4.12) IO w/dmcrypt causing audio underruns

[vcaputo]

On Mon, Dec 18, 2017 at 10:25:33AM +0100, Enric Balletbo Serra wrote:
> Hi Vito,
> 
> 2017-12-01 22:33 GMT+01:00  :
> > On Wed, Nov 29, 2017 at 10:39:19AM -0800, vcap...@pengaru.com wrote:
> >> Hello,
> >>
> >> Recently I noticed substantial audio dropouts when listening to MP3s in
> >> `cmus` while doing big and churny `git checkout` commands in my linux git
> >> tree.
> >>
> >> It's not something I've done much of over the last couple months so I
> >> hadn't noticed until yesterday, but didn't remember this being a problem in
> >> recent history.
> >>
> >> As there's quite an accumulation of similarly configured and built kernels
> >> in my grub menu, it was trivial to determine approximately when this began:
> >>
> >> 4.11.0: no dropouts
> >> 4.12.0-rc7: dropouts
> >> 4.14.0-rc6: dropouts (seem more substantial as well, didn't investigate)
> >>
> >> Watching top while this is going on in the various kernel versions, it's
> >> apparent that the kworker behavior changed.  Both the priority and quantity
> >> of running kworker threads is elevated in kernels experiencing dropouts.
> >>
> >> Searching through the commit history for v4.11..v4.12 uncovered:
> >>
> >> commit a1b89132dc4f61071bdeaab92ea958e0953380a1
> >> Author: Tim Murray 
> >> Date:   Fri Apr 21 11:11:36 2017 +0200
> >>
> >> dm crypt: use WQ_HIGHPRI for the IO and crypt workqueues
> >>
> >> Running dm-crypt with workqueues at the standard priority results in IO
> >> competing for CPU time with standard user apps, which can lead to
> >> pipeline bubbles and seriously degraded performance.  Move to using
> >> WQ_HIGHPRI workqueues to protect against that.
> >>
> >> Signed-off-by: Tim Murray 
> >> Signed-off-by: Enric Balletbo i Serra 
> >> Signed-off-by: Mike Snitzer 
> >>
> >> ---
> >>
> >> Reverting a1b8913 from 4.14.0-rc6, my current kernel, eliminates the
> >> problem completely.
> >>
> >> Looking at the diff in that commit, it looks like the commit message isn't
> >> even accurate; not only is the priority of the dmcrypt workqueues being
> >> changed - they're also being made "CPU intensive" workqueues as well.
> >>
> >> This combination appears to result in both elevated scheduling priority and
> >> greater quantity of participant worker threads effectively starving any
> >> normal priority user task under periods of heavy IO on dmcrypt volumes.
> >>
> >> I don't know what the right solution is here.  It seems to me we're lacking
> >> the appropriate mechanism for charging CPU resources consumed on behalf of
> >> user processes in kworker threads to the work-causing process.
> >>
> >> What effectively happens is my normal `git` user process is able to
> >> greatly amplify what share of CPU it takes from the system by generating IO
> >> on what happens to be a high-priority CPU-intensive storage volume.
> >>
> >> It looks potentially complicated to fix properly, but I suspect at its core
> >> this may be a fairly longstanding shortcoming of the page cache and its
> >> asynchronous design.  Something that has been exacerbated substantially by
> >> the introduction of CPU-intensive storage subsystems like dmcrypt.
> >>
> >> If we imagine the whole stack simplified, where all the IO was being done
> >> synchronously in-band, and the dmcrypt kernel code simply ran in the
> >> IO-causing process context, it would be getting charged to the calling
> >> process and scheduled accordingly.  The resource accounting and scheduling
> >> problems all emerge with the page cache, buffered IO, and async background
> >> writeback in a pool of unrelated worker threads, etc.  That's how it
> >> appears to me anyways...
> >>
> >> The system used is a X61s Thinkpad 1.8Ghz with 840 EVO SSD, lvm on dmcrypt.
> >> The kernel .config is attached in case it's of interest.
> >>
> >> Thanks,
> >> Vito Caputo
> >
> >
> >
> > Ping...
> >
> > Could somebody please at least ACK receiving this so I'm not left wondering
> > if my mails to lkml are somehow winding up flagged as spam, thanks!
> 
> Sorry I did not notice your email before you ping me directly. It's
> interesting that issue, though we didn't notice this problem. It's a
> bit far since I tested this patch but I'll setup the environment again
> and do more tests to understand better what is happening.
> 

Any update on this?

I still experience it on 4.15-rc7 when doing sustained heavyweight git
checkouts without a1b8913 reverted.
 
Thanks,
Vito Caputo

Re: [REGRESSION] (>= v4.12) IO w/dmcrypt causing audio underruns

[vcaputo]

On Mon, Dec 18, 2017 at 10:25:33AM +0100, Enric Balletbo Serra wrote:
> Hi Vito,
> 
> 2017-12-01 22:33 GMT+01:00  :
> > On Wed, Nov 29, 2017 at 10:39:19AM -0800, vcap...@pengaru.com wrote:
> >> Hello,
> >>
> >> Recently I noticed substantial audio dropouts when listening to MP3s in
> >> `cmus` while doing big and churny `git checkout` commands in my linux git
> >> tree.
> >>
> >> It's not something I've done much of over the last couple months so I
> >> hadn't noticed until yesterday, but didn't remember this being a problem in
> >> recent history.
> >>
> >> As there's quite an accumulation of similarly configured and built kernels
> >> in my grub menu, it was trivial to determine approximately when this began:
> >>
> >> 4.11.0: no dropouts
> >> 4.12.0-rc7: dropouts
> >> 4.14.0-rc6: dropouts (seem more substantial as well, didn't investigate)
> >>
> >> Watching top while this is going on in the various kernel versions, it's
> >> apparent that the kworker behavior changed.  Both the priority and quantity
> >> of running kworker threads is elevated in kernels experiencing dropouts.
> >>
> >> Searching through the commit history for v4.11..v4.12 uncovered:
> >>
> >> commit a1b89132dc4f61071bdeaab92ea958e0953380a1
> >> Author: Tim Murray 
> >> Date:   Fri Apr 21 11:11:36 2017 +0200
> >>
> >> dm crypt: use WQ_HIGHPRI for the IO and crypt workqueues
> >>
> >> Running dm-crypt with workqueues at the standard priority results in IO
> >> competing for CPU time with standard user apps, which can lead to
> >> pipeline bubbles and seriously degraded performance.  Move to using
> >> WQ_HIGHPRI workqueues to protect against that.
> >>
> >> Signed-off-by: Tim Murray 
> >> Signed-off-by: Enric Balletbo i Serra 
> >> Signed-off-by: Mike Snitzer 
> >>
> >> ---
> >>
> >> Reverting a1b8913 from 4.14.0-rc6, my current kernel, eliminates the
> >> problem completely.
> >>
> >> Looking at the diff in that commit, it looks like the commit message isn't
> >> even accurate; not only is the priority of the dmcrypt workqueues being
> >> changed - they're also being made "CPU intensive" workqueues as well.
> >>
> >> This combination appears to result in both elevated scheduling priority and
> >> greater quantity of participant worker threads effectively starving any
> >> normal priority user task under periods of heavy IO on dmcrypt volumes.
> >>
> >> I don't know what the right solution is here.  It seems to me we're lacking
> >> the appropriate mechanism for charging CPU resources consumed on behalf of
> >> user processes in kworker threads to the work-causing process.
> >>
> >> What effectively happens is my normal `git` user process is able to
> >> greatly amplify what share of CPU it takes from the system by generating IO
> >> on what happens to be a high-priority CPU-intensive storage volume.
> >>
> >> It looks potentially complicated to fix properly, but I suspect at its core
> >> this may be a fairly longstanding shortcoming of the page cache and its
> >> asynchronous design.  Something that has been exacerbated substantially by
> >> the introduction of CPU-intensive storage subsystems like dmcrypt.
> >>
> >> If we imagine the whole stack simplified, where all the IO was being done
> >> synchronously in-band, and the dmcrypt kernel code simply ran in the
> >> IO-causing process context, it would be getting charged to the calling
> >> process and scheduled accordingly.  The resource accounting and scheduling
> >> problems all emerge with the page cache, buffered IO, and async background
> >> writeback in a pool of unrelated worker threads, etc.  That's how it
> >> appears to me anyways...
> >>
> >> The system used is a X61s Thinkpad 1.8Ghz with 840 EVO SSD, lvm on dmcrypt.
> >> The kernel .config is attached in case it's of interest.
> >>
> >> Thanks,
> >> Vito Caputo
> >
> >
> >
> > Ping...
> >
> > Could somebody please at least ACK receiving this so I'm not left wondering
> > if my mails to lkml are somehow winding up flagged as spam, thanks!
> 
> Sorry I did not notice your email before you ping me directly. It's
> interesting that issue, though we didn't notice this problem. It's a
> bit far since I tested this patch but I'll setup the environment again
> and do more tests to understand better what is happening.
> 

Any update on this?

I still experience it on 4.15-rc7 when doing sustained heavyweight git
checkouts without a1b8913 reverted.
 
Thanks,
Vito Caputo

Re: Commit fc72ae40e303 broke x86-64 build environment.

[vcaputo]

On Sat, Jan 13, 2018 at 11:13:13PM -0600, Rob Landley wrote:
> You've made the ORC unwinder part of allnoconfig, which means trying to
> build "make ARCH=x86_64 allnoconfig" requires installing a new package
> (libelf-dev) or else the build breaks.
> 
> What's worse, if I go into menuconfig and switch it back to frame
> pointer, the build STILL breaks:
> 
> $ make -j 8
> Makefile:932: *** "Cannot generate ORC metadata for
> CONFIG_UNWINDER_ORC=y, please install libelf-dev, libelf-devel or
> elfutils-libelf-devel".  Stop.
> $ grep UNWIND .config
> # CONFIG_UNWINDER_ORC is not set
> CONFIG_UNWINDER_FRAME_POINTER=y
> # CONFIG_UNWINDER_GUESS is not set
> 
> As far as I can tell, x86-64 doesn't build anymore without libelf-dev.
> It's a new hard requirement for the build.
> 

FYI this has already been brought up on lkml:
https://patchwork.kernel.org/patch/10137237/

IIRC you can get things working by deleting include/config/auto.conf
when you've switched back to CONFIG_UNWINDER_FRAME_POINTER.

I don't believe anything other than making CONFIG_UNWINDER_ORC the
default was intentional.  The frustration is just a consequence of some
build system bug.

Regards,
Vito Caputo

Re: Commit fc72ae40e303 broke x86-64 build environment.

[vcaputo]

On Sat, Jan 13, 2018 at 11:13:13PM -0600, Rob Landley wrote:
> You've made the ORC unwinder part of allnoconfig, which means trying to
> build "make ARCH=x86_64 allnoconfig" requires installing a new package
> (libelf-dev) or else the build breaks.
> 
> What's worse, if I go into menuconfig and switch it back to frame
> pointer, the build STILL breaks:
> 
> $ make -j 8
> Makefile:932: *** "Cannot generate ORC metadata for
> CONFIG_UNWINDER_ORC=y, please install libelf-dev, libelf-devel or
> elfutils-libelf-devel".  Stop.
> $ grep UNWIND .config
> # CONFIG_UNWINDER_ORC is not set
> CONFIG_UNWINDER_FRAME_POINTER=y
> # CONFIG_UNWINDER_GUESS is not set
> 
> As far as I can tell, x86-64 doesn't build anymore without libelf-dev.
> It's a new hard requirement for the build.
> 

FYI this has already been brought up on lkml:
https://patchwork.kernel.org/patch/10137237/

IIRC you can get things working by deleting include/config/auto.conf
when you've switched back to CONFIG_UNWINDER_FRAME_POINTER.

I don't believe anything other than making CONFIG_UNWINDER_ORC the
default was intentional.  The frustration is just a consequence of some
build system bug.

Regards,
Vito Caputo

Re: Linux 4.15-rc7

[vcaputo]

On Fri, Jan 12, 2018 at 11:08:58PM +0100, Arnd Bergmann wrote:
> On Fri, Jan 12, 2018 at 11:04 PM,   wrote:
> > On Fri, Jan 12, 2018 at 09:11:38PM +0100, Arnd Bergmann wrote:
> >
> >> - I haven't found a definite answer on whether Netburst-based CPUs
> >>   are affected by meltdown at all. Some people claim it's affected,
> >>   others say it's not. If the code from https://github.com/IAIK/meltdown
> >>   is successful on your Celeron D, then we know it's affected, if not,
> >>   then you could decide to not care about KPTI (Spectre would still
> >>   be an issue).
> >>
> >
> > I tried that when the code was first made public, but libkdump doesn't
> > support 32-bit; it's full of 64-bit register use in the assembly bits.
> 
> Apparently 32-bit support was added on Wednesday, maybe you
> can try again with today's version.
> 

Thanks for informing me of this, I hadn't noticed.

I just tried it out, and confirmed the Celeron D is vulnerable to
meltdown.

Regards,
Vito Caputo

Re: Linux 4.15-rc7

[vcaputo]

On Fri, Jan 12, 2018 at 11:08:58PM +0100, Arnd Bergmann wrote:
> On Fri, Jan 12, 2018 at 11:04 PM,   wrote:
> > On Fri, Jan 12, 2018 at 09:11:38PM +0100, Arnd Bergmann wrote:
> >
> >> - I haven't found a definite answer on whether Netburst-based CPUs
> >>   are affected by meltdown at all. Some people claim it's affected,
> >>   others say it's not. If the code from https://github.com/IAIK/meltdown
> >>   is successful on your Celeron D, then we know it's affected, if not,
> >>   then you could decide to not care about KPTI (Spectre would still
> >>   be an issue).
> >>
> >
> > I tried that when the code was first made public, but libkdump doesn't
> > support 32-bit; it's full of 64-bit register use in the assembly bits.
> 
> Apparently 32-bit support was added on Wednesday, maybe you
> can try again with today's version.
> 

Thanks for informing me of this, I hadn't noticed.

I just tried it out, and confirmed the Celeron D is vulnerable to
meltdown.

Regards,
Vito Caputo

Re: Linux 4.15-rc7

[vcaputo]

On Fri, Jan 12, 2018 at 09:11:38PM +0100, Arnd Bergmann wrote:
> On Fri, Jan 12, 2018 at 6:20 PM,   wrote:
> > On Fri, Jan 12, 2018 at 02:23:20PM +0100, Arnd Bergmann wrote:
> 
> >> Could you be more specific which 32-bit x86 chips you have that are
> >> affected by Meltdown? Do you mean pre-2004 Pentiums or Core-Duo
> >> laptops? I would guess that Cyrix/Natsemi/AMD 6x86/MediaGX/Geode
> >> and AMD NexGen K6/K7 also affected by Spectre but probably not
> >> Meltdown, and most other 32-bit microarchitectures seem to be purely
> >> in-order.
> >>
> >
> > I have some Celeron D, 4GiB dedicated servers with a 32-bit stack.
> > They've proven to be very reliable boxes, and are the most affordable
> > baremetal x86 machines I've found.  I'd appreciate a PTI implementation
> > on them.
> 
> That's an interesting setup for a number of reasons:
> 
> - Celeron D are mostly 64-bit CPUs, but it depends on the particular
>   model/stepping, so if you have a couple of them, you might be able to
>   avoid the meltdown bug by running a 64-bit kernel with KPTI at least on
>   some of them, or trivially replace the CPU on others. This usually
>   works without changing user space, and tends to result in a faster
>   system than running a 32-bit kernel as you avoid highmem.
>

This may be possible, I'll need to try booting a x86_64 kernel on one
and see.  I would rather not change all of userspace.

> - I haven't found a definite answer on whether Netburst-based CPUs
>   are affected by meltdown at all. Some people claim it's affected,
>   others say it's not. If the code from https://github.com/IAIK/meltdown
>   is successful on your Celeron D, then we know it's affected, if not,
>   then you could decide to not care about KPTI (Spectre would still
>   be an issue).
>

I tried that when the code was first made public, but libkdump doesn't
support 32-bit; it's full of 64-bit register use in the assembly bits.

> - A 32-bit system running with mostly highmem (only the low 768 MB
>   out of 4GB are directly mapped) means some of the exploits are
>   harder to do in practice, as most of the page cache is not visible
>   in the kernel, and reading data from other processes will fail more
>   often that succeed.
>

Well that's good news.

> - Economically, it seems barely worth running these if you pay for
>   the electricity: the CPU costs a few dollars/euros, it only takes
>   a couple of weeks of continuous operation to exceed that in
>   operating cost. Replacing the mainboard with a modern low end
>   all-in-one board at 10W might pay off within a year. If you don't pay
>   for electricity, that obviously doesn't work.
> 

I don't pay for the electricity, these are old dedicated servers hosted
by a third party.  Not my hardware, and any more modern dedicated x86
servers I've found are substantially more expensive and always SMP.

This particular hosting provider has tried selling me upgrades to their
current low-end offering (which is still SMP), the price basically
doubles.  These boxes are mostly idle, performing just personal email
and ssh duties.  For this situation reliability and security is the
priority, power efficiency and performance are not.

Thanks,
Vito Caputo

Re: Linux 4.15-rc7

[vcaputo]

On Fri, Jan 12, 2018 at 09:11:38PM +0100, Arnd Bergmann wrote:
> On Fri, Jan 12, 2018 at 6:20 PM,   wrote:
> > On Fri, Jan 12, 2018 at 02:23:20PM +0100, Arnd Bergmann wrote:
> 
> >> Could you be more specific which 32-bit x86 chips you have that are
> >> affected by Meltdown? Do you mean pre-2004 Pentiums or Core-Duo
> >> laptops? I would guess that Cyrix/Natsemi/AMD 6x86/MediaGX/Geode
> >> and AMD NexGen K6/K7 also affected by Spectre but probably not
> >> Meltdown, and most other 32-bit microarchitectures seem to be purely
> >> in-order.
> >>
> >
> > I have some Celeron D, 4GiB dedicated servers with a 32-bit stack.
> > They've proven to be very reliable boxes, and are the most affordable
> > baremetal x86 machines I've found.  I'd appreciate a PTI implementation
> > on them.
> 
> That's an interesting setup for a number of reasons:
> 
> - Celeron D are mostly 64-bit CPUs, but it depends on the particular
>   model/stepping, so if you have a couple of them, you might be able to
>   avoid the meltdown bug by running a 64-bit kernel with KPTI at least on
>   some of them, or trivially replace the CPU on others. This usually
>   works without changing user space, and tends to result in a faster
>   system than running a 32-bit kernel as you avoid highmem.
>

This may be possible, I'll need to try booting a x86_64 kernel on one
and see.  I would rather not change all of userspace.

> - I haven't found a definite answer on whether Netburst-based CPUs
>   are affected by meltdown at all. Some people claim it's affected,
>   others say it's not. If the code from https://github.com/IAIK/meltdown
>   is successful on your Celeron D, then we know it's affected, if not,
>   then you could decide to not care about KPTI (Spectre would still
>   be an issue).
>

I tried that when the code was first made public, but libkdump doesn't
support 32-bit; it's full of 64-bit register use in the assembly bits.

> - A 32-bit system running with mostly highmem (only the low 768 MB
>   out of 4GB are directly mapped) means some of the exploits are
>   harder to do in practice, as most of the page cache is not visible
>   in the kernel, and reading data from other processes will fail more
>   often that succeed.
>

Well that's good news.

> - Economically, it seems barely worth running these if you pay for
>   the electricity: the CPU costs a few dollars/euros, it only takes
>   a couple of weeks of continuous operation to exceed that in
>   operating cost. Replacing the mainboard with a modern low end
>   all-in-one board at 10W might pay off within a year. If you don't pay
>   for electricity, that obviously doesn't work.
> 

I don't pay for the electricity, these are old dedicated servers hosted
by a third party.  Not my hardware, and any more modern dedicated x86
servers I've found are substantially more expensive and always SMP.

This particular hosting provider has tried selling me upgrades to their
current low-end offering (which is still SMP), the price basically
doubles.  These boxes are mostly idle, performing just personal email
and ssh duties.  For this situation reliability and security is the
priority, power efficiency and performance are not.

Thanks,
Vito Caputo

Re: Linux 4.15-rc7

[vcaputo]

On Fri, Jan 12, 2018 at 02:23:20PM +0100, Arnd Bergmann wrote:
> On Fri, Jan 12, 2018 at 12:06 PM, Pavel Machek  wrote:
> > Hi!
> >
> >> Wasn't/Isn't the 4G/4G  memory layout for 32 bits essentially KPTI?
> >
> > Good point. Is that still supported? Was it ever?
> >
> > Umm. I seem to recall that 4G/4G layout was out of tree but never
> > merged.
> 
> I think that's correct: it was in RHEL3 and RHEL4 but never merged
> upstream.
> 
> However, there is an important difference between KPTI and X86_4G:
> The former unmaps the kernel pages from the user space page tables,
> but keeps both the linear mapping and the user pages visible in
> kernel mode, while the latter must have also unmapped user space
> pages from kernel mode, requiring a more expensive get_user/put_user
> implementation.
> 
> Kees mentioned an idea to also unmap user pages from kernel
> mode as an additional safeguard on top of KPTI, which would get
> it even closer to the X86_4G implementation:
> https://outflux.net/blog/archives/2018/01/04/smep-emulation-in-pti/
> 
> Could you be more specific which 32-bit x86 chips you have that are
> affected by Meltdown? Do you mean pre-2004 Pentiums or Core-Duo
> laptops? I would guess that Cyrix/Natsemi/AMD 6x86/MediaGX/Geode
> and AMD NexGen K6/K7 also affected by Spectre but probably not
> Meltdown, and most other 32-bit microarchitectures seem to be purely
> in-order.
> 

I have some Celeron D, 4GiB dedicated servers with a 32-bit stack.
They've proven to be very reliable boxes, and are the most affordable
baremetal x86 machines I've found.  I'd appreciate a PTI implementation
on them.

Thanks,
Vito Caputo

Re: Linux 4.15-rc7

[vcaputo]

On Fri, Jan 12, 2018 at 02:23:20PM +0100, Arnd Bergmann wrote:
> On Fri, Jan 12, 2018 at 12:06 PM, Pavel Machek  wrote:
> > Hi!
> >
> >> Wasn't/Isn't the 4G/4G  memory layout for 32 bits essentially KPTI?
> >
> > Good point. Is that still supported? Was it ever?
> >
> > Umm. I seem to recall that 4G/4G layout was out of tree but never
> > merged.
> 
> I think that's correct: it was in RHEL3 and RHEL4 but never merged
> upstream.
> 
> However, there is an important difference between KPTI and X86_4G:
> The former unmaps the kernel pages from the user space page tables,
> but keeps both the linear mapping and the user pages visible in
> kernel mode, while the latter must have also unmapped user space
> pages from kernel mode, requiring a more expensive get_user/put_user
> implementation.
> 
> Kees mentioned an idea to also unmap user pages from kernel
> mode as an additional safeguard on top of KPTI, which would get
> it even closer to the X86_4G implementation:
> https://outflux.net/blog/archives/2018/01/04/smep-emulation-in-pti/
> 
> Could you be more specific which 32-bit x86 chips you have that are
> affected by Meltdown? Do you mean pre-2004 Pentiums or Core-Duo
> laptops? I would guess that Cyrix/Natsemi/AMD 6x86/MediaGX/Geode
> and AMD NexGen K6/K7 also affected by Spectre but probably not
> Meltdown, and most other 32-bit microarchitectures seem to be purely
> in-order.
> 

I have some Celeron D, 4GiB dedicated servers with a 32-bit stack.
They've proven to be very reliable boxes, and are the most affordable
baremetal x86 machines I've found.  I'd appreciate a PTI implementation
on them.

Thanks,
Vito Caputo

Re: [BUG] 4.15-rc4 'do_IRQ: 0.33 No irq handler for vector'

[vcaputo]

On Sat, Dec 30, 2017 at 08:47:23PM +, Alan Cox wrote:
> On Mon, 18 Dec 2017 20:39:17 -0800
> vcap...@pengaru.com wrote:
> 
> > Hello everyone,
> > 
> > While investigating `journalctl -k` for anything associated with broken
> > audio, this was found:
> > 
> >   Dec 18 16:09:28 iridesce kernel: do_IRQ: 0.33 No irq handler for vector
> 
> That first one is a bit weird. But it looks like your machine got very
> upset.
> 
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: exception Emask 0x0 SAct 0xfe0 
> > SErr 0x0 action 0x6 frozen
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: failed command: WRITE FPDMA 
> > QUEUED
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: cmd 
> > 61/08:28:30:92:79/00:00:1c:00:00/40 tag 5 ncq dma 4096 out
> > res 
> > 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: status: { DRDY }
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: failed command: WRITE FPDMA 
> > QUEUED
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: cmd 
> > 61/08:30:10:20:68/00:00:00:00:00/40 tag 6 ncq dma 4096 out
> > res 
> > 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: status: { DRDY }
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: failed command: WRITE FPDMA 
> > QUEUED
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: cmd 
> > 61/08:38:30:5d:ec/00:00:1c:00:00/40 tag 7 ncq dma 4096 out
> > res 
> > 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: status: { DRDY }
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: failed command: WRITE FPDMA 
> > QUEUED
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: cmd 
> > 61/48:40:c0:75:64/00:00:00:00:00/40 tag 8 ncq dma 36864 out
> > res 
> > 40/00:fe:00:00:00/00:00:00:00:00/40 Emask 0x4 (timeout)
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: status: { DRDY }
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: failed command: WRITE FPDMA 
> > QUEUED
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: cmd 
> > 61/08:48:20:20:ec/00:00:1c:00:00/40 tag 9 ncq dma 4096 out
> > res 
> > 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: status: { DRDY }
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: failed command: WRITE FPDMA 
> > QUEUED
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: cmd 
> > 61/08:50:80:20:78/00:00:1c:00:00/40 tag 10 ncq dma 4096 out
> > res 
> > 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: status: { DRDY }
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: failed command: WRITE FPDMA 
> > QUEUED
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: cmd 
> > 61/08:58:90:3e:78/00:00:1c:00:00/40 tag 11 ncq dma 4096 out
> > res 
> > 40/00:01:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: status: { DRDY }
> 
> Your hard disk stopped responding
> 
> >   Dec 18 16:10:21 iridesce kernel: ata1: hard resetting link
> >   Dec 18 16:10:21 iridesce kernel: ata1: SATA link up 3.0 Gbps (SStatus 123 
> > SControl 300)
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: ACPI cmd ef/02:00:00:00:00:a0 
> > (SET FEATURES) succeeded
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: ACPI cmd f5/00:00:00:00:00:a0 
> > (SECURITY FREEZE LOCK) filtered out
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: ACPI cmd ef/10:03:00:00:00:a0 
> > (SET FEATURES) filtered out
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: supports DRM functions and may 
> > not be fully accessible
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: NCQ Send/Recv Log not supported
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: ACPI cmd ef/02:00:00:00:00:a0 
> > (SET FEATURES) succeeded
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: ACPI cmd f5/00:00:00:00:00:a0 
> > (SECURITY FREEZE LOCK) filtered out
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: ACPI cmd ef/10:03:00:00:00:a0 
> > (SET FEATURES) filtered out
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: supports DRM functions and may 
> > not be fully accessible
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: NCQ Send/Recv Log not supported
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: configured for UDMA/133
> 
> And while it kind of came back
> 
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: device reported invalid CHS 
> > sector 0
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: device reported invalid CHS 
> > sector 0
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: device reported invalid CHS 
> > sector 0
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: device reported invalid CHS 
> > sector 0
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: device 

Re: [BUG] 4.15-rc4 'do_IRQ: 0.33 No irq handler for vector'

[vcaputo]

On Sat, Dec 30, 2017 at 08:47:23PM +, Alan Cox wrote:
> On Mon, 18 Dec 2017 20:39:17 -0800
> vcap...@pengaru.com wrote:
> 
> > Hello everyone,
> > 
> > While investigating `journalctl -k` for anything associated with broken
> > audio, this was found:
> > 
> >   Dec 18 16:09:28 iridesce kernel: do_IRQ: 0.33 No irq handler for vector
> 
> That first one is a bit weird. But it looks like your machine got very
> upset.
> 
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: exception Emask 0x0 SAct 0xfe0 
> > SErr 0x0 action 0x6 frozen
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: failed command: WRITE FPDMA 
> > QUEUED
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: cmd 
> > 61/08:28:30:92:79/00:00:1c:00:00/40 tag 5 ncq dma 4096 out
> > res 
> > 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: status: { DRDY }
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: failed command: WRITE FPDMA 
> > QUEUED
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: cmd 
> > 61/08:30:10:20:68/00:00:00:00:00/40 tag 6 ncq dma 4096 out
> > res 
> > 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: status: { DRDY }
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: failed command: WRITE FPDMA 
> > QUEUED
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: cmd 
> > 61/08:38:30:5d:ec/00:00:1c:00:00/40 tag 7 ncq dma 4096 out
> > res 
> > 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: status: { DRDY }
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: failed command: WRITE FPDMA 
> > QUEUED
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: cmd 
> > 61/48:40:c0:75:64/00:00:00:00:00/40 tag 8 ncq dma 36864 out
> > res 
> > 40/00:fe:00:00:00/00:00:00:00:00/40 Emask 0x4 (timeout)
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: status: { DRDY }
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: failed command: WRITE FPDMA 
> > QUEUED
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: cmd 
> > 61/08:48:20:20:ec/00:00:1c:00:00/40 tag 9 ncq dma 4096 out
> > res 
> > 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: status: { DRDY }
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: failed command: WRITE FPDMA 
> > QUEUED
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: cmd 
> > 61/08:50:80:20:78/00:00:1c:00:00/40 tag 10 ncq dma 4096 out
> > res 
> > 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: status: { DRDY }
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: failed command: WRITE FPDMA 
> > QUEUED
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: cmd 
> > 61/08:58:90:3e:78/00:00:1c:00:00/40 tag 11 ncq dma 4096 out
> > res 
> > 40/00:01:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: status: { DRDY }
> 
> Your hard disk stopped responding
> 
> >   Dec 18 16:10:21 iridesce kernel: ata1: hard resetting link
> >   Dec 18 16:10:21 iridesce kernel: ata1: SATA link up 3.0 Gbps (SStatus 123 
> > SControl 300)
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: ACPI cmd ef/02:00:00:00:00:a0 
> > (SET FEATURES) succeeded
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: ACPI cmd f5/00:00:00:00:00:a0 
> > (SECURITY FREEZE LOCK) filtered out
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: ACPI cmd ef/10:03:00:00:00:a0 
> > (SET FEATURES) filtered out
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: supports DRM functions and may 
> > not be fully accessible
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: NCQ Send/Recv Log not supported
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: ACPI cmd ef/02:00:00:00:00:a0 
> > (SET FEATURES) succeeded
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: ACPI cmd f5/00:00:00:00:00:a0 
> > (SECURITY FREEZE LOCK) filtered out
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: ACPI cmd ef/10:03:00:00:00:a0 
> > (SET FEATURES) filtered out
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: supports DRM functions and may 
> > not be fully accessible
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: NCQ Send/Recv Log not supported
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: configured for UDMA/133
> 
> And while it kind of came back
> 
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: device reported invalid CHS 
> > sector 0
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: device reported invalid CHS 
> > sector 0
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: device reported invalid CHS 
> > sector 0
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: device reported invalid CHS 
> > sector 0
> >   Dec 18 16:10:21 iridesce kernel: ata1.00: device 

Re: [Regression 4.15] Can't kill CONFIG_UNWINDER_ORC with fire or plague.

[vcaputo]

On Fri, Dec 29, 2017 at 01:18:30PM -0500, Paul Gortmaker wrote:
> [Re: [Regression 4.15] Can't kill CONFIG_UNWINDER_ORC with fire or plague.] 
> On 29/12/2017 (Fri 10:47) Josh Poimboeuf wrote:
> 
> > This seems to be related to a kconfig quirk where only silentoldconfig
> > updates the include/config/auto.conf file.  The other config targets
> > (oldconfig, defconfig, etc) don't touch it.  It seems intentional, but I
> > have no idea why.
> > 
> > That causes the Makefile to get stale data for 'CONFIG_*' variables when
> > it includes auto.conf.  So I don't think this is specific to the ORC
> > check.  It seems like it could also cause bugs elsewhere.
> 
> OK, good - you agree with my initial diagnosis of stale auto.conf then.
> Not sure what Randy was testing when he said he couldn't reproduce it.
> 

FWIW just to add another data point, I tripped over this repeatedly
during a recent bisect of v4.14..v4.15-rc4.

My ad-hoc solution was to just delete the stale auto.conf and
reconfigure, IIRC.  It was pretty annoying at the time, and I assumed
everyone would be tripping over it so didn't bother making noise on
lkml.  Looks like I was mistaken...

Cheers,
Vito Caputo


> > The below (ugly) patch fixes it, though I'm not sure this is the best
> > way to do it.  We probably need Masahiro or Michal to chime in here.
> 
> Yep, that is why I intentionally put the kbuild folks on the To/Cc of
> the original report (and ran away screaming at the prospect of debugging
> Makefiles on xmas day).  But with holidays and all, it might not be
> until early January before they have a chance to reply.
> 
> Paul.
> --
> 
> > 
> > diff --git a/scripts/kconfig/Makefile b/scripts/kconfig/Makefile
> > index 297c1bf35140..3a0d92040a9c 100644
> > --- a/scripts/kconfig/Makefile
> > +++ b/scripts/kconfig/Makefile
> > @@ -21,18 +21,23 @@ unexport CONFIG_
> >  
> >  xconfig: $(obj)/qconf
> > $< $(silent) $(Kconfig)
> > +   $(Q)$(MAKE) -f $(srctree)/Makefile silentoldconfig
> >  
> >  gconfig: $(obj)/gconf
> > $< $(silent) $(Kconfig)
> > +   $(Q)$(MAKE) -f $(srctree)/Makefile silentoldconfig
> >  
> >  menuconfig: $(obj)/mconf
> > $< $(silent) $(Kconfig)
> > +   $(Q)$(MAKE) -f $(srctree)/Makefile silentoldconfig
> >  
> >  config: $(obj)/conf
> > $< $(silent) --oldaskconfig $(Kconfig)
> > +   $(Q)$(MAKE) -f $(srctree)/Makefile silentoldconfig
> >  
> >  nconfig: $(obj)/nconf
> > $< $(silent) $(Kconfig)
> > +   $(Q)$(MAKE) -f $(srctree)/Makefile silentoldconfig
> >  
> >  silentoldconfig: $(obj)/conf
> > $(Q)mkdir -p include/config include/generated
> > @@ -85,6 +90,7 @@ PHONY += $(simple-targets)
> >  
> >  $(simple-targets): $(obj)/conf
> > $< $(silent) --$@ $(Kconfig)
> > +   $(Q)$(MAKE) -f $(srctree)/Makefile silentoldconfig
> >  
> >  PHONY += oldnoconfig savedefconfig defconfig
> >  
> > @@ -108,9 +114,11 @@ else
> > $(Q)$(MAKE) -f $(srctree)/Makefile $(KBUILD_DEFCONFIG)
> >  endif
> >  endif
> > +   $(Q)$(MAKE) -f $(srctree)/Makefile silentoldconfig
> >  
> >  %_defconfig: $(obj)/conf
> > $(Q)$< $(silent) --defconfig=arch/$(SRCARCH)/configs/$@ $(Kconfig)
> > +   $(Q)$(MAKE) -f $(srctree)/Makefile silentoldconfig
> >  
> >  configfiles=$(wildcard $(srctree)/kernel/configs/$@ 
> > $(srctree)/arch/$(SRCARCH)/configs/$@)
> >  
> > @@ -118,6 +126,7 @@ configfiles=$(wildcard $(srctree)/kernel/configs/$@ 
> > $(srctree)/arch/$(SRCARCH)/c
> > $(if $(call configfiles),, $(error No configuration exists for this 
> > target on this architecture))
> > $(Q)$(CONFIG_SHELL) $(srctree)/scripts/kconfig/merge_config.sh -m 
> > .config $(configfiles)
> > +$(Q)yes "" | $(MAKE) -f $(srctree)/Makefile oldconfig
> > +   $(Q)$(MAKE) -f $(srctree)/Makefile silentoldconfig
> >  
> >  PHONY += kvmconfig
> >  kvmconfig: kvm_guest.config
> > @@ -130,6 +139,7 @@ xenconfig: xen.config
> >  PHONY += tinyconfig
> >  tinyconfig:
> > $(Q)$(MAKE) -f $(srctree)/Makefile allnoconfig tiny.config
> > +   $(Q)$(MAKE) -f $(srctree)/Makefile silentoldconfig
> >  
> >  # Help text used by make help
> >  help:

Re: [Regression 4.15] Can't kill CONFIG_UNWINDER_ORC with fire or plague.

[vcaputo]

On Fri, Dec 29, 2017 at 01:18:30PM -0500, Paul Gortmaker wrote:
> [Re: [Regression 4.15] Can't kill CONFIG_UNWINDER_ORC with fire or plague.] 
> On 29/12/2017 (Fri 10:47) Josh Poimboeuf wrote:
> 
> > This seems to be related to a kconfig quirk where only silentoldconfig
> > updates the include/config/auto.conf file.  The other config targets
> > (oldconfig, defconfig, etc) don't touch it.  It seems intentional, but I
> > have no idea why.
> > 
> > That causes the Makefile to get stale data for 'CONFIG_*' variables when
> > it includes auto.conf.  So I don't think this is specific to the ORC
> > check.  It seems like it could also cause bugs elsewhere.
> 
> OK, good - you agree with my initial diagnosis of stale auto.conf then.
> Not sure what Randy was testing when he said he couldn't reproduce it.
> 

FWIW just to add another data point, I tripped over this repeatedly
during a recent bisect of v4.14..v4.15-rc4.

My ad-hoc solution was to just delete the stale auto.conf and
reconfigure, IIRC.  It was pretty annoying at the time, and I assumed
everyone would be tripping over it so didn't bother making noise on
lkml.  Looks like I was mistaken...

Cheers,
Vito Caputo


> > The below (ugly) patch fixes it, though I'm not sure this is the best
> > way to do it.  We probably need Masahiro or Michal to chime in here.
> 
> Yep, that is why I intentionally put the kbuild folks on the To/Cc of
> the original report (and ran away screaming at the prospect of debugging
> Makefiles on xmas day).  But with holidays and all, it might not be
> until early January before they have a chance to reply.
> 
> Paul.
> --
> 
> > 
> > diff --git a/scripts/kconfig/Makefile b/scripts/kconfig/Makefile
> > index 297c1bf35140..3a0d92040a9c 100644
> > --- a/scripts/kconfig/Makefile
> > +++ b/scripts/kconfig/Makefile
> > @@ -21,18 +21,23 @@ unexport CONFIG_
> >  
> >  xconfig: $(obj)/qconf
> > $< $(silent) $(Kconfig)
> > +   $(Q)$(MAKE) -f $(srctree)/Makefile silentoldconfig
> >  
> >  gconfig: $(obj)/gconf
> > $< $(silent) $(Kconfig)
> > +   $(Q)$(MAKE) -f $(srctree)/Makefile silentoldconfig
> >  
> >  menuconfig: $(obj)/mconf
> > $< $(silent) $(Kconfig)
> > +   $(Q)$(MAKE) -f $(srctree)/Makefile silentoldconfig
> >  
> >  config: $(obj)/conf
> > $< $(silent) --oldaskconfig $(Kconfig)
> > +   $(Q)$(MAKE) -f $(srctree)/Makefile silentoldconfig
> >  
> >  nconfig: $(obj)/nconf
> > $< $(silent) $(Kconfig)
> > +   $(Q)$(MAKE) -f $(srctree)/Makefile silentoldconfig
> >  
> >  silentoldconfig: $(obj)/conf
> > $(Q)mkdir -p include/config include/generated
> > @@ -85,6 +90,7 @@ PHONY += $(simple-targets)
> >  
> >  $(simple-targets): $(obj)/conf
> > $< $(silent) --$@ $(Kconfig)
> > +   $(Q)$(MAKE) -f $(srctree)/Makefile silentoldconfig
> >  
> >  PHONY += oldnoconfig savedefconfig defconfig
> >  
> > @@ -108,9 +114,11 @@ else
> > $(Q)$(MAKE) -f $(srctree)/Makefile $(KBUILD_DEFCONFIG)
> >  endif
> >  endif
> > +   $(Q)$(MAKE) -f $(srctree)/Makefile silentoldconfig
> >  
> >  %_defconfig: $(obj)/conf
> > $(Q)$< $(silent) --defconfig=arch/$(SRCARCH)/configs/$@ $(Kconfig)
> > +   $(Q)$(MAKE) -f $(srctree)/Makefile silentoldconfig
> >  
> >  configfiles=$(wildcard $(srctree)/kernel/configs/$@ 
> > $(srctree)/arch/$(SRCARCH)/configs/$@)
> >  
> > @@ -118,6 +126,7 @@ configfiles=$(wildcard $(srctree)/kernel/configs/$@ 
> > $(srctree)/arch/$(SRCARCH)/c
> > $(if $(call configfiles),, $(error No configuration exists for this 
> > target on this architecture))
> > $(Q)$(CONFIG_SHELL) $(srctree)/scripts/kconfig/merge_config.sh -m 
> > .config $(configfiles)
> > +$(Q)yes "" | $(MAKE) -f $(srctree)/Makefile oldconfig
> > +   $(Q)$(MAKE) -f $(srctree)/Makefile silentoldconfig
> >  
> >  PHONY += kvmconfig
> >  kvmconfig: kvm_guest.config
> > @@ -130,6 +139,7 @@ xenconfig: xen.config
> >  PHONY += tinyconfig
> >  tinyconfig:
> > $(Q)$(MAKE) -f $(srctree)/Makefile allnoconfig tiny.config
> > +   $(Q)$(MAKE) -f $(srctree)/Makefile silentoldconfig
> >  
> >  # Help text used by make help
> >  help:

Re: thinkpad x60: sound problems in 4.15-rc1 was Re: thinkpad x60: sound problems in 4.14.0-next-20171114

[vcaputo]

On Thu, Dec 28, 2017 at 11:41:45AM +0100, Thomas Gleixner wrote:
> On Sun, 24 Dec 2017, vcap...@pengaru.com wrote:
> > On Sat, Dec 23, 2017 at 09:33:37PM +0100, Thomas Gleixner wrote:
> > > > It seems like the affinity changes are assuming a strict adherence to
> > > > the CPU mask when the underlying hardware is treating it more as a hint.
> > > > Perhaps handlers still need to be maintained on all CPUs in a given apic
> > > > domain, regardless of what the masks are configured as, to cover these
> > > > situations.
> > > 
> > > That's odd. I'll have a look after the holidays.
> > > 
> > 
> > Ok, just FYI I've reproduced it on rc5 as well.
> > 
> > I may be offline a bit at the start of the new year, in case you've got
> > something for me to test and I'm unresponsive.
> 
> Can you try the patch below?
> 

Looks fixed so far, I'll try living in 4.15-rc5 now and will report back
if anything goes sideways.

Thanks Thomas!


> Thanks,
> 
>   tglx
> 
> 8<---
> --- a/arch/x86/kernel/apic/apic_flat_64.c
> +++ b/arch/x86/kernel/apic/apic_flat_64.c
> @@ -151,7 +151,7 @@ static struct apic apic_flat __ro_after_
>   .apic_id_valid  = default_apic_id_valid,
>   .apic_id_registered = flat_apic_id_registered,
>  
> - .irq_delivery_mode  = dest_LowestPrio,
> + .irq_delivery_mode  = dest_Fixed,
>   .irq_dest_mode  = 1, /* logical */
>  
>   .disable_esr= 0,
> --- a/arch/x86/kernel/apic/probe_32.c
> +++ b/arch/x86/kernel/apic/probe_32.c
> @@ -105,7 +105,7 @@ static struct apic apic_default __ro_aft
>   .apic_id_valid  = default_apic_id_valid,
>   .apic_id_registered = default_apic_id_registered,
>  
> - .irq_delivery_mode  = dest_LowestPrio,
> + .irq_delivery_mode  = dest_Fixed,
>   /* logical delivery broadcast to all CPUs: */
>   .irq_dest_mode  = 1,
>  
> --- a/arch/x86/kernel/apic/x2apic_cluster.c
> +++ b/arch/x86/kernel/apic/x2apic_cluster.c
> @@ -184,7 +184,7 @@ static struct apic apic_x2apic_cluster _
>   .apic_id_valid  = x2apic_apic_id_valid,
>   .apic_id_registered = x2apic_apic_id_registered,
>  
> - .irq_delivery_mode  = dest_LowestPrio,
> + .irq_delivery_mode  = dest_Fixed,
>   .irq_dest_mode  = 1, /* logical */
>  
>   .disable_esr= 0,

Re: thinkpad x60: sound problems in 4.15-rc1 was Re: thinkpad x60: sound problems in 4.14.0-next-20171114

[vcaputo]

On Thu, Dec 28, 2017 at 11:41:45AM +0100, Thomas Gleixner wrote:
> On Sun, 24 Dec 2017, vcap...@pengaru.com wrote:
> > On Sat, Dec 23, 2017 at 09:33:37PM +0100, Thomas Gleixner wrote:
> > > > It seems like the affinity changes are assuming a strict adherence to
> > > > the CPU mask when the underlying hardware is treating it more as a hint.
> > > > Perhaps handlers still need to be maintained on all CPUs in a given apic
> > > > domain, regardless of what the masks are configured as, to cover these
> > > > situations.
> > > 
> > > That's odd. I'll have a look after the holidays.
> > > 
> > 
> > Ok, just FYI I've reproduced it on rc5 as well.
> > 
> > I may be offline a bit at the start of the new year, in case you've got
> > something for me to test and I'm unresponsive.
> 
> Can you try the patch below?
> 

Looks fixed so far, I'll try living in 4.15-rc5 now and will report back
if anything goes sideways.

Thanks Thomas!


> Thanks,
> 
>   tglx
> 
> 8<---
> --- a/arch/x86/kernel/apic/apic_flat_64.c
> +++ b/arch/x86/kernel/apic/apic_flat_64.c
> @@ -151,7 +151,7 @@ static struct apic apic_flat __ro_after_
>   .apic_id_valid  = default_apic_id_valid,
>   .apic_id_registered = flat_apic_id_registered,
>  
> - .irq_delivery_mode  = dest_LowestPrio,
> + .irq_delivery_mode  = dest_Fixed,
>   .irq_dest_mode  = 1, /* logical */
>  
>   .disable_esr= 0,
> --- a/arch/x86/kernel/apic/probe_32.c
> +++ b/arch/x86/kernel/apic/probe_32.c
> @@ -105,7 +105,7 @@ static struct apic apic_default __ro_aft
>   .apic_id_valid  = default_apic_id_valid,
>   .apic_id_registered = default_apic_id_registered,
>  
> - .irq_delivery_mode  = dest_LowestPrio,
> + .irq_delivery_mode  = dest_Fixed,
>   /* logical delivery broadcast to all CPUs: */
>   .irq_dest_mode  = 1,
>  
> --- a/arch/x86/kernel/apic/x2apic_cluster.c
> +++ b/arch/x86/kernel/apic/x2apic_cluster.c
> @@ -184,7 +184,7 @@ static struct apic apic_x2apic_cluster _
>   .apic_id_valid  = x2apic_apic_id_valid,
>   .apic_id_registered = x2apic_apic_id_registered,
>  
> - .irq_delivery_mode  = dest_LowestPrio,
> + .irq_delivery_mode  = dest_Fixed,
>   .irq_dest_mode  = 1, /* logical */
>  
>   .disable_esr= 0,

Re: thinkpad x60: sound problems in 4.15-rc1 was Re: thinkpad x60: sound problems in 4.14.0-next-20171114

[vcaputo]

On Sat, Dec 23, 2017 at 09:33:37PM +0100, Thomas Gleixner wrote:
> On Sat, 23 Dec 2017, vcap...@pengaru.com wrote:
> > On Fri, Dec 22, 2017 at 09:37:01PM -0800, vcap...@pengaru.com wrote:
> > Added the following instrumentation:
> > 
> > diff --git a/arch/x86/kernel/apic/vector.c b/arch/x86/kernel/apic/vector.c
> > index 93edc2236282..7034eda4d494 100644
> > --- a/arch/x86/kernel/apic/vector.c
> > +++ b/arch/x86/kernel/apic/vector.c
> > @@ -228,6 +228,9 @@ static int __assign_irq_vector(int irq, struct 
> > apic_chip_data *d,
> > cpumask_and(vector_searchmask, vector_searchmask, mask);
> > BUG_ON(apic->cpu_mask_to_apicid(vector_searchmask, irqdata,
> > >cfg.dest_apicid));
> > +
> > +   printk("allocated vector=%i maskfirst=%i\n", d->cfg.vector, 
> > cpumask_first(vector_searchmask));
> > +
> > return 0;
> >  }
> >  
> > This is what I see:
> > 
> > Upon playing song in cmus (on AC power since boot):
> >  Dec 22 22:26:52 iridesce kernel: allocated vector=35 maskfirst=1
> > 
> > Yank AC:
> >  Dec 22 22:27:14 iridesce kernel: allocated vector=51 maskfirst=1
> >  Dec 22 22:27:15 iridesce kernel: do_IRQ: 0.35 No irq handler for vector
> > 
> > So CPU 0 vector 35 got an interrupt when maskfirst=1 for 35 as seen in
> > the added printk.
> > 
> > It seems like the affinity changes are assuming a strict adherence to
> > the CPU mask when the underlying hardware is treating it more as a hint.
> > Perhaps handlers still need to be maintained on all CPUs in a given apic
> > domain, regardless of what the masks are configured as, to cover these
> > situations.
> 
> That's odd. I'll have a look after the holidays.
> 

Ok, just FYI I've reproduced it on rc5 as well.

I may be offline a bit at the start of the new year, in case you've got
something for me to test and I'm unresponsive.

Regards,
Vito Caputo

Re: thinkpad x60: sound problems in 4.15-rc1 was Re: thinkpad x60: sound problems in 4.14.0-next-20171114

[vcaputo]

On Sat, Dec 23, 2017 at 09:33:37PM +0100, Thomas Gleixner wrote:
> On Sat, 23 Dec 2017, vcap...@pengaru.com wrote:
> > On Fri, Dec 22, 2017 at 09:37:01PM -0800, vcap...@pengaru.com wrote:
> > Added the following instrumentation:
> > 
> > diff --git a/arch/x86/kernel/apic/vector.c b/arch/x86/kernel/apic/vector.c
> > index 93edc2236282..7034eda4d494 100644
> > --- a/arch/x86/kernel/apic/vector.c
> > +++ b/arch/x86/kernel/apic/vector.c
> > @@ -228,6 +228,9 @@ static int __assign_irq_vector(int irq, struct 
> > apic_chip_data *d,
> > cpumask_and(vector_searchmask, vector_searchmask, mask);
> > BUG_ON(apic->cpu_mask_to_apicid(vector_searchmask, irqdata,
> > >cfg.dest_apicid));
> > +
> > +   printk("allocated vector=%i maskfirst=%i\n", d->cfg.vector, 
> > cpumask_first(vector_searchmask));
> > +
> > return 0;
> >  }
> >  
> > This is what I see:
> > 
> > Upon playing song in cmus (on AC power since boot):
> >  Dec 22 22:26:52 iridesce kernel: allocated vector=35 maskfirst=1
> > 
> > Yank AC:
> >  Dec 22 22:27:14 iridesce kernel: allocated vector=51 maskfirst=1
> >  Dec 22 22:27:15 iridesce kernel: do_IRQ: 0.35 No irq handler for vector
> > 
> > So CPU 0 vector 35 got an interrupt when maskfirst=1 for 35 as seen in
> > the added printk.
> > 
> > It seems like the affinity changes are assuming a strict adherence to
> > the CPU mask when the underlying hardware is treating it more as a hint.
> > Perhaps handlers still need to be maintained on all CPUs in a given apic
> > domain, regardless of what the masks are configured as, to cover these
> > situations.
> 
> That's odd. I'll have a look after the holidays.
> 

Ok, just FYI I've reproduced it on rc5 as well.

I may be offline a bit at the start of the new year, in case you've got
something for me to test and I'm unresponsive.

Regards,
Vito Caputo

Re: thinkpad x60: sound problems in 4.15-rc1 was Re: thinkpad x60: sound problems in 4.14.0-next-20171114

[vcaputo]

On Fri, Dec 22, 2017 at 09:37:01PM -0800, vcap...@pengaru.com wrote:
> On Wed, Dec 20, 2017 at 01:33:45AM +0100, Thomas Gleixner wrote:
> > On Tue, 19 Dec 2017, vcap...@pengaru.com wrote:
> > > On Wed, Dec 20, 2017 at 12:22:12AM +0100, Pavel Machek wrote:
> > > > You forgot to mention commit id :-).
> > > > 
> > > 
> > > That is very strange, anyhow:
> > > 
> > >  commit fdba46ffb4c203b6e6794163493fd310f98bb4be
> > >  Author: Thomas Gleixner 
> > >  Date:   Wed Sep 13 23:29:27 2017 +0200
> > > 
> > >  x86/apic: Get rid of multi CPU affinity
> > > 
> > > 
> > > Will try reverting soon, just a bit busy today out in the desert and the 
> > > sun
> > > is going down so my solar panel is useless.
> > 
> > The revert is not trivial.
> > 
> > What is the exact problem and how do you reproduce that?
> > 
> > Thanks,
> > 
> 
> So I had some time today to poke at this some more.  Since it looks to
> be easily reproduced by simply pulling the AC power while playing music
> or doing IO, and dmesg clearly reports using mwait, I tried booting with
> idle=nomwait to see if that made any difference.  It didn't, the same
> thing still occurs.
> 
> In trying to make sense of this totally unfamiliar apic code and better
> understand these changes, I came across this comment which seemed a bit
> telling:
> 
>   40 void flat_vector_allocation_domain(int cpu, struct cpumask *retmask,
>   41const struct cpumask *mask)
>   42 {
>   43 /*
>   44  * Careful. Some cpus do not strictly honor the set of cpus
>   45  * specified in the interrupt destination when using lowest
>   46  * priority interrupt delivery mode.
>   47  *
>   48  * In particular there was a hyperthreading cpu observed to
>   49  * deliver interrupts to the wrong hyperthread when only one
>   50  * hyperthread was specified in the interrupt desitination.
>   51  */
>   52 cpumask_clear(retmask);
>   53 cpumask_bits(retmask)[0] = APIC_ALL_CPUS;
>   54 }
> 
> It's this allocation domain mask hook which has been bypassed by the
> offending commit.  The existing approach is more robust in the face of
> relaxed adherence to destination cpumasks since it's all-inclusive,
> whereas the new code is exclusive to a specific cpu.
> 
> Is it possible what I'm observing is just another manifestation of
> what's being described in that comment?  This is a core 2 duo, so not
> hyper-threaded.  But maybe something funny happens when switching
> cstates in response to interrupts - like maybe the wrong cpu can be used
> if it can save power vs. powering up another?  Just thinking out loud
> here.
> 
> In any case, 4.15-rc4 is quite unusable on my machine because of this.
> 

Some more food for thought:

Added the following instrumentation:

diff --git a/arch/x86/kernel/apic/vector.c b/arch/x86/kernel/apic/vector.c
index 93edc2236282..7034eda4d494 100644
--- a/arch/x86/kernel/apic/vector.c
+++ b/arch/x86/kernel/apic/vector.c
@@ -228,6 +228,9 @@ static int __assign_irq_vector(int irq, struct 
apic_chip_data *d,
cpumask_and(vector_searchmask, vector_searchmask, mask);
BUG_ON(apic->cpu_mask_to_apicid(vector_searchmask, irqdata,
>cfg.dest_apicid));
+
+   printk("allocated vector=%i maskfirst=%i\n", d->cfg.vector, 
cpumask_first(vector_searchmask));
+
return 0;
 }
 
This is what I see:

Upon playing song in cmus (on AC power since boot):
 Dec 22 22:26:52 iridesce kernel: allocated vector=35 maskfirst=1

Yank AC:
 Dec 22 22:27:14 iridesce kernel: allocated vector=51 maskfirst=1
 Dec 22 22:27:15 iridesce kernel: do_IRQ: 0.35 No irq handler for vector

So CPU 0 vector 35 got an interrupt when maskfirst=1 for 35 as seen in
the added printk.

It seems like the affinity changes are assuming a strict adherence to
the CPU mask when the underlying hardware is treating it more as a hint.
Perhaps handlers still need to be maintained on all CPUs in a given apic
domain, regardless of what the masks are configured as, to cover these
situations.

Regards,
Vito Caputo

Re: thinkpad x60: sound problems in 4.15-rc1 was Re: thinkpad x60: sound problems in 4.14.0-next-20171114

[vcaputo]

On Fri, Dec 22, 2017 at 09:37:01PM -0800, vcap...@pengaru.com wrote:
> On Wed, Dec 20, 2017 at 01:33:45AM +0100, Thomas Gleixner wrote:
> > On Tue, 19 Dec 2017, vcap...@pengaru.com wrote:
> > > On Wed, Dec 20, 2017 at 12:22:12AM +0100, Pavel Machek wrote:
> > > > You forgot to mention commit id :-).
> > > > 
> > > 
> > > That is very strange, anyhow:
> > > 
> > >  commit fdba46ffb4c203b6e6794163493fd310f98bb4be
> > >  Author: Thomas Gleixner 
> > >  Date:   Wed Sep 13 23:29:27 2017 +0200
> > > 
> > >  x86/apic: Get rid of multi CPU affinity
> > > 
> > > 
> > > Will try reverting soon, just a bit busy today out in the desert and the 
> > > sun
> > > is going down so my solar panel is useless.
> > 
> > The revert is not trivial.
> > 
> > What is the exact problem and how do you reproduce that?
> > 
> > Thanks,
> > 
> 
> So I had some time today to poke at this some more.  Since it looks to
> be easily reproduced by simply pulling the AC power while playing music
> or doing IO, and dmesg clearly reports using mwait, I tried booting with
> idle=nomwait to see if that made any difference.  It didn't, the same
> thing still occurs.
> 
> In trying to make sense of this totally unfamiliar apic code and better
> understand these changes, I came across this comment which seemed a bit
> telling:
> 
>   40 void flat_vector_allocation_domain(int cpu, struct cpumask *retmask,
>   41const struct cpumask *mask)
>   42 {
>   43 /*
>   44  * Careful. Some cpus do not strictly honor the set of cpus
>   45  * specified in the interrupt destination when using lowest
>   46  * priority interrupt delivery mode.
>   47  *
>   48  * In particular there was a hyperthreading cpu observed to
>   49  * deliver interrupts to the wrong hyperthread when only one
>   50  * hyperthread was specified in the interrupt desitination.
>   51  */
>   52 cpumask_clear(retmask);
>   53 cpumask_bits(retmask)[0] = APIC_ALL_CPUS;
>   54 }
> 
> It's this allocation domain mask hook which has been bypassed by the
> offending commit.  The existing approach is more robust in the face of
> relaxed adherence to destination cpumasks since it's all-inclusive,
> whereas the new code is exclusive to a specific cpu.
> 
> Is it possible what I'm observing is just another manifestation of
> what's being described in that comment?  This is a core 2 duo, so not
> hyper-threaded.  But maybe something funny happens when switching
> cstates in response to interrupts - like maybe the wrong cpu can be used
> if it can save power vs. powering up another?  Just thinking out loud
> here.
> 
> In any case, 4.15-rc4 is quite unusable on my machine because of this.
> 

Some more food for thought:

Added the following instrumentation:

diff --git a/arch/x86/kernel/apic/vector.c b/arch/x86/kernel/apic/vector.c
index 93edc2236282..7034eda4d494 100644
--- a/arch/x86/kernel/apic/vector.c
+++ b/arch/x86/kernel/apic/vector.c
@@ -228,6 +228,9 @@ static int __assign_irq_vector(int irq, struct 
apic_chip_data *d,
cpumask_and(vector_searchmask, vector_searchmask, mask);
BUG_ON(apic->cpu_mask_to_apicid(vector_searchmask, irqdata,
>cfg.dest_apicid));
+
+   printk("allocated vector=%i maskfirst=%i\n", d->cfg.vector, 
cpumask_first(vector_searchmask));
+
return 0;
 }
 
This is what I see:

Upon playing song in cmus (on AC power since boot):
 Dec 22 22:26:52 iridesce kernel: allocated vector=35 maskfirst=1

Yank AC:
 Dec 22 22:27:14 iridesce kernel: allocated vector=51 maskfirst=1
 Dec 22 22:27:15 iridesce kernel: do_IRQ: 0.35 No irq handler for vector

So CPU 0 vector 35 got an interrupt when maskfirst=1 for 35 as seen in
the added printk.

It seems like the affinity changes are assuming a strict adherence to
the CPU mask when the underlying hardware is treating it more as a hint.
Perhaps handlers still need to be maintained on all CPUs in a given apic
domain, regardless of what the masks are configured as, to cover these
situations.

Regards,
Vito Caputo

Re: thinkpad x60: sound problems in 4.15-rc1 was Re: thinkpad x60: sound problems in 4.14.0-next-20171114

[vcaputo]

On Wed, Dec 20, 2017 at 01:33:45AM +0100, Thomas Gleixner wrote:
> On Tue, 19 Dec 2017, vcap...@pengaru.com wrote:
> > On Wed, Dec 20, 2017 at 12:22:12AM +0100, Pavel Machek wrote:
> > > You forgot to mention commit id :-).
> > > 
> > 
> > That is very strange, anyhow:
> > 
> >  commit fdba46ffb4c203b6e6794163493fd310f98bb4be
> >  Author: Thomas Gleixner 
> >  Date:   Wed Sep 13 23:29:27 2017 +0200
> > 
> >  x86/apic: Get rid of multi CPU affinity
> > 
> > 
> > Will try reverting soon, just a bit busy today out in the desert and the sun
> > is going down so my solar panel is useless.
> 
> The revert is not trivial.
> 
> What is the exact problem and how do you reproduce that?
> 
> Thanks,
> 

So I had some time today to poke at this some more.  Since it looks to
be easily reproduced by simply pulling the AC power while playing music
or doing IO, and dmesg clearly reports using mwait, I tried booting with
idle=nomwait to see if that made any difference.  It didn't, the same
thing still occurs.

In trying to make sense of this totally unfamiliar apic code and better
understand these changes, I came across this comment which seemed a bit
telling:

  40 void flat_vector_allocation_domain(int cpu, struct cpumask *retmask,
  41const struct cpumask *mask)
  42 {
  43 /*
  44  * Careful. Some cpus do not strictly honor the set of cpus
  45  * specified in the interrupt destination when using lowest
  46  * priority interrupt delivery mode.
  47  *
  48  * In particular there was a hyperthreading cpu observed to
  49  * deliver interrupts to the wrong hyperthread when only one
  50  * hyperthread was specified in the interrupt desitination.
  51  */
  52 cpumask_clear(retmask);
  53 cpumask_bits(retmask)[0] = APIC_ALL_CPUS;
  54 }

It's this allocation domain mask hook which has been bypassed by the
offending commit.  The existing approach is more robust in the face of
relaxed adherence to destination cpumasks since it's all-inclusive,
whereas the new code is exclusive to a specific cpu.

Is it possible what I'm observing is just another manifestation of
what's being described in that comment?  This is a core 2 duo, so not
hyper-threaded.  But maybe something funny happens when switching
cstates in response to interrupts - like maybe the wrong cpu can be used
if it can save power vs. powering up another?  Just thinking out loud
here.

In any case, 4.15-rc4 is quite unusable on my machine because of this.

Pavel, do you observe the same behavior on your x60, WRT AC power?

I've dropped Takashi from the CC list as this pretty clearly isn't a
sound-specific problem.

Thanks,
Vito Caputo

Re: thinkpad x60: sound problems in 4.15-rc1 was Re: thinkpad x60: sound problems in 4.14.0-next-20171114

[vcaputo]

On Wed, Dec 20, 2017 at 01:33:45AM +0100, Thomas Gleixner wrote:
> On Tue, 19 Dec 2017, vcap...@pengaru.com wrote:
> > On Wed, Dec 20, 2017 at 12:22:12AM +0100, Pavel Machek wrote:
> > > You forgot to mention commit id :-).
> > > 
> > 
> > That is very strange, anyhow:
> > 
> >  commit fdba46ffb4c203b6e6794163493fd310f98bb4be
> >  Author: Thomas Gleixner 
> >  Date:   Wed Sep 13 23:29:27 2017 +0200
> > 
> >  x86/apic: Get rid of multi CPU affinity
> > 
> > 
> > Will try reverting soon, just a bit busy today out in the desert and the sun
> > is going down so my solar panel is useless.
> 
> The revert is not trivial.
> 
> What is the exact problem and how do you reproduce that?
> 
> Thanks,
> 

So I had some time today to poke at this some more.  Since it looks to
be easily reproduced by simply pulling the AC power while playing music
or doing IO, and dmesg clearly reports using mwait, I tried booting with
idle=nomwait to see if that made any difference.  It didn't, the same
thing still occurs.

In trying to make sense of this totally unfamiliar apic code and better
understand these changes, I came across this comment which seemed a bit
telling:

  40 void flat_vector_allocation_domain(int cpu, struct cpumask *retmask,
  41const struct cpumask *mask)
  42 {
  43 /*
  44  * Careful. Some cpus do not strictly honor the set of cpus
  45  * specified in the interrupt destination when using lowest
  46  * priority interrupt delivery mode.
  47  *
  48  * In particular there was a hyperthreading cpu observed to
  49  * deliver interrupts to the wrong hyperthread when only one
  50  * hyperthread was specified in the interrupt desitination.
  51  */
  52 cpumask_clear(retmask);
  53 cpumask_bits(retmask)[0] = APIC_ALL_CPUS;
  54 }

It's this allocation domain mask hook which has been bypassed by the
offending commit.  The existing approach is more robust in the face of
relaxed adherence to destination cpumasks since it's all-inclusive,
whereas the new code is exclusive to a specific cpu.

Is it possible what I'm observing is just another manifestation of
what's being described in that comment?  This is a core 2 duo, so not
hyper-threaded.  But maybe something funny happens when switching
cstates in response to interrupts - like maybe the wrong cpu can be used
if it can save power vs. powering up another?  Just thinking out loud
here.

In any case, 4.15-rc4 is quite unusable on my machine because of this.

Pavel, do you observe the same behavior on your x60, WRT AC power?

I've dropped Takashi from the CC list as this pretty clearly isn't a
sound-specific problem.

Thanks,
Vito Caputo

Re: thinkpad x60: sound problems in 4.15-rc1 was Re: thinkpad x60: sound problems in 4.14.0-next-20171114

[vcaputo]

On Wed, Dec 20, 2017 at 01:33:45AM +0100, Thomas Gleixner wrote:
> On Tue, 19 Dec 2017, vcap...@pengaru.com wrote:
> > On Wed, Dec 20, 2017 at 12:22:12AM +0100, Pavel Machek wrote:
> > > You forgot to mention commit id :-).
> > > 
> > 
> > That is very strange, anyhow:
> > 
> >  commit fdba46ffb4c203b6e6794163493fd310f98bb4be
> >  Author: Thomas Gleixner 
> >  Date:   Wed Sep 13 23:29:27 2017 +0200
> > 
> >  x86/apic: Get rid of multi CPU affinity
> > 
> > 
> > Will try reverting soon, just a bit busy today out in the desert and the sun
> > is going down so my solar panel is useless.
> 
> The revert is not trivial.
> 
> What is the exact problem and how do you reproduce that?
> 

Dang.

Ostensibly the problem is audio playback looping what seems to be stuck in
a DMA buffer when I pause the audio.

I also saw once during the bisect on a 'bad' commit, a reproduction of
this, which hung everything doing IO until it the ata1 reset happened:

[   36.606657] Monitor-Mwait will be used to enter C-3 state
[   36.628663] do_IRQ: 0.35 No irq handler for vector
[   37.875724] do_IRQ: 0.194 No irq handler for vector
[   69.099090] ata1.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
[   69.099165] ata1.00: failed command: FLUSH CACHE EXT
[   69.099211] ata1.00: cmd ea/00:00:00:00:00/00:00:00:00:00/a0 tag 6
res 40/00:01:00:00:00/00:00:00:00:00/00 Emask 0x4 
(timeout)
[   69.099285] ata1.00: status: { DRDY }
[   69.099309] ata1: hard resetting link
[   69.406185] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[   69.409255] ata1.00: ACPI cmd ef/02:00:00:00:00:a0 (SET FEATURES) succeeded
[   69.409259] ata1.00: ACPI cmd f5/00:00:00:00:00:a0 (SECURITY FREEZE LOCK) 
filtered out
[   69.409261] ata1.00: ACPI cmd ef/10:03:00:00:00:a0 (SET FEATURES) filtered 
out
[   69.409433] ata1.00: supports DRM functions and may not be fully accessible
[   69.409819] ata1.00: NCQ Send/Recv Log not supported
[   69.411644] ata1.00: ACPI cmd ef/02:00:00:00:00:a0 (SET FEATURES) succeeded
[   69.411649] ata1.00: ACPI cmd f5/00:00:00:00:00:a0 (SECURITY FREEZE LOCK) 
filtered out
[   69.411654] ata1.00: ACPI cmd ef/10:03:00:00:00:a0 (SET FEATURES) filtered 
out
[   69.411842] ata1.00: supports DRM functions and may not be fully accessible
[   69.412242] ata1.00: NCQ Send/Recv Log not supported
[   69.412719] ata1.00: configured for UDMA/133
[   69.412721] ata1.00: retrying FLUSH 0xea Emask 0x4
[   69.412964] ata1: EH complete

But that is more difficult to reproduce, it doesn't seem to happen
regularly.  Infact, I thought that was independent from the audio problem
at first but now it's become clear they're all related.

The 'bad' commits always showed the 'do_IRQ: No irq handler for vector'
lines, sometimes they appear for the first time on the console when I
yanked the power cord while playing music in the quick repro cycle without
running X or anything.

What I've been doing to reproduce this is simply boot to multi-user.target,
login, run `cmus`, and play a song.  Next pull the AC power from the
thinkpad x61s.

Immediately the audio gets messed up, pausing the audio doesn't pause it,
it just loops the last tiny buffer contents.

That's all I've got right now, but it doesn't seem to be limited to the
audio problem as shown by the ata1 reset.  Also I've searched through all
my stored journals for anything resembling that ata1 problem, and there's
not a single occurrence going back ~300 boots across a handful of kernel
versions.

If you'd like to prep a patch for me to test, I'm happy to test, but I'm
not sure how online I'll be for the next 24 hours.  It's some kind of
miracle we're communicating as-is, I'm in the middle of the damn desert but
somehow there's a cell signal.

Thanks,
Vito Caputo

Re: thinkpad x60: sound problems in 4.15-rc1 was Re: thinkpad x60: sound problems in 4.14.0-next-20171114

[vcaputo]

On Wed, Dec 20, 2017 at 01:33:45AM +0100, Thomas Gleixner wrote:
> On Tue, 19 Dec 2017, vcap...@pengaru.com wrote:
> > On Wed, Dec 20, 2017 at 12:22:12AM +0100, Pavel Machek wrote:
> > > You forgot to mention commit id :-).
> > > 
> > 
> > That is very strange, anyhow:
> > 
> >  commit fdba46ffb4c203b6e6794163493fd310f98bb4be
> >  Author: Thomas Gleixner 
> >  Date:   Wed Sep 13 23:29:27 2017 +0200
> > 
> >  x86/apic: Get rid of multi CPU affinity
> > 
> > 
> > Will try reverting soon, just a bit busy today out in the desert and the sun
> > is going down so my solar panel is useless.
> 
> The revert is not trivial.
> 
> What is the exact problem and how do you reproduce that?
> 

Dang.

Ostensibly the problem is audio playback looping what seems to be stuck in
a DMA buffer when I pause the audio.

I also saw once during the bisect on a 'bad' commit, a reproduction of
this, which hung everything doing IO until it the ata1 reset happened:

[   36.606657] Monitor-Mwait will be used to enter C-3 state
[   36.628663] do_IRQ: 0.35 No irq handler for vector
[   37.875724] do_IRQ: 0.194 No irq handler for vector
[   69.099090] ata1.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
[   69.099165] ata1.00: failed command: FLUSH CACHE EXT
[   69.099211] ata1.00: cmd ea/00:00:00:00:00/00:00:00:00:00/a0 tag 6
res 40/00:01:00:00:00/00:00:00:00:00/00 Emask 0x4 
(timeout)
[   69.099285] ata1.00: status: { DRDY }
[   69.099309] ata1: hard resetting link
[   69.406185] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[   69.409255] ata1.00: ACPI cmd ef/02:00:00:00:00:a0 (SET FEATURES) succeeded
[   69.409259] ata1.00: ACPI cmd f5/00:00:00:00:00:a0 (SECURITY FREEZE LOCK) 
filtered out
[   69.409261] ata1.00: ACPI cmd ef/10:03:00:00:00:a0 (SET FEATURES) filtered 
out
[   69.409433] ata1.00: supports DRM functions and may not be fully accessible
[   69.409819] ata1.00: NCQ Send/Recv Log not supported
[   69.411644] ata1.00: ACPI cmd ef/02:00:00:00:00:a0 (SET FEATURES) succeeded
[   69.411649] ata1.00: ACPI cmd f5/00:00:00:00:00:a0 (SECURITY FREEZE LOCK) 
filtered out
[   69.411654] ata1.00: ACPI cmd ef/10:03:00:00:00:a0 (SET FEATURES) filtered 
out
[   69.411842] ata1.00: supports DRM functions and may not be fully accessible
[   69.412242] ata1.00: NCQ Send/Recv Log not supported
[   69.412719] ata1.00: configured for UDMA/133
[   69.412721] ata1.00: retrying FLUSH 0xea Emask 0x4
[   69.412964] ata1: EH complete

But that is more difficult to reproduce, it doesn't seem to happen
regularly.  Infact, I thought that was independent from the audio problem
at first but now it's become clear they're all related.

The 'bad' commits always showed the 'do_IRQ: No irq handler for vector'
lines, sometimes they appear for the first time on the console when I
yanked the power cord while playing music in the quick repro cycle without
running X or anything.

What I've been doing to reproduce this is simply boot to multi-user.target,
login, run `cmus`, and play a song.  Next pull the AC power from the
thinkpad x61s.

Immediately the audio gets messed up, pausing the audio doesn't pause it,
it just loops the last tiny buffer contents.

That's all I've got right now, but it doesn't seem to be limited to the
audio problem as shown by the ata1 reset.  Also I've searched through all
my stored journals for anything resembling that ata1 problem, and there's
not a single occurrence going back ~300 boots across a handful of kernel
versions.

If you'd like to prep a patch for me to test, I'm happy to test, but I'm
not sure how online I'll be for the next 24 hours.  It's some kind of
miracle we're communicating as-is, I'm in the middle of the damn desert but
somehow there's a cell signal.

Thanks,
Vito Caputo

Re: thinkpad x60: sound problems in 4.15-rc1 was Re: thinkpad x60: sound problems in 4.14.0-next-20171114

[vcaputo]

On Wed, Dec 20, 2017 at 12:22:12AM +0100, Pavel Machek wrote:
> Hi!
> 
> > > > > > > > > > > Reappeared, 4.15-rc1.
> > > > > > > > > > > 
> > > > > > > > > > > [   40.473822] PM: suspend exit
> > > > > > > > > > > [   40.526027] sdhci-pci :15:00.2: Will use DMA mode 
> > > > > > > > > > > even though
> > > > > > > > > > > HW doesn't fully claim to support it.
> > > > > > > > > > > [   40.569765] e1000e: eth1 NIC Link is Down
> > > > > > > > > > > [   40.578257] sdhci-pci :15:00.2: Will use DMA mode 
> > > > > > > > > > > even though
> > > > > > > > > > > HW doesn't fully claim to support it.
> > > > > > > > > > > [   40.648476] sdhci-pci :15:00.2: Will use DMA mode 
> > > > > > > > > > > even though
> > > > > > > > > > > HW doesn't fully claim to support it.
> > > > > > > > > > > [   40.737339] sdhci-pci :15:00.2: Will use DMA mode 
> > > > > > > > > > > even though
> > > > > > > > > > > HW doesn't fully claim to support it.
> > > > > > > > > > > [   43.018955] wlan0: authenticate with 00:00:00:00:00:01
> > > > > > > > > > > [   43.019072] wlan0: send auth to 00:00:00:00:00:01 (try 
> > > > > > > > > > > 1/3)
> > > > > > > > > > > [   43.023955] wlan0: authenticated
> > > > > > > > > > > [   43.031721] wlan0: associate with 00:00:00:00:00:01 
> > > > > > > > > > > (try 1/3)
> > > > > > > > > > > [   43.039733] wlan0: RX AssocResp from 00:00:00:00:00:01 
> > > > > > > > > > > (capab=0x401
> > > > > > > > > > > status=0 aid=1)
> > > > > > > > > > > [   43.042712] wlan0: associated
> > > > > > > > > > > [  480.662456] snd_hda_intel :00:1b.0: IRQ timing 
> > > > > > > > > > > workaround is
> > > > > > > > > > > activated for card #0. Suggest a bigger bdl_pos_adj.
> > > > > > > > > > 
> > > > > > > > > > This message is often superfluous, so don't take this too 
> > > > > > > > > > seriously.
> > > > > > > > > > 
> > > > > > > > > > 
> > > > > > > > > > > pavel@amd:~$
> > > > > > > > > > > 
> > > > > > > > > > > Again, mplayer has problems, mpg123 works. This time 
> > > > > > > > > > > mplayer started
> > > > > > > > > > > playing video (w/o sound) after long delay.
> > > > > > > > > > > 
> > > > > > > > > > > Uh. huh. And now problems appeared in mpg123, too, and 
> > > > > > > > > > > then went away
> > > > > > > > > > > in mpg123 _and_ mplayer. Interesting.
> > > > > > > > > > > 
> > > > > > > > > > > I suspect some pulseaudio fun. chromium always has sound 
> > > > > > > > > > > problems,
> > > > > > > > > > > then I restart chromium and everything is ok. But 
> > > > > > > > > > > something changed in
> > > > > > > > > > > -next and 4.15-rc1, because mplayer did not have problems 
> > > > > > > > > > > before.
> > > > > > > > > > 
> > > > > > > > > > Hm, there is no code change at all in sound/*.  If it 
> > > > > > > > > > happens only in
> > > > > > > > > > linux-next, it must be something else...
> > > > > > > > > 
> > > > > > > > > It happened first in -next, now it is in 4.15-rc1.
> > > > > > > > 
> > > > > > > > So you meant a possible regression between 4.14 and 4.15-rc1?
> > > > > > > 
> > > > > > > Yes.
> > > > > > 
> > > > > > Hm, as far as I see, the only significant difference is the commit
> > > > > > 20e3f985bb875fea4f86b04eba4b6cc29bfd6b71
> > > > > > ALSA: pcm: update tstamp only if audio_tstamp changed
> > > > > > 
> > > > > > Another change d6c0615f510bc1ee26cfb2b9a3343ac99b9c46fb
> > > > > > ALSA: hda - Fix yet remaining issue with vmaster 0dB
> > > > > > initialization
> > > > > > is basically for fixing a previous wrong fix, and it should 
> > > > > > influence
> > > > > > on all use cases, not only for a specific application.
> > > > > 
> > > > > Happened again, this time on -rc3. It is more than "audio is silent"
> > > > > -- apps behave strangely. Let me test with
> > > > > 20e3f985bb875fea4f86b04eba4b6cc29bfd6b71 reverted.
> > > > > 
> > > > > Hmm. This is 4th regression this release cycle :-(.
> > > > 
> > > > 
> > > > Today I jumped to 4.15-rc4 from 4.14-rc6, and have noticed some oddities
> > > > with audio in youtube under firefox which I never experienced before.
> > > > 
> > > > If I pause the playback, the audio seems to infinitely loop on whatever
> > > > is in the dma buffer.  Resuming playback works but now the expected
> > > > audio has repeated pops and clicks mixed in with it.
> > > > 
> > > > Even closing firefox doesn't seem to stop the looping buffer...
> > > > 
> > > > Machine is an x61s 1.8ghz thinkpad, x86_64, debian stretch, .config 
> > > > attached.
> > > > 
> > > > This for me is a 4.15 blocker, and I presume it's related to Pavel's
> > > > experience as the x60 isn't much different AFAIK.
> > > > 
> > > 
> > > Just reproduced this, it seems to be trivial to repro and doesn't
> > > actually require pausing or anything.  Simply watching a youtube video
> > > causes the audio to get messed up after a short period.
> > > 
> > > I monitored `journalctl --dmesg --follow` while reproducing this and saw
> > > this line appear at 

Re: thinkpad x60: sound problems in 4.15-rc1 was Re: thinkpad x60: sound problems in 4.14.0-next-20171114

[vcaputo]

On Wed, Dec 20, 2017 at 12:22:12AM +0100, Pavel Machek wrote:
> Hi!
> 
> > > > > > > > > > > Reappeared, 4.15-rc1.
> > > > > > > > > > > 
> > > > > > > > > > > [   40.473822] PM: suspend exit
> > > > > > > > > > > [   40.526027] sdhci-pci :15:00.2: Will use DMA mode 
> > > > > > > > > > > even though
> > > > > > > > > > > HW doesn't fully claim to support it.
> > > > > > > > > > > [   40.569765] e1000e: eth1 NIC Link is Down
> > > > > > > > > > > [   40.578257] sdhci-pci :15:00.2: Will use DMA mode 
> > > > > > > > > > > even though
> > > > > > > > > > > HW doesn't fully claim to support it.
> > > > > > > > > > > [   40.648476] sdhci-pci :15:00.2: Will use DMA mode 
> > > > > > > > > > > even though
> > > > > > > > > > > HW doesn't fully claim to support it.
> > > > > > > > > > > [   40.737339] sdhci-pci :15:00.2: Will use DMA mode 
> > > > > > > > > > > even though
> > > > > > > > > > > HW doesn't fully claim to support it.
> > > > > > > > > > > [   43.018955] wlan0: authenticate with 00:00:00:00:00:01
> > > > > > > > > > > [   43.019072] wlan0: send auth to 00:00:00:00:00:01 (try 
> > > > > > > > > > > 1/3)
> > > > > > > > > > > [   43.023955] wlan0: authenticated
> > > > > > > > > > > [   43.031721] wlan0: associate with 00:00:00:00:00:01 
> > > > > > > > > > > (try 1/3)
> > > > > > > > > > > [   43.039733] wlan0: RX AssocResp from 00:00:00:00:00:01 
> > > > > > > > > > > (capab=0x401
> > > > > > > > > > > status=0 aid=1)
> > > > > > > > > > > [   43.042712] wlan0: associated
> > > > > > > > > > > [  480.662456] snd_hda_intel :00:1b.0: IRQ timing 
> > > > > > > > > > > workaround is
> > > > > > > > > > > activated for card #0. Suggest a bigger bdl_pos_adj.
> > > > > > > > > > 
> > > > > > > > > > This message is often superfluous, so don't take this too 
> > > > > > > > > > seriously.
> > > > > > > > > > 
> > > > > > > > > > 
> > > > > > > > > > > pavel@amd:~$
> > > > > > > > > > > 
> > > > > > > > > > > Again, mplayer has problems, mpg123 works. This time 
> > > > > > > > > > > mplayer started
> > > > > > > > > > > playing video (w/o sound) after long delay.
> > > > > > > > > > > 
> > > > > > > > > > > Uh. huh. And now problems appeared in mpg123, too, and 
> > > > > > > > > > > then went away
> > > > > > > > > > > in mpg123 _and_ mplayer. Interesting.
> > > > > > > > > > > 
> > > > > > > > > > > I suspect some pulseaudio fun. chromium always has sound 
> > > > > > > > > > > problems,
> > > > > > > > > > > then I restart chromium and everything is ok. But 
> > > > > > > > > > > something changed in
> > > > > > > > > > > -next and 4.15-rc1, because mplayer did not have problems 
> > > > > > > > > > > before.
> > > > > > > > > > 
> > > > > > > > > > Hm, there is no code change at all in sound/*.  If it 
> > > > > > > > > > happens only in
> > > > > > > > > > linux-next, it must be something else...
> > > > > > > > > 
> > > > > > > > > It happened first in -next, now it is in 4.15-rc1.
> > > > > > > > 
> > > > > > > > So you meant a possible regression between 4.14 and 4.15-rc1?
> > > > > > > 
> > > > > > > Yes.
> > > > > > 
> > > > > > Hm, as far as I see, the only significant difference is the commit
> > > > > > 20e3f985bb875fea4f86b04eba4b6cc29bfd6b71
> > > > > > ALSA: pcm: update tstamp only if audio_tstamp changed
> > > > > > 
> > > > > > Another change d6c0615f510bc1ee26cfb2b9a3343ac99b9c46fb
> > > > > > ALSA: hda - Fix yet remaining issue with vmaster 0dB
> > > > > > initialization
> > > > > > is basically for fixing a previous wrong fix, and it should 
> > > > > > influence
> > > > > > on all use cases, not only for a specific application.
> > > > > 
> > > > > Happened again, this time on -rc3. It is more than "audio is silent"
> > > > > -- apps behave strangely. Let me test with
> > > > > 20e3f985bb875fea4f86b04eba4b6cc29bfd6b71 reverted.
> > > > > 
> > > > > Hmm. This is 4th regression this release cycle :-(.
> > > > 
> > > > 
> > > > Today I jumped to 4.15-rc4 from 4.14-rc6, and have noticed some oddities
> > > > with audio in youtube under firefox which I never experienced before.
> > > > 
> > > > If I pause the playback, the audio seems to infinitely loop on whatever
> > > > is in the dma buffer.  Resuming playback works but now the expected
> > > > audio has repeated pops and clicks mixed in with it.
> > > > 
> > > > Even closing firefox doesn't seem to stop the looping buffer...
> > > > 
> > > > Machine is an x61s 1.8ghz thinkpad, x86_64, debian stretch, .config 
> > > > attached.
> > > > 
> > > > This for me is a 4.15 blocker, and I presume it's related to Pavel's
> > > > experience as the x60 isn't much different AFAIK.
> > > > 
> > > 
> > > Just reproduced this, it seems to be trivial to repro and doesn't
> > > actually require pausing or anything.  Simply watching a youtube video
> > > causes the audio to get messed up after a short period.
> > > 
> > > I monitored `journalctl --dmesg --follow` while reproducing this and saw
> > > this line appear at 

Re: thinkpad x60: sound problems in 4.15-rc1 was Re: thinkpad x60: sound problems in 4.14.0-next-20171114

[vcaputo]

On Mon, Dec 18, 2017 at 08:54:15PM -0800, vcap...@pengaru.com wrote:
> On Mon, Dec 18, 2017 at 06:06:58PM -0800, vcap...@pengaru.com wrote:
> > On Thu, Dec 14, 2017 at 10:57:30AM +0100, Pavel Machek wrote:
> > > On Tue 2017-11-28 08:00:20, Takashi Iwai wrote:
> > > > On Mon, 27 Nov 2017 19:44:12 +0100,
> > > > Pavel Machek wrote:
> > > > > 
> > > > > On Mon 2017-11-27 19:35:32, Takashi Iwai wrote:
> > > > > > On Mon, 27 Nov 2017 19:31:51 +0100,
> > > > > > Pavel Machek wrote:
> > > > > > > 
> > > > > > > On Mon 2017-11-27 17:33:28, Takashi Iwai wrote:
> > > > > > > > On Mon, 27 Nov 2017 17:30:11 +0100,
> > > > > > > > Pavel Machek wrote:
> > > > > > > > > 
> > > > > > > > > On Wed 2017-11-15 12:11:20, Pavel Machek wrote:
> > > > > > > > > > On Wed 2017-11-15 11:43:34, Takashi Iwai wrote:
> > > > > > > > > > > On Wed, 15 Nov 2017 11:05:33 +0100,
> > > > > > > > > > > Pavel Machek wrote:
> > > > > > > > > > > > 
> > > > > > > > > > > > Hi!
> > > > > > > > > > > > 
> > > > > > > > > > > > There are some sound problems in 4.14.0-next-20171114:
> > > > > > > > > > > > 
> > > > > > > > > > > > mplayer shows pictures from video, but does not play.
> > > > > > > > > > > > 
> > > > > > > > > > > > vlc plays video, but w/o sound
> > > > > > > > > > > > 
> > > > > > > > > > > > mpg123 works.
> > > > > > > > > > > > 
> > > > > > > > > > > > Hw is thinkpad X60. Any ideas?
> > > > > > > > > > > 
> > > > > > > > > > > Nothing comes to my mind for now, sorry.
> > > > > > > > > > > 
> > > > > > > > > > > Is it a regression, right?  There are only few changes in 
> > > > > > > > > > > both
> > > > > > > > > > > HD-audio and ALSA core sides, and they should be fairly 
> > > > > > > > > > > harmless.
> > > > > > > > > > 
> > > > > > > > > > Regression: yes. Reproducible: not sure. It went away after 
> > > > > > > > > > a reboot
> > > > > > > > > > :-(.
> > > > > > > > > 
> > > > > > > > > Reappeared, 4.15-rc1.
> > > > > > > > > 
> > > > > > > > > [   40.473822] PM: suspend exit
> > > > > > > > > [   40.526027] sdhci-pci :15:00.2: Will use DMA mode even 
> > > > > > > > > though
> > > > > > > > > HW doesn't fully claim to support it.
> > > > > > > > > [   40.569765] e1000e: eth1 NIC Link is Down
> > > > > > > > > [   40.578257] sdhci-pci :15:00.2: Will use DMA mode even 
> > > > > > > > > though
> > > > > > > > > HW doesn't fully claim to support it.
> > > > > > > > > [   40.648476] sdhci-pci :15:00.2: Will use DMA mode even 
> > > > > > > > > though
> > > > > > > > > HW doesn't fully claim to support it.
> > > > > > > > > [   40.737339] sdhci-pci :15:00.2: Will use DMA mode even 
> > > > > > > > > though
> > > > > > > > > HW doesn't fully claim to support it.
> > > > > > > > > [   43.018955] wlan0: authenticate with 00:00:00:00:00:01
> > > > > > > > > [   43.019072] wlan0: send auth to 00:00:00:00:00:01 (try 1/3)
> > > > > > > > > [   43.023955] wlan0: authenticated
> > > > > > > > > [   43.031721] wlan0: associate with 00:00:00:00:00:01 (try 
> > > > > > > > > 1/3)
> > > > > > > > > [   43.039733] wlan0: RX AssocResp from 00:00:00:00:00:01 
> > > > > > > > > (capab=0x401
> > > > > > > > > status=0 aid=1)
> > > > > > > > > [   43.042712] wlan0: associated
> > > > > > > > > [  480.662456] snd_hda_intel :00:1b.0: IRQ timing 
> > > > > > > > > workaround is
> > > > > > > > > activated for card #0. Suggest a bigger bdl_pos_adj.
> > > > > > > > 
> > > > > > > > This message is often superfluous, so don't take this too 
> > > > > > > > seriously.
> > > > > > > > 
> > > > > > > > 
> > > > > > > > > pavel@amd:~$
> > > > > > > > > 
> > > > > > > > > Again, mplayer has problems, mpg123 works. This time mplayer 
> > > > > > > > > started
> > > > > > > > > playing video (w/o sound) after long delay.
> > > > > > > > > 
> > > > > > > > > Uh. huh. And now problems appeared in mpg123, too, and then 
> > > > > > > > > went away
> > > > > > > > > in mpg123 _and_ mplayer. Interesting.
> > > > > > > > > 
> > > > > > > > > I suspect some pulseaudio fun. chromium always has sound 
> > > > > > > > > problems,
> > > > > > > > > then I restart chromium and everything is ok. But something 
> > > > > > > > > changed in
> > > > > > > > > -next and 4.15-rc1, because mplayer did not have problems 
> > > > > > > > > before.
> > > > > > > > 
> > > > > > > > Hm, there is no code change at all in sound/*.  If it happens 
> > > > > > > > only in
> > > > > > > > linux-next, it must be something else...
> > > > > > > 
> > > > > > > It happened first in -next, now it is in 4.15-rc1.
> > > > > > 
> > > > > > So you meant a possible regression between 4.14 and 4.15-rc1?
> > > > > 
> > > > > Yes.
> > > > 
> > > > Hm, as far as I see, the only significant difference is the commit
> > > > 20e3f985bb875fea4f86b04eba4b6cc29bfd6b71
> > > > ALSA: pcm: update tstamp only if audio_tstamp changed
> > > > 
> > > > Another change d6c0615f510bc1ee26cfb2b9a3343ac99b9c46fb
> > > > ALSA: hda - Fix yet remaining issue with vmaster 0dB
> > > > 

Re: thinkpad x60: sound problems in 4.15-rc1 was Re: thinkpad x60: sound problems in 4.14.0-next-20171114

[vcaputo]

On Mon, Dec 18, 2017 at 08:54:15PM -0800, vcap...@pengaru.com wrote:
> On Mon, Dec 18, 2017 at 06:06:58PM -0800, vcap...@pengaru.com wrote:
> > On Thu, Dec 14, 2017 at 10:57:30AM +0100, Pavel Machek wrote:
> > > On Tue 2017-11-28 08:00:20, Takashi Iwai wrote:
> > > > On Mon, 27 Nov 2017 19:44:12 +0100,
> > > > Pavel Machek wrote:
> > > > > 
> > > > > On Mon 2017-11-27 19:35:32, Takashi Iwai wrote:
> > > > > > On Mon, 27 Nov 2017 19:31:51 +0100,
> > > > > > Pavel Machek wrote:
> > > > > > > 
> > > > > > > On Mon 2017-11-27 17:33:28, Takashi Iwai wrote:
> > > > > > > > On Mon, 27 Nov 2017 17:30:11 +0100,
> > > > > > > > Pavel Machek wrote:
> > > > > > > > > 
> > > > > > > > > On Wed 2017-11-15 12:11:20, Pavel Machek wrote:
> > > > > > > > > > On Wed 2017-11-15 11:43:34, Takashi Iwai wrote:
> > > > > > > > > > > On Wed, 15 Nov 2017 11:05:33 +0100,
> > > > > > > > > > > Pavel Machek wrote:
> > > > > > > > > > > > 
> > > > > > > > > > > > Hi!
> > > > > > > > > > > > 
> > > > > > > > > > > > There are some sound problems in 4.14.0-next-20171114:
> > > > > > > > > > > > 
> > > > > > > > > > > > mplayer shows pictures from video, but does not play.
> > > > > > > > > > > > 
> > > > > > > > > > > > vlc plays video, but w/o sound
> > > > > > > > > > > > 
> > > > > > > > > > > > mpg123 works.
> > > > > > > > > > > > 
> > > > > > > > > > > > Hw is thinkpad X60. Any ideas?
> > > > > > > > > > > 
> > > > > > > > > > > Nothing comes to my mind for now, sorry.
> > > > > > > > > > > 
> > > > > > > > > > > Is it a regression, right?  There are only few changes in 
> > > > > > > > > > > both
> > > > > > > > > > > HD-audio and ALSA core sides, and they should be fairly 
> > > > > > > > > > > harmless.
> > > > > > > > > > 
> > > > > > > > > > Regression: yes. Reproducible: not sure. It went away after 
> > > > > > > > > > a reboot
> > > > > > > > > > :-(.
> > > > > > > > > 
> > > > > > > > > Reappeared, 4.15-rc1.
> > > > > > > > > 
> > > > > > > > > [   40.473822] PM: suspend exit
> > > > > > > > > [   40.526027] sdhci-pci :15:00.2: Will use DMA mode even 
> > > > > > > > > though
> > > > > > > > > HW doesn't fully claim to support it.
> > > > > > > > > [   40.569765] e1000e: eth1 NIC Link is Down
> > > > > > > > > [   40.578257] sdhci-pci :15:00.2: Will use DMA mode even 
> > > > > > > > > though
> > > > > > > > > HW doesn't fully claim to support it.
> > > > > > > > > [   40.648476] sdhci-pci :15:00.2: Will use DMA mode even 
> > > > > > > > > though
> > > > > > > > > HW doesn't fully claim to support it.
> > > > > > > > > [   40.737339] sdhci-pci :15:00.2: Will use DMA mode even 
> > > > > > > > > though
> > > > > > > > > HW doesn't fully claim to support it.
> > > > > > > > > [   43.018955] wlan0: authenticate with 00:00:00:00:00:01
> > > > > > > > > [   43.019072] wlan0: send auth to 00:00:00:00:00:01 (try 1/3)
> > > > > > > > > [   43.023955] wlan0: authenticated
> > > > > > > > > [   43.031721] wlan0: associate with 00:00:00:00:00:01 (try 
> > > > > > > > > 1/3)
> > > > > > > > > [   43.039733] wlan0: RX AssocResp from 00:00:00:00:00:01 
> > > > > > > > > (capab=0x401
> > > > > > > > > status=0 aid=1)
> > > > > > > > > [   43.042712] wlan0: associated
> > > > > > > > > [  480.662456] snd_hda_intel :00:1b.0: IRQ timing 
> > > > > > > > > workaround is
> > > > > > > > > activated for card #0. Suggest a bigger bdl_pos_adj.
> > > > > > > > 
> > > > > > > > This message is often superfluous, so don't take this too 
> > > > > > > > seriously.
> > > > > > > > 
> > > > > > > > 
> > > > > > > > > pavel@amd:~$
> > > > > > > > > 
> > > > > > > > > Again, mplayer has problems, mpg123 works. This time mplayer 
> > > > > > > > > started
> > > > > > > > > playing video (w/o sound) after long delay.
> > > > > > > > > 
> > > > > > > > > Uh. huh. And now problems appeared in mpg123, too, and then 
> > > > > > > > > went away
> > > > > > > > > in mpg123 _and_ mplayer. Interesting.
> > > > > > > > > 
> > > > > > > > > I suspect some pulseaudio fun. chromium always has sound 
> > > > > > > > > problems,
> > > > > > > > > then I restart chromium and everything is ok. But something 
> > > > > > > > > changed in
> > > > > > > > > -next and 4.15-rc1, because mplayer did not have problems 
> > > > > > > > > before.
> > > > > > > > 
> > > > > > > > Hm, there is no code change at all in sound/*.  If it happens 
> > > > > > > > only in
> > > > > > > > linux-next, it must be something else...
> > > > > > > 
> > > > > > > It happened first in -next, now it is in 4.15-rc1.
> > > > > > 
> > > > > > So you meant a possible regression between 4.14 and 4.15-rc1?
> > > > > 
> > > > > Yes.
> > > > 
> > > > Hm, as far as I see, the only significant difference is the commit
> > > > 20e3f985bb875fea4f86b04eba4b6cc29bfd6b71
> > > > ALSA: pcm: update tstamp only if audio_tstamp changed
> > > > 
> > > > Another change d6c0615f510bc1ee26cfb2b9a3343ac99b9c46fb
> > > > ALSA: hda - Fix yet remaining issue with vmaster 0dB
> > > > 

Re: thinkpad x60: sound problems in 4.15-rc1 was Re: thinkpad x60: sound problems in 4.14.0-next-20171114

[vcaputo]

On Mon, Dec 18, 2017 at 06:06:58PM -0800, vcap...@pengaru.com wrote:
> On Thu, Dec 14, 2017 at 10:57:30AM +0100, Pavel Machek wrote:
> > On Tue 2017-11-28 08:00:20, Takashi Iwai wrote:
> > > On Mon, 27 Nov 2017 19:44:12 +0100,
> > > Pavel Machek wrote:
> > > > 
> > > > On Mon 2017-11-27 19:35:32, Takashi Iwai wrote:
> > > > > On Mon, 27 Nov 2017 19:31:51 +0100,
> > > > > Pavel Machek wrote:
> > > > > > 
> > > > > > On Mon 2017-11-27 17:33:28, Takashi Iwai wrote:
> > > > > > > On Mon, 27 Nov 2017 17:30:11 +0100,
> > > > > > > Pavel Machek wrote:
> > > > > > > > 
> > > > > > > > On Wed 2017-11-15 12:11:20, Pavel Machek wrote:
> > > > > > > > > On Wed 2017-11-15 11:43:34, Takashi Iwai wrote:
> > > > > > > > > > On Wed, 15 Nov 2017 11:05:33 +0100,
> > > > > > > > > > Pavel Machek wrote:
> > > > > > > > > > > 
> > > > > > > > > > > Hi!
> > > > > > > > > > > 
> > > > > > > > > > > There are some sound problems in 4.14.0-next-20171114:
> > > > > > > > > > > 
> > > > > > > > > > > mplayer shows pictures from video, but does not play.
> > > > > > > > > > > 
> > > > > > > > > > > vlc plays video, but w/o sound
> > > > > > > > > > > 
> > > > > > > > > > > mpg123 works.
> > > > > > > > > > > 
> > > > > > > > > > > Hw is thinkpad X60. Any ideas?
> > > > > > > > > > 
> > > > > > > > > > Nothing comes to my mind for now, sorry.
> > > > > > > > > > 
> > > > > > > > > > Is it a regression, right?  There are only few changes in 
> > > > > > > > > > both
> > > > > > > > > > HD-audio and ALSA core sides, and they should be fairly 
> > > > > > > > > > harmless.
> > > > > > > > > 
> > > > > > > > > Regression: yes. Reproducible: not sure. It went away after a 
> > > > > > > > > reboot
> > > > > > > > > :-(.
> > > > > > > > 
> > > > > > > > Reappeared, 4.15-rc1.
> > > > > > > > 
> > > > > > > > [   40.473822] PM: suspend exit
> > > > > > > > [   40.526027] sdhci-pci :15:00.2: Will use DMA mode even 
> > > > > > > > though
> > > > > > > > HW doesn't fully claim to support it.
> > > > > > > > [   40.569765] e1000e: eth1 NIC Link is Down
> > > > > > > > [   40.578257] sdhci-pci :15:00.2: Will use DMA mode even 
> > > > > > > > though
> > > > > > > > HW doesn't fully claim to support it.
> > > > > > > > [   40.648476] sdhci-pci :15:00.2: Will use DMA mode even 
> > > > > > > > though
> > > > > > > > HW doesn't fully claim to support it.
> > > > > > > > [   40.737339] sdhci-pci :15:00.2: Will use DMA mode even 
> > > > > > > > though
> > > > > > > > HW doesn't fully claim to support it.
> > > > > > > > [   43.018955] wlan0: authenticate with 00:00:00:00:00:01
> > > > > > > > [   43.019072] wlan0: send auth to 00:00:00:00:00:01 (try 1/3)
> > > > > > > > [   43.023955] wlan0: authenticated
> > > > > > > > [   43.031721] wlan0: associate with 00:00:00:00:00:01 (try 1/3)
> > > > > > > > [   43.039733] wlan0: RX AssocResp from 00:00:00:00:00:01 
> > > > > > > > (capab=0x401
> > > > > > > > status=0 aid=1)
> > > > > > > > [   43.042712] wlan0: associated
> > > > > > > > [  480.662456] snd_hda_intel :00:1b.0: IRQ timing 
> > > > > > > > workaround is
> > > > > > > > activated for card #0. Suggest a bigger bdl_pos_adj.
> > > > > > > 
> > > > > > > This message is often superfluous, so don't take this too 
> > > > > > > seriously.
> > > > > > > 
> > > > > > > 
> > > > > > > > pavel@amd:~$
> > > > > > > > 
> > > > > > > > Again, mplayer has problems, mpg123 works. This time mplayer 
> > > > > > > > started
> > > > > > > > playing video (w/o sound) after long delay.
> > > > > > > > 
> > > > > > > > Uh. huh. And now problems appeared in mpg123, too, and then 
> > > > > > > > went away
> > > > > > > > in mpg123 _and_ mplayer. Interesting.
> > > > > > > > 
> > > > > > > > I suspect some pulseaudio fun. chromium always has sound 
> > > > > > > > problems,
> > > > > > > > then I restart chromium and everything is ok. But something 
> > > > > > > > changed in
> > > > > > > > -next and 4.15-rc1, because mplayer did not have problems 
> > > > > > > > before.
> > > > > > > 
> > > > > > > Hm, there is no code change at all in sound/*.  If it happens 
> > > > > > > only in
> > > > > > > linux-next, it must be something else...
> > > > > > 
> > > > > > It happened first in -next, now it is in 4.15-rc1.
> > > > > 
> > > > > So you meant a possible regression between 4.14 and 4.15-rc1?
> > > > 
> > > > Yes.
> > > 
> > > Hm, as far as I see, the only significant difference is the commit
> > > 20e3f985bb875fea4f86b04eba4b6cc29bfd6b71
> > > ALSA: pcm: update tstamp only if audio_tstamp changed
> > > 
> > > Another change d6c0615f510bc1ee26cfb2b9a3343ac99b9c46fb
> > > ALSA: hda - Fix yet remaining issue with vmaster 0dB
> > > initialization
> > > is basically for fixing a previous wrong fix, and it should influence
> > > on all use cases, not only for a specific application.
> > 
> > Happened again, this time on -rc3. It is more than "audio is silent"
> > -- apps behave strangely. Let me test with
> > 

Re: thinkpad x60: sound problems in 4.15-rc1 was Re: thinkpad x60: sound problems in 4.14.0-next-20171114

[vcaputo]

On Mon, Dec 18, 2017 at 06:06:58PM -0800, vcap...@pengaru.com wrote:
> On Thu, Dec 14, 2017 at 10:57:30AM +0100, Pavel Machek wrote:
> > On Tue 2017-11-28 08:00:20, Takashi Iwai wrote:
> > > On Mon, 27 Nov 2017 19:44:12 +0100,
> > > Pavel Machek wrote:
> > > > 
> > > > On Mon 2017-11-27 19:35:32, Takashi Iwai wrote:
> > > > > On Mon, 27 Nov 2017 19:31:51 +0100,
> > > > > Pavel Machek wrote:
> > > > > > 
> > > > > > On Mon 2017-11-27 17:33:28, Takashi Iwai wrote:
> > > > > > > On Mon, 27 Nov 2017 17:30:11 +0100,
> > > > > > > Pavel Machek wrote:
> > > > > > > > 
> > > > > > > > On Wed 2017-11-15 12:11:20, Pavel Machek wrote:
> > > > > > > > > On Wed 2017-11-15 11:43:34, Takashi Iwai wrote:
> > > > > > > > > > On Wed, 15 Nov 2017 11:05:33 +0100,
> > > > > > > > > > Pavel Machek wrote:
> > > > > > > > > > > 
> > > > > > > > > > > Hi!
> > > > > > > > > > > 
> > > > > > > > > > > There are some sound problems in 4.14.0-next-20171114:
> > > > > > > > > > > 
> > > > > > > > > > > mplayer shows pictures from video, but does not play.
> > > > > > > > > > > 
> > > > > > > > > > > vlc plays video, but w/o sound
> > > > > > > > > > > 
> > > > > > > > > > > mpg123 works.
> > > > > > > > > > > 
> > > > > > > > > > > Hw is thinkpad X60. Any ideas?
> > > > > > > > > > 
> > > > > > > > > > Nothing comes to my mind for now, sorry.
> > > > > > > > > > 
> > > > > > > > > > Is it a regression, right?  There are only few changes in 
> > > > > > > > > > both
> > > > > > > > > > HD-audio and ALSA core sides, and they should be fairly 
> > > > > > > > > > harmless.
> > > > > > > > > 
> > > > > > > > > Regression: yes. Reproducible: not sure. It went away after a 
> > > > > > > > > reboot
> > > > > > > > > :-(.
> > > > > > > > 
> > > > > > > > Reappeared, 4.15-rc1.
> > > > > > > > 
> > > > > > > > [   40.473822] PM: suspend exit
> > > > > > > > [   40.526027] sdhci-pci :15:00.2: Will use DMA mode even 
> > > > > > > > though
> > > > > > > > HW doesn't fully claim to support it.
> > > > > > > > [   40.569765] e1000e: eth1 NIC Link is Down
> > > > > > > > [   40.578257] sdhci-pci :15:00.2: Will use DMA mode even 
> > > > > > > > though
> > > > > > > > HW doesn't fully claim to support it.
> > > > > > > > [   40.648476] sdhci-pci :15:00.2: Will use DMA mode even 
> > > > > > > > though
> > > > > > > > HW doesn't fully claim to support it.
> > > > > > > > [   40.737339] sdhci-pci :15:00.2: Will use DMA mode even 
> > > > > > > > though
> > > > > > > > HW doesn't fully claim to support it.
> > > > > > > > [   43.018955] wlan0: authenticate with 00:00:00:00:00:01
> > > > > > > > [   43.019072] wlan0: send auth to 00:00:00:00:00:01 (try 1/3)
> > > > > > > > [   43.023955] wlan0: authenticated
> > > > > > > > [   43.031721] wlan0: associate with 00:00:00:00:00:01 (try 1/3)
> > > > > > > > [   43.039733] wlan0: RX AssocResp from 00:00:00:00:00:01 
> > > > > > > > (capab=0x401
> > > > > > > > status=0 aid=1)
> > > > > > > > [   43.042712] wlan0: associated
> > > > > > > > [  480.662456] snd_hda_intel :00:1b.0: IRQ timing 
> > > > > > > > workaround is
> > > > > > > > activated for card #0. Suggest a bigger bdl_pos_adj.
> > > > > > > 
> > > > > > > This message is often superfluous, so don't take this too 
> > > > > > > seriously.
> > > > > > > 
> > > > > > > 
> > > > > > > > pavel@amd:~$
> > > > > > > > 
> > > > > > > > Again, mplayer has problems, mpg123 works. This time mplayer 
> > > > > > > > started
> > > > > > > > playing video (w/o sound) after long delay.
> > > > > > > > 
> > > > > > > > Uh. huh. And now problems appeared in mpg123, too, and then 
> > > > > > > > went away
> > > > > > > > in mpg123 _and_ mplayer. Interesting.
> > > > > > > > 
> > > > > > > > I suspect some pulseaudio fun. chromium always has sound 
> > > > > > > > problems,
> > > > > > > > then I restart chromium and everything is ok. But something 
> > > > > > > > changed in
> > > > > > > > -next and 4.15-rc1, because mplayer did not have problems 
> > > > > > > > before.
> > > > > > > 
> > > > > > > Hm, there is no code change at all in sound/*.  If it happens 
> > > > > > > only in
> > > > > > > linux-next, it must be something else...
> > > > > > 
> > > > > > It happened first in -next, now it is in 4.15-rc1.
> > > > > 
> > > > > So you meant a possible regression between 4.14 and 4.15-rc1?
> > > > 
> > > > Yes.
> > > 
> > > Hm, as far as I see, the only significant difference is the commit
> > > 20e3f985bb875fea4f86b04eba4b6cc29bfd6b71
> > > ALSA: pcm: update tstamp only if audio_tstamp changed
> > > 
> > > Another change d6c0615f510bc1ee26cfb2b9a3343ac99b9c46fb
> > > ALSA: hda - Fix yet remaining issue with vmaster 0dB
> > > initialization
> > > is basically for fixing a previous wrong fix, and it should influence
> > > on all use cases, not only for a specific application.
> > 
> > Happened again, this time on -rc3. It is more than "audio is silent"
> > -- apps behave strangely. Let me test with
> > 

[BUG] 4.15-rc4 'do_IRQ: 0.33 No irq handler for vector'

[vcaputo]

Hello everyone,

While investigating `journalctl -k` for anything associated with broken
audio, this was found:

  Dec 18 16:09:28 iridesce kernel: do_IRQ: 0.33 No irq handler for vector
  Dec 18 16:10:21 iridesce kernel: ata1.00: exception Emask 0x0 SAct 0xfe0 SErr 
0x0 action 0x6 frozen
  Dec 18 16:10:21 iridesce kernel: ata1.00: failed command: WRITE FPDMA QUEUED
  Dec 18 16:10:21 iridesce kernel: ata1.00: cmd 
61/08:28:30:92:79/00:00:1c:00:00/40 tag 5 ncq dma 4096 out
res 
40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
  Dec 18 16:10:21 iridesce kernel: ata1.00: status: { DRDY }
  Dec 18 16:10:21 iridesce kernel: ata1.00: failed command: WRITE FPDMA QUEUED
  Dec 18 16:10:21 iridesce kernel: ata1.00: cmd 
61/08:30:10:20:68/00:00:00:00:00/40 tag 6 ncq dma 4096 out
res 
40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
  Dec 18 16:10:21 iridesce kernel: ata1.00: status: { DRDY }
  Dec 18 16:10:21 iridesce kernel: ata1.00: failed command: WRITE FPDMA QUEUED
  Dec 18 16:10:21 iridesce kernel: ata1.00: cmd 
61/08:38:30:5d:ec/00:00:1c:00:00/40 tag 7 ncq dma 4096 out
res 
40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
  Dec 18 16:10:21 iridesce kernel: ata1.00: status: { DRDY }
  Dec 18 16:10:21 iridesce kernel: ata1.00: failed command: WRITE FPDMA QUEUED
  Dec 18 16:10:21 iridesce kernel: ata1.00: cmd 
61/48:40:c0:75:64/00:00:00:00:00/40 tag 8 ncq dma 36864 out
res 
40/00:fe:00:00:00/00:00:00:00:00/40 Emask 0x4 (timeout)
  Dec 18 16:10:21 iridesce kernel: ata1.00: status: { DRDY }
  Dec 18 16:10:21 iridesce kernel: ata1.00: failed command: WRITE FPDMA QUEUED
  Dec 18 16:10:21 iridesce kernel: ata1.00: cmd 
61/08:48:20:20:ec/00:00:1c:00:00/40 tag 9 ncq dma 4096 out
res 
40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
  Dec 18 16:10:21 iridesce kernel: ata1.00: status: { DRDY }
  Dec 18 16:10:21 iridesce kernel: ata1.00: failed command: WRITE FPDMA QUEUED
  Dec 18 16:10:21 iridesce kernel: ata1.00: cmd 
61/08:50:80:20:78/00:00:1c:00:00/40 tag 10 ncq dma 4096 out
res 
40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
  Dec 18 16:10:21 iridesce kernel: ata1.00: status: { DRDY }
  Dec 18 16:10:21 iridesce kernel: ata1.00: failed command: WRITE FPDMA QUEUED
  Dec 18 16:10:21 iridesce kernel: ata1.00: cmd 
61/08:58:90:3e:78/00:00:1c:00:00/40 tag 11 ncq dma 4096 out
res 
40/00:01:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
  Dec 18 16:10:21 iridesce kernel: ata1.00: status: { DRDY }
  Dec 18 16:10:21 iridesce kernel: ata1: hard resetting link
  Dec 18 16:10:21 iridesce kernel: ata1: SATA link up 3.0 Gbps (SStatus 123 
SControl 300)
  Dec 18 16:10:21 iridesce kernel: ata1.00: ACPI cmd ef/02:00:00:00:00:a0 (SET 
FEATURES) succeeded
  Dec 18 16:10:21 iridesce kernel: ata1.00: ACPI cmd f5/00:00:00:00:00:a0 
(SECURITY FREEZE LOCK) filtered out
  Dec 18 16:10:21 iridesce kernel: ata1.00: ACPI cmd ef/10:03:00:00:00:a0 (SET 
FEATURES) filtered out
  Dec 18 16:10:21 iridesce kernel: ata1.00: supports DRM functions and may not 
be fully accessible
  Dec 18 16:10:21 iridesce kernel: ata1.00: NCQ Send/Recv Log not supported
  Dec 18 16:10:21 iridesce kernel: ata1.00: ACPI cmd ef/02:00:00:00:00:a0 (SET 
FEATURES) succeeded
  Dec 18 16:10:21 iridesce kernel: ata1.00: ACPI cmd f5/00:00:00:00:00:a0 
(SECURITY FREEZE LOCK) filtered out
  Dec 18 16:10:21 iridesce kernel: ata1.00: ACPI cmd ef/10:03:00:00:00:a0 (SET 
FEATURES) filtered out
  Dec 18 16:10:21 iridesce kernel: ata1.00: supports DRM functions and may not 
be fully accessible
  Dec 18 16:10:21 iridesce kernel: ata1.00: NCQ Send/Recv Log not supported
  Dec 18 16:10:21 iridesce kernel: ata1.00: configured for UDMA/133
  Dec 18 16:10:21 iridesce kernel: ata1.00: device reported invalid CHS sector 0
  Dec 18 16:10:21 iridesce kernel: ata1.00: device reported invalid CHS sector 0
  Dec 18 16:10:21 iridesce kernel: ata1.00: device reported invalid CHS sector 0
  Dec 18 16:10:21 iridesce kernel: ata1.00: device reported invalid CHS sector 0
  Dec 18 16:10:21 iridesce kernel: ata1.00: device reported invalid CHS sector 0
  Dec 18 16:10:21 iridesce kernel: ata1: EH complete

I have approximately ~300 boots logged back to kernel 4.13.0-rc7, and
there's not a single occurrence of this kind of thing in any of them.

Now 4.15-rc4 is my first 4.15 kernel, so I don't know if this was
introduced in one of the RCs or if it's 4.15-*.

I also don't know if this is associated with the audio problem I
observed.  Nor do I know if this is reproducible.  It hasn't recurred
yet, and I'm now in my second boot of 4.15-rc4.  The problem is, I'm not
likely to spend much time in 4.15-rc4 with audio malfunctioning, as this
is my primary laptop and music tends 

[BUG] 4.15-rc4 'do_IRQ: 0.33 No irq handler for vector'

[vcaputo]

Hello everyone,

While investigating `journalctl -k` for anything associated with broken
audio, this was found:

  Dec 18 16:09:28 iridesce kernel: do_IRQ: 0.33 No irq handler for vector
  Dec 18 16:10:21 iridesce kernel: ata1.00: exception Emask 0x0 SAct 0xfe0 SErr 
0x0 action 0x6 frozen
  Dec 18 16:10:21 iridesce kernel: ata1.00: failed command: WRITE FPDMA QUEUED
  Dec 18 16:10:21 iridesce kernel: ata1.00: cmd 
61/08:28:30:92:79/00:00:1c:00:00/40 tag 5 ncq dma 4096 out
res 
40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
  Dec 18 16:10:21 iridesce kernel: ata1.00: status: { DRDY }
  Dec 18 16:10:21 iridesce kernel: ata1.00: failed command: WRITE FPDMA QUEUED
  Dec 18 16:10:21 iridesce kernel: ata1.00: cmd 
61/08:30:10:20:68/00:00:00:00:00/40 tag 6 ncq dma 4096 out
res 
40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
  Dec 18 16:10:21 iridesce kernel: ata1.00: status: { DRDY }
  Dec 18 16:10:21 iridesce kernel: ata1.00: failed command: WRITE FPDMA QUEUED
  Dec 18 16:10:21 iridesce kernel: ata1.00: cmd 
61/08:38:30:5d:ec/00:00:1c:00:00/40 tag 7 ncq dma 4096 out
res 
40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
  Dec 18 16:10:21 iridesce kernel: ata1.00: status: { DRDY }
  Dec 18 16:10:21 iridesce kernel: ata1.00: failed command: WRITE FPDMA QUEUED
  Dec 18 16:10:21 iridesce kernel: ata1.00: cmd 
61/48:40:c0:75:64/00:00:00:00:00/40 tag 8 ncq dma 36864 out
res 
40/00:fe:00:00:00/00:00:00:00:00/40 Emask 0x4 (timeout)
  Dec 18 16:10:21 iridesce kernel: ata1.00: status: { DRDY }
  Dec 18 16:10:21 iridesce kernel: ata1.00: failed command: WRITE FPDMA QUEUED
  Dec 18 16:10:21 iridesce kernel: ata1.00: cmd 
61/08:48:20:20:ec/00:00:1c:00:00/40 tag 9 ncq dma 4096 out
res 
40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
  Dec 18 16:10:21 iridesce kernel: ata1.00: status: { DRDY }
  Dec 18 16:10:21 iridesce kernel: ata1.00: failed command: WRITE FPDMA QUEUED
  Dec 18 16:10:21 iridesce kernel: ata1.00: cmd 
61/08:50:80:20:78/00:00:1c:00:00/40 tag 10 ncq dma 4096 out
res 
40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
  Dec 18 16:10:21 iridesce kernel: ata1.00: status: { DRDY }
  Dec 18 16:10:21 iridesce kernel: ata1.00: failed command: WRITE FPDMA QUEUED
  Dec 18 16:10:21 iridesce kernel: ata1.00: cmd 
61/08:58:90:3e:78/00:00:1c:00:00/40 tag 11 ncq dma 4096 out
res 
40/00:01:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
  Dec 18 16:10:21 iridesce kernel: ata1.00: status: { DRDY }
  Dec 18 16:10:21 iridesce kernel: ata1: hard resetting link
  Dec 18 16:10:21 iridesce kernel: ata1: SATA link up 3.0 Gbps (SStatus 123 
SControl 300)
  Dec 18 16:10:21 iridesce kernel: ata1.00: ACPI cmd ef/02:00:00:00:00:a0 (SET 
FEATURES) succeeded
  Dec 18 16:10:21 iridesce kernel: ata1.00: ACPI cmd f5/00:00:00:00:00:a0 
(SECURITY FREEZE LOCK) filtered out
  Dec 18 16:10:21 iridesce kernel: ata1.00: ACPI cmd ef/10:03:00:00:00:a0 (SET 
FEATURES) filtered out
  Dec 18 16:10:21 iridesce kernel: ata1.00: supports DRM functions and may not 
be fully accessible
  Dec 18 16:10:21 iridesce kernel: ata1.00: NCQ Send/Recv Log not supported
  Dec 18 16:10:21 iridesce kernel: ata1.00: ACPI cmd ef/02:00:00:00:00:a0 (SET 
FEATURES) succeeded
  Dec 18 16:10:21 iridesce kernel: ata1.00: ACPI cmd f5/00:00:00:00:00:a0 
(SECURITY FREEZE LOCK) filtered out
  Dec 18 16:10:21 iridesce kernel: ata1.00: ACPI cmd ef/10:03:00:00:00:a0 (SET 
FEATURES) filtered out
  Dec 18 16:10:21 iridesce kernel: ata1.00: supports DRM functions and may not 
be fully accessible
  Dec 18 16:10:21 iridesce kernel: ata1.00: NCQ Send/Recv Log not supported
  Dec 18 16:10:21 iridesce kernel: ata1.00: configured for UDMA/133
  Dec 18 16:10:21 iridesce kernel: ata1.00: device reported invalid CHS sector 0
  Dec 18 16:10:21 iridesce kernel: ata1.00: device reported invalid CHS sector 0
  Dec 18 16:10:21 iridesce kernel: ata1.00: device reported invalid CHS sector 0
  Dec 18 16:10:21 iridesce kernel: ata1.00: device reported invalid CHS sector 0
  Dec 18 16:10:21 iridesce kernel: ata1.00: device reported invalid CHS sector 0
  Dec 18 16:10:21 iridesce kernel: ata1: EH complete

I have approximately ~300 boots logged back to kernel 4.13.0-rc7, and
there's not a single occurrence of this kind of thing in any of them.

Now 4.15-rc4 is my first 4.15 kernel, so I don't know if this was
introduced in one of the RCs or if it's 4.15-*.

I also don't know if this is associated with the audio problem I
observed.  Nor do I know if this is reproducible.  It hasn't recurred
yet, and I'm now in my second boot of 4.15-rc4.  The problem is, I'm not
likely to spend much time in 4.15-rc4 with audio malfunctioning, as this
is my primary laptop and music tends 

Re: thinkpad x60: sound problems in 4.15-rc1 was Re: thinkpad x60: sound problems in 4.14.0-next-20171114

[vcaputo]

On Thu, Dec 14, 2017 at 10:57:30AM +0100, Pavel Machek wrote:
> On Tue 2017-11-28 08:00:20, Takashi Iwai wrote:
> > On Mon, 27 Nov 2017 19:44:12 +0100,
> > Pavel Machek wrote:
> > > 
> > > On Mon 2017-11-27 19:35:32, Takashi Iwai wrote:
> > > > On Mon, 27 Nov 2017 19:31:51 +0100,
> > > > Pavel Machek wrote:
> > > > > 
> > > > > On Mon 2017-11-27 17:33:28, Takashi Iwai wrote:
> > > > > > On Mon, 27 Nov 2017 17:30:11 +0100,
> > > > > > Pavel Machek wrote:
> > > > > > > 
> > > > > > > On Wed 2017-11-15 12:11:20, Pavel Machek wrote:
> > > > > > > > On Wed 2017-11-15 11:43:34, Takashi Iwai wrote:
> > > > > > > > > On Wed, 15 Nov 2017 11:05:33 +0100,
> > > > > > > > > Pavel Machek wrote:
> > > > > > > > > > 
> > > > > > > > > > Hi!
> > > > > > > > > > 
> > > > > > > > > > There are some sound problems in 4.14.0-next-20171114:
> > > > > > > > > > 
> > > > > > > > > > mplayer shows pictures from video, but does not play.
> > > > > > > > > > 
> > > > > > > > > > vlc plays video, but w/o sound
> > > > > > > > > > 
> > > > > > > > > > mpg123 works.
> > > > > > > > > > 
> > > > > > > > > > Hw is thinkpad X60. Any ideas?
> > > > > > > > > 
> > > > > > > > > Nothing comes to my mind for now, sorry.
> > > > > > > > > 
> > > > > > > > > Is it a regression, right?  There are only few changes in both
> > > > > > > > > HD-audio and ALSA core sides, and they should be fairly 
> > > > > > > > > harmless.
> > > > > > > > 
> > > > > > > > Regression: yes. Reproducible: not sure. It went away after a 
> > > > > > > > reboot
> > > > > > > > :-(.
> > > > > > > 
> > > > > > > Reappeared, 4.15-rc1.
> > > > > > > 
> > > > > > > [   40.473822] PM: suspend exit
> > > > > > > [   40.526027] sdhci-pci :15:00.2: Will use DMA mode even 
> > > > > > > though
> > > > > > > HW doesn't fully claim to support it.
> > > > > > > [   40.569765] e1000e: eth1 NIC Link is Down
> > > > > > > [   40.578257] sdhci-pci :15:00.2: Will use DMA mode even 
> > > > > > > though
> > > > > > > HW doesn't fully claim to support it.
> > > > > > > [   40.648476] sdhci-pci :15:00.2: Will use DMA mode even 
> > > > > > > though
> > > > > > > HW doesn't fully claim to support it.
> > > > > > > [   40.737339] sdhci-pci :15:00.2: Will use DMA mode even 
> > > > > > > though
> > > > > > > HW doesn't fully claim to support it.
> > > > > > > [   43.018955] wlan0: authenticate with 00:00:00:00:00:01
> > > > > > > [   43.019072] wlan0: send auth to 00:00:00:00:00:01 (try 1/3)
> > > > > > > [   43.023955] wlan0: authenticated
> > > > > > > [   43.031721] wlan0: associate with 00:00:00:00:00:01 (try 1/3)
> > > > > > > [   43.039733] wlan0: RX AssocResp from 00:00:00:00:00:01 
> > > > > > > (capab=0x401
> > > > > > > status=0 aid=1)
> > > > > > > [   43.042712] wlan0: associated
> > > > > > > [  480.662456] snd_hda_intel :00:1b.0: IRQ timing workaround 
> > > > > > > is
> > > > > > > activated for card #0. Suggest a bigger bdl_pos_adj.
> > > > > > 
> > > > > > This message is often superfluous, so don't take this too seriously.
> > > > > > 
> > > > > > 
> > > > > > > pavel@amd:~$
> > > > > > > 
> > > > > > > Again, mplayer has problems, mpg123 works. This time mplayer 
> > > > > > > started
> > > > > > > playing video (w/o sound) after long delay.
> > > > > > > 
> > > > > > > Uh. huh. And now problems appeared in mpg123, too, and then went 
> > > > > > > away
> > > > > > > in mpg123 _and_ mplayer. Interesting.
> > > > > > > 
> > > > > > > I suspect some pulseaudio fun. chromium always has sound problems,
> > > > > > > then I restart chromium and everything is ok. But something 
> > > > > > > changed in
> > > > > > > -next and 4.15-rc1, because mplayer did not have problems before.
> > > > > > 
> > > > > > Hm, there is no code change at all in sound/*.  If it happens only 
> > > > > > in
> > > > > > linux-next, it must be something else...
> > > > > 
> > > > > It happened first in -next, now it is in 4.15-rc1.
> > > > 
> > > > So you meant a possible regression between 4.14 and 4.15-rc1?
> > > 
> > > Yes.
> > 
> > Hm, as far as I see, the only significant difference is the commit
> > 20e3f985bb875fea4f86b04eba4b6cc29bfd6b71
> > ALSA: pcm: update tstamp only if audio_tstamp changed
> > 
> > Another change d6c0615f510bc1ee26cfb2b9a3343ac99b9c46fb
> > ALSA: hda - Fix yet remaining issue with vmaster 0dB
> > initialization
> > is basically for fixing a previous wrong fix, and it should influence
> > on all use cases, not only for a specific application.
> 
> Happened again, this time on -rc3. It is more than "audio is silent"
> -- apps behave strangely. Let me test with
> 20e3f985bb875fea4f86b04eba4b6cc29bfd6b71 reverted.
> 
> Hmm. This is 4th regression this release cycle :-(.


Today I jumped to 4.15-rc4 from 4.14-rc6, and have noticed some oddities
with audio in youtube under firefox which I never experienced before.

If I pause the playback, the audio seems to infinitely loop on whatever
is in the dma buffer.  Resuming playback works but now 

Re: thinkpad x60: sound problems in 4.15-rc1 was Re: thinkpad x60: sound problems in 4.14.0-next-20171114

[vcaputo]

On Thu, Dec 14, 2017 at 10:57:30AM +0100, Pavel Machek wrote:
> On Tue 2017-11-28 08:00:20, Takashi Iwai wrote:
> > On Mon, 27 Nov 2017 19:44:12 +0100,
> > Pavel Machek wrote:
> > > 
> > > On Mon 2017-11-27 19:35:32, Takashi Iwai wrote:
> > > > On Mon, 27 Nov 2017 19:31:51 +0100,
> > > > Pavel Machek wrote:
> > > > > 
> > > > > On Mon 2017-11-27 17:33:28, Takashi Iwai wrote:
> > > > > > On Mon, 27 Nov 2017 17:30:11 +0100,
> > > > > > Pavel Machek wrote:
> > > > > > > 
> > > > > > > On Wed 2017-11-15 12:11:20, Pavel Machek wrote:
> > > > > > > > On Wed 2017-11-15 11:43:34, Takashi Iwai wrote:
> > > > > > > > > On Wed, 15 Nov 2017 11:05:33 +0100,
> > > > > > > > > Pavel Machek wrote:
> > > > > > > > > > 
> > > > > > > > > > Hi!
> > > > > > > > > > 
> > > > > > > > > > There are some sound problems in 4.14.0-next-20171114:
> > > > > > > > > > 
> > > > > > > > > > mplayer shows pictures from video, but does not play.
> > > > > > > > > > 
> > > > > > > > > > vlc plays video, but w/o sound
> > > > > > > > > > 
> > > > > > > > > > mpg123 works.
> > > > > > > > > > 
> > > > > > > > > > Hw is thinkpad X60. Any ideas?
> > > > > > > > > 
> > > > > > > > > Nothing comes to my mind for now, sorry.
> > > > > > > > > 
> > > > > > > > > Is it a regression, right?  There are only few changes in both
> > > > > > > > > HD-audio and ALSA core sides, and they should be fairly 
> > > > > > > > > harmless.
> > > > > > > > 
> > > > > > > > Regression: yes. Reproducible: not sure. It went away after a 
> > > > > > > > reboot
> > > > > > > > :-(.
> > > > > > > 
> > > > > > > Reappeared, 4.15-rc1.
> > > > > > > 
> > > > > > > [   40.473822] PM: suspend exit
> > > > > > > [   40.526027] sdhci-pci :15:00.2: Will use DMA mode even 
> > > > > > > though
> > > > > > > HW doesn't fully claim to support it.
> > > > > > > [   40.569765] e1000e: eth1 NIC Link is Down
> > > > > > > [   40.578257] sdhci-pci :15:00.2: Will use DMA mode even 
> > > > > > > though
> > > > > > > HW doesn't fully claim to support it.
> > > > > > > [   40.648476] sdhci-pci :15:00.2: Will use DMA mode even 
> > > > > > > though
> > > > > > > HW doesn't fully claim to support it.
> > > > > > > [   40.737339] sdhci-pci :15:00.2: Will use DMA mode even 
> > > > > > > though
> > > > > > > HW doesn't fully claim to support it.
> > > > > > > [   43.018955] wlan0: authenticate with 00:00:00:00:00:01
> > > > > > > [   43.019072] wlan0: send auth to 00:00:00:00:00:01 (try 1/3)
> > > > > > > [   43.023955] wlan0: authenticated
> > > > > > > [   43.031721] wlan0: associate with 00:00:00:00:00:01 (try 1/3)
> > > > > > > [   43.039733] wlan0: RX AssocResp from 00:00:00:00:00:01 
> > > > > > > (capab=0x401
> > > > > > > status=0 aid=1)
> > > > > > > [   43.042712] wlan0: associated
> > > > > > > [  480.662456] snd_hda_intel :00:1b.0: IRQ timing workaround 
> > > > > > > is
> > > > > > > activated for card #0. Suggest a bigger bdl_pos_adj.
> > > > > > 
> > > > > > This message is often superfluous, so don't take this too seriously.
> > > > > > 
> > > > > > 
> > > > > > > pavel@amd:~$
> > > > > > > 
> > > > > > > Again, mplayer has problems, mpg123 works. This time mplayer 
> > > > > > > started
> > > > > > > playing video (w/o sound) after long delay.
> > > > > > > 
> > > > > > > Uh. huh. And now problems appeared in mpg123, too, and then went 
> > > > > > > away
> > > > > > > in mpg123 _and_ mplayer. Interesting.
> > > > > > > 
> > > > > > > I suspect some pulseaudio fun. chromium always has sound problems,
> > > > > > > then I restart chromium and everything is ok. But something 
> > > > > > > changed in
> > > > > > > -next and 4.15-rc1, because mplayer did not have problems before.
> > > > > > 
> > > > > > Hm, there is no code change at all in sound/*.  If it happens only 
> > > > > > in
> > > > > > linux-next, it must be something else...
> > > > > 
> > > > > It happened first in -next, now it is in 4.15-rc1.
> > > > 
> > > > So you meant a possible regression between 4.14 and 4.15-rc1?
> > > 
> > > Yes.
> > 
> > Hm, as far as I see, the only significant difference is the commit
> > 20e3f985bb875fea4f86b04eba4b6cc29bfd6b71
> > ALSA: pcm: update tstamp only if audio_tstamp changed
> > 
> > Another change d6c0615f510bc1ee26cfb2b9a3343ac99b9c46fb
> > ALSA: hda - Fix yet remaining issue with vmaster 0dB
> > initialization
> > is basically for fixing a previous wrong fix, and it should influence
> > on all use cases, not only for a specific application.
> 
> Happened again, this time on -rc3. It is more than "audio is silent"
> -- apps behave strangely. Let me test with
> 20e3f985bb875fea4f86b04eba4b6cc29bfd6b71 reverted.
> 
> Hmm. This is 4th regression this release cycle :-(.


Today I jumped to 4.15-rc4 from 4.14-rc6, and have noticed some oddities
with audio in youtube under firefox which I never experienced before.

If I pause the playback, the audio seems to infinitely loop on whatever
is in the dma buffer.  Resuming playback works but now 

Re: PROBLEM: NULL pointer dereference in kernel 4.14.6

[vcaputo]

On Sun, Dec 17, 2017 at 05:49:44PM +, Bronek Kozicki wrote:
> I just upgraded to 4.14.7 and tried to reproduce this error, this time under 
> strace. As you can see this happens when systemctl tries to read a specific 
> entry under /sys/fs . In case this matters, the entry is for a small virtual 
> machine running under qemu/kvm and managed by libvirt.
> 
> open("/sys/fs/cgroup/unified/machine.slice", 
> O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 5
> fstat(5, {st_mode=S_IFDIR|0755, st_size=0, ...}) = 0
> getdents(5, /* 12 entries */, 32768)= 464
> openat(AT_FDCWD, 
> "/sys/fs/cgroup/unified/machine.slice/machine-qemu\\x2d1\\x2dkartuzy\\x2dspice.scope/cgroup.procs",
>  O_RDONLY|O_CLOEXEC) = 8
> fstat(8, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
> read(8,  )  = ?
> +++ killed by SIGKILL +++
> [1]12078 killed strace -- systemctl status
> 
> 

This recently came through lkml, may be related:
https://marc.info/?l=linux-kernel=151320108922415=2

CCd Tejun


> B.
> 
> 
> [ 1889.226051] 
> 
> [ 1889.235286] UBSAN: Undefined behaviour in kernel/cgroup/pids.c:67:9
> [ 1889.241563] member access within null pointer of type 'struct pids_cgroup'
> [ 1889.249920] 
> 
> [ 1889.259698] BUG: unable to handle kernel NULL pointer dereference at 
> 00b0
> [ 1889.267524] IP: pids_free+0x28/0xb0
> [ 1889.272394] PGD 0 P4D 0
> [ 1889.274925] Oops:  [#1] SMP
> [ 1889.278061] Modules linked in: ebtable_filter ebtables ip6table_filter 
> ip6_tables iptable_filter devlink joydev hid_logitech_hidpp mxm_wmi 
> intel_rapl sb_edac x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel 
> kvm crct10dif_pclmul crc32_pclmu
> l crc32c_intel ghash_clmulni_intel pcbc aesni_intel aes_x86_64 crypto_simd 
> glue_helper cryptd ext4 intel_cstate crc16 mbcache jbd2 fscrypto 
> nls_iso8859_1 nls_cp437 evdev input_leds led_class vfat fat intel_rapl_perf 
> mac_hid pcspkr hid_logitech_dj igb ptp mei_me pps_
> core i2c_i801 i2c_algo_bit mei lpc_ich ioatdma tpm_tis tpm_tis_core dca 
> shpchp tpm wmi button sch_fq_codel sg ip_tables x_tables usbhid hid zfs(PO) 
> zunicode(PO) zavl(PO) icp(PO) sd_mod serio_raw atkbd libps2 isci ahci libsas 
> libahci xhci_pci ehci_pci mpt3sas xhci_hc
> d ehci_hcd raid_class libata
> [ 1889.349864]  scsi_transport_sas usbcore scsi_mod usb_common i8042 serio 
> zcommon(PO) znvpair(PO) spl(O) nvme nvme_core bridge stp llc vhost_net tun 
> tap vhost vfio_pci irqbypass vfio_virqfd vfio_iommu_type1 vfio
> [ 1889.368439] CPU: 1 PID: 12084 Comm: systemctl Tainted: PW  O
> 4.14.7-1-ARCH #1
> [ 1889.376525] Hardware name: Supermicro X9DA7/E/X9DA7/E, BIOS 3.0a 07/02/2014
> [ 1889.383474] task: 93149aaec140 task.stack: a88c3836c000
> [ 1889.389387] RIP: 0010:pids_free+0x28/0xb0
> [ 1889.393388] RSP: 0018:a88c3836fcc8 EFLAGS: 00010282
> [ 1889.398605] RAX:  RBX:  RCX: 
> 0006
> [ 1889.405731] RDX:  RSI: 0202 RDI: 
> 0202
> [ 1889.412854] RBP: 931499ab2d58 R08: 079a R09: 
> 
> [ 1889.419979] R10: 001f5954 R11: 0003d040 R12: 
> 56e21a48
> [ 1889.427102] R13: a91de5c0 R14: 93247b0598c0 R15: 
> a91cd0a0
> [ 1889.434227] FS:  7f18eee6b8c0() GS:931ebfa4() 
> knlGS:
> [ 1889.442302] CS:  0010 DS:  ES:  CR0: 80050033
> [ 1889.448041] CR2: 00b0 CR3: 000611019003 CR4: 
> 001626e0
> [ 1889.455164] Call Trace:
> [ 1889.457610]  cgroup_free+0xaa/0x190
> [ 1889.461095]  __put_task_struct+0x68/0x230
> [ 1889.465105]  ? seq_printf+0x4e/0x70
> [ 1889.468591]  css_task_iter_next+0x74/0x90
> [ 1889.472594]  kernfs_seq_next+0x58/0x110
> [ 1889.476424]  seq_read+0x36c/0x620
> [ 1889.479735]  __vfs_read+0x54/0x2e0
> [ 1889.483134]  vfs_read+0x9d/0x200
> [ 1889.486358]  SyS_read+0x52/0xc0
> [ 1889.489494]  do_syscall_64+0x69/0x1e0
> [ 1889.493152]  entry_SYSCALL64_slow_path+0x25/0x25
> [ 1889.497771] RIP: 0033:0x7f18ee784a11
> [ 1889.501341] RSP: 002b:7ffd56942618 EFLAGS: 0246 ORIG_RAX: 
> 
> [ 1889.508897] RAX: ffda RBX: 559a9ae6d260 RCX: 
> 7f18ee784a11
> [ 1889.516022] RDX: 1000 RSI: 559a9ae80f70 RDI: 
> 0008
> [ 1889.523145] RBP: 0d68 R08: 0003 R09: 
> ffb0
> [ 1889.530270] R10: 1000 R11: 0246 R12: 
> 7f18eea4b700
> [ 1889.537395] R13: 7f18eea4c240 R14: 559a9ae6d260 R15: 
> 
> [ 1889.544518] Code: 44 00 00 0f 1f 44 00 00 48 81 ff c8 f7 ff ff 55 53 48 89 
> fb 74 4c 48 8b 9b 38 08 00 00 48 85 db 74 7c 48 8b 5b 50 48 85 db 74 63 <48> 
> 83 bb b0 00 00 00 00 74 2a 48 c7 c5 60 2e 1e a9 48 89 df e8
> [ 1889.563368] RIP: 

Re: PROBLEM: NULL pointer dereference in kernel 4.14.6

[vcaputo]

On Sun, Dec 17, 2017 at 05:49:44PM +, Bronek Kozicki wrote:
> I just upgraded to 4.14.7 and tried to reproduce this error, this time under 
> strace. As you can see this happens when systemctl tries to read a specific 
> entry under /sys/fs . In case this matters, the entry is for a small virtual 
> machine running under qemu/kvm and managed by libvirt.
> 
> open("/sys/fs/cgroup/unified/machine.slice", 
> O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 5
> fstat(5, {st_mode=S_IFDIR|0755, st_size=0, ...}) = 0
> getdents(5, /* 12 entries */, 32768)= 464
> openat(AT_FDCWD, 
> "/sys/fs/cgroup/unified/machine.slice/machine-qemu\\x2d1\\x2dkartuzy\\x2dspice.scope/cgroup.procs",
>  O_RDONLY|O_CLOEXEC) = 8
> fstat(8, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
> read(8,  )  = ?
> +++ killed by SIGKILL +++
> [1]12078 killed strace -- systemctl status
> 
> 

This recently came through lkml, may be related:
https://marc.info/?l=linux-kernel=151320108922415=2

CCd Tejun


> B.
> 
> 
> [ 1889.226051] 
> 
> [ 1889.235286] UBSAN: Undefined behaviour in kernel/cgroup/pids.c:67:9
> [ 1889.241563] member access within null pointer of type 'struct pids_cgroup'
> [ 1889.249920] 
> 
> [ 1889.259698] BUG: unable to handle kernel NULL pointer dereference at 
> 00b0
> [ 1889.267524] IP: pids_free+0x28/0xb0
> [ 1889.272394] PGD 0 P4D 0
> [ 1889.274925] Oops:  [#1] SMP
> [ 1889.278061] Modules linked in: ebtable_filter ebtables ip6table_filter 
> ip6_tables iptable_filter devlink joydev hid_logitech_hidpp mxm_wmi 
> intel_rapl sb_edac x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel 
> kvm crct10dif_pclmul crc32_pclmu
> l crc32c_intel ghash_clmulni_intel pcbc aesni_intel aes_x86_64 crypto_simd 
> glue_helper cryptd ext4 intel_cstate crc16 mbcache jbd2 fscrypto 
> nls_iso8859_1 nls_cp437 evdev input_leds led_class vfat fat intel_rapl_perf 
> mac_hid pcspkr hid_logitech_dj igb ptp mei_me pps_
> core i2c_i801 i2c_algo_bit mei lpc_ich ioatdma tpm_tis tpm_tis_core dca 
> shpchp tpm wmi button sch_fq_codel sg ip_tables x_tables usbhid hid zfs(PO) 
> zunicode(PO) zavl(PO) icp(PO) sd_mod serio_raw atkbd libps2 isci ahci libsas 
> libahci xhci_pci ehci_pci mpt3sas xhci_hc
> d ehci_hcd raid_class libata
> [ 1889.349864]  scsi_transport_sas usbcore scsi_mod usb_common i8042 serio 
> zcommon(PO) znvpair(PO) spl(O) nvme nvme_core bridge stp llc vhost_net tun 
> tap vhost vfio_pci irqbypass vfio_virqfd vfio_iommu_type1 vfio
> [ 1889.368439] CPU: 1 PID: 12084 Comm: systemctl Tainted: PW  O
> 4.14.7-1-ARCH #1
> [ 1889.376525] Hardware name: Supermicro X9DA7/E/X9DA7/E, BIOS 3.0a 07/02/2014
> [ 1889.383474] task: 93149aaec140 task.stack: a88c3836c000
> [ 1889.389387] RIP: 0010:pids_free+0x28/0xb0
> [ 1889.393388] RSP: 0018:a88c3836fcc8 EFLAGS: 00010282
> [ 1889.398605] RAX:  RBX:  RCX: 
> 0006
> [ 1889.405731] RDX:  RSI: 0202 RDI: 
> 0202
> [ 1889.412854] RBP: 931499ab2d58 R08: 079a R09: 
> 
> [ 1889.419979] R10: 001f5954 R11: 0003d040 R12: 
> 56e21a48
> [ 1889.427102] R13: a91de5c0 R14: 93247b0598c0 R15: 
> a91cd0a0
> [ 1889.434227] FS:  7f18eee6b8c0() GS:931ebfa4() 
> knlGS:
> [ 1889.442302] CS:  0010 DS:  ES:  CR0: 80050033
> [ 1889.448041] CR2: 00b0 CR3: 000611019003 CR4: 
> 001626e0
> [ 1889.455164] Call Trace:
> [ 1889.457610]  cgroup_free+0xaa/0x190
> [ 1889.461095]  __put_task_struct+0x68/0x230
> [ 1889.465105]  ? seq_printf+0x4e/0x70
> [ 1889.468591]  css_task_iter_next+0x74/0x90
> [ 1889.472594]  kernfs_seq_next+0x58/0x110
> [ 1889.476424]  seq_read+0x36c/0x620
> [ 1889.479735]  __vfs_read+0x54/0x2e0
> [ 1889.483134]  vfs_read+0x9d/0x200
> [ 1889.486358]  SyS_read+0x52/0xc0
> [ 1889.489494]  do_syscall_64+0x69/0x1e0
> [ 1889.493152]  entry_SYSCALL64_slow_path+0x25/0x25
> [ 1889.497771] RIP: 0033:0x7f18ee784a11
> [ 1889.501341] RSP: 002b:7ffd56942618 EFLAGS: 0246 ORIG_RAX: 
> 
> [ 1889.508897] RAX: ffda RBX: 559a9ae6d260 RCX: 
> 7f18ee784a11
> [ 1889.516022] RDX: 1000 RSI: 559a9ae80f70 RDI: 
> 0008
> [ 1889.523145] RBP: 0d68 R08: 0003 R09: 
> ffb0
> [ 1889.530270] R10: 1000 R11: 0246 R12: 
> 7f18eea4b700
> [ 1889.537395] R13: 7f18eea4c240 R14: 559a9ae6d260 R15: 
> 
> [ 1889.544518] Code: 44 00 00 0f 1f 44 00 00 48 81 ff c8 f7 ff ff 55 53 48 89 
> fb 74 4c 48 8b 9b 38 08 00 00 48 85 db 74 7c 48 8b 5b 50 48 85 db 74 63 <48> 
> 83 bb b0 00 00 00 00 74 2a 48 c7 c5 60 2e 1e a9 48 89 df e8
> [ 1889.563368] RIP: 

Re: Detecting RWF_NOWAIT support

[vcaputo]

On Sat, Dec 16, 2017 at 10:03:38AM -0800, vcap...@pengaru.com wrote:
> On Sat, Dec 16, 2017 at 04:49:08PM +0200, Avi Kivity wrote:
> > 
> > 
> > On 12/14/2017 09:15 PM, Goldwyn Rodrigues wrote:
> > > 
> > > On 12/14/2017 11:38 AM, Avi Kivity wrote:
> > > > I'm looking to add support for RWF_NOWAIT within a linux-aio iocb.
> > > > Naturally, I need to detect at runtime whether the kernel support
> > > > RWF_NOWAIT or not.
> > > > 
> > > > 
> > > > The only method I could find was to issue an I/O with RWF_NOWAIT set,
> > > > and look for errors. This is somewhat less than perfect:
> > > > 
> > > >   - from the error, I can't tell whether RWF_NOWAIT was the problem, or
> > > > something else. If I enable a number of new features, I have to run
> > > > through all combinations to figure out which ones are supported and
> > > > which are not.
> > > Here is the return codes for RWF_NOWAIT
> > > EINVAL - not supported (older kernel)
> > > EOPNOTSUPP - not supported
> > > EAGAIN - supported but could not complete because I/O will be delayed
> > 
> > Which of these are returned from io_submit() and which are returned in the
> > iocb?
> > 
> > > 0 - supported and I/O completed (success).
> > > 
> > > >   - RWF_NOWAIT support is per-filesystem, so I can't just remember not 
> > > > to
> > > > enable RWF_NOWAIT globally, I have to track it per file.
> > > Yes, the support is per filesystem. So, the application must know if the
> > > filesystem supports it, possibly by performing a small I/O.
> > 
> > So the application must know about filesystem mount points, and be prepared
> > to create a file and try to write it (in case the filesystem is empty) or
> > alter its behavior during runtime depending on the errors it sees.
> 
> Can't the application simply add a "nowait" flag to its open file
> descriptor encapsulation struct, then in the constructor perform a
> zero-length RWF_NOWAIT write immediately after opening the fd to set the
> flag?  Then all writes branch according to the flag.
> 
> According to write(2):
> 
>If  count is zero and fd refers to a regular file, then write()
>may return a failure status if one of the errors below is
>detected.  If no errors are detected,  or error detection is not
>performed, 0 will be returned without causing any other effect.
>If count is zero and fd refers to a file  other than a regular
>file, the results are not specified.
> 
> So the zero-length RWF_NOWAIT write should return zero, unless it's not
> supported.
> 

Oh, I assumed this new flag applied to pwritev2() flags.  Disregard my
comment, I see the ambiguity causing your question Avi and do not know
the best approach.

Cheers,
Vito Caputo

Re: Detecting RWF_NOWAIT support

[vcaputo]

On Sat, Dec 16, 2017 at 10:03:38AM -0800, vcap...@pengaru.com wrote:
> On Sat, Dec 16, 2017 at 04:49:08PM +0200, Avi Kivity wrote:
> > 
> > 
> > On 12/14/2017 09:15 PM, Goldwyn Rodrigues wrote:
> > > 
> > > On 12/14/2017 11:38 AM, Avi Kivity wrote:
> > > > I'm looking to add support for RWF_NOWAIT within a linux-aio iocb.
> > > > Naturally, I need to detect at runtime whether the kernel support
> > > > RWF_NOWAIT or not.
> > > > 
> > > > 
> > > > The only method I could find was to issue an I/O with RWF_NOWAIT set,
> > > > and look for errors. This is somewhat less than perfect:
> > > > 
> > > >   - from the error, I can't tell whether RWF_NOWAIT was the problem, or
> > > > something else. If I enable a number of new features, I have to run
> > > > through all combinations to figure out which ones are supported and
> > > > which are not.
> > > Here is the return codes for RWF_NOWAIT
> > > EINVAL - not supported (older kernel)
> > > EOPNOTSUPP - not supported
> > > EAGAIN - supported but could not complete because I/O will be delayed
> > 
> > Which of these are returned from io_submit() and which are returned in the
> > iocb?
> > 
> > > 0 - supported and I/O completed (success).
> > > 
> > > >   - RWF_NOWAIT support is per-filesystem, so I can't just remember not 
> > > > to
> > > > enable RWF_NOWAIT globally, I have to track it per file.
> > > Yes, the support is per filesystem. So, the application must know if the
> > > filesystem supports it, possibly by performing a small I/O.
> > 
> > So the application must know about filesystem mount points, and be prepared
> > to create a file and try to write it (in case the filesystem is empty) or
> > alter its behavior during runtime depending on the errors it sees.
> 
> Can't the application simply add a "nowait" flag to its open file
> descriptor encapsulation struct, then in the constructor perform a
> zero-length RWF_NOWAIT write immediately after opening the fd to set the
> flag?  Then all writes branch according to the flag.
> 
> According to write(2):
> 
>If  count is zero and fd refers to a regular file, then write()
>may return a failure status if one of the errors below is
>detected.  If no errors are detected,  or error detection is not
>performed, 0 will be returned without causing any other effect.
>If count is zero and fd refers to a file  other than a regular
>file, the results are not specified.
> 
> So the zero-length RWF_NOWAIT write should return zero, unless it's not
> supported.
> 

Oh, I assumed this new flag applied to pwritev2() flags.  Disregard my
comment, I see the ambiguity causing your question Avi and do not know
the best approach.

Cheers,
Vito Caputo

Re: Detecting RWF_NOWAIT support

[vcaputo]

On Sat, Dec 16, 2017 at 04:49:08PM +0200, Avi Kivity wrote:
> 
> 
> On 12/14/2017 09:15 PM, Goldwyn Rodrigues wrote:
> > 
> > On 12/14/2017 11:38 AM, Avi Kivity wrote:
> > > I'm looking to add support for RWF_NOWAIT within a linux-aio iocb.
> > > Naturally, I need to detect at runtime whether the kernel support
> > > RWF_NOWAIT or not.
> > > 
> > > 
> > > The only method I could find was to issue an I/O with RWF_NOWAIT set,
> > > and look for errors. This is somewhat less than perfect:
> > > 
> > >   - from the error, I can't tell whether RWF_NOWAIT was the problem, or
> > > something else. If I enable a number of new features, I have to run
> > > through all combinations to figure out which ones are supported and
> > > which are not.
> > Here is the return codes for RWF_NOWAIT
> > EINVAL - not supported (older kernel)
> > EOPNOTSUPP - not supported
> > EAGAIN - supported but could not complete because I/O will be delayed
> 
> Which of these are returned from io_submit() and which are returned in the
> iocb?
> 
> > 0 - supported and I/O completed (success).
> > 
> > >   - RWF_NOWAIT support is per-filesystem, so I can't just remember not to
> > > enable RWF_NOWAIT globally, I have to track it per file.
> > Yes, the support is per filesystem. So, the application must know if the
> > filesystem supports it, possibly by performing a small I/O.
> 
> So the application must know about filesystem mount points, and be prepared
> to create a file and try to write it (in case the filesystem is empty) or
> alter its behavior during runtime depending on the errors it sees.

Can't the application simply add a "nowait" flag to its open file
descriptor encapsulation struct, then in the constructor perform a
zero-length RWF_NOWAIT write immediately after opening the fd to set the
flag?  Then all writes branch according to the flag.

According to write(2):

   If  count is zero and fd refers to a regular file, then write()
   may return a failure status if one of the errors below is
   detected.  If no errors are detected,  or error detection is not
   performed, 0 will be returned without causing any other effect.
   If count is zero and fd refers to a file  other than a regular
   file, the results are not specified.

So the zero-length RWF_NOWAIT write should return zero, unless it's not
supported.

Regards,
Vito Caputo

Re: Detecting RWF_NOWAIT support

[vcaputo]

On Sat, Dec 16, 2017 at 04:49:08PM +0200, Avi Kivity wrote:
> 
> 
> On 12/14/2017 09:15 PM, Goldwyn Rodrigues wrote:
> > 
> > On 12/14/2017 11:38 AM, Avi Kivity wrote:
> > > I'm looking to add support for RWF_NOWAIT within a linux-aio iocb.
> > > Naturally, I need to detect at runtime whether the kernel support
> > > RWF_NOWAIT or not.
> > > 
> > > 
> > > The only method I could find was to issue an I/O with RWF_NOWAIT set,
> > > and look for errors. This is somewhat less than perfect:
> > > 
> > >   - from the error, I can't tell whether RWF_NOWAIT was the problem, or
> > > something else. If I enable a number of new features, I have to run
> > > through all combinations to figure out which ones are supported and
> > > which are not.
> > Here is the return codes for RWF_NOWAIT
> > EINVAL - not supported (older kernel)
> > EOPNOTSUPP - not supported
> > EAGAIN - supported but could not complete because I/O will be delayed
> 
> Which of these are returned from io_submit() and which are returned in the
> iocb?
> 
> > 0 - supported and I/O completed (success).
> > 
> > >   - RWF_NOWAIT support is per-filesystem, so I can't just remember not to
> > > enable RWF_NOWAIT globally, I have to track it per file.
> > Yes, the support is per filesystem. So, the application must know if the
> > filesystem supports it, possibly by performing a small I/O.
> 
> So the application must know about filesystem mount points, and be prepared
> to create a file and try to write it (in case the filesystem is empty) or
> alter its behavior during runtime depending on the errors it sees.

Can't the application simply add a "nowait" flag to its open file
descriptor encapsulation struct, then in the constructor perform a
zero-length RWF_NOWAIT write immediately after opening the fd to set the
flag?  Then all writes branch according to the flag.

According to write(2):

   If  count is zero and fd refers to a regular file, then write()
   may return a failure status if one of the errors below is
   detected.  If no errors are detected,  or error detection is not
   performed, 0 will be returned without causing any other effect.
   If count is zero and fd refers to a file  other than a regular
   file, the results are not specified.

So the zero-length RWF_NOWAIT write should return zero, unless it's not
supported.

Regards,
Vito Caputo

Re: [REGRESSION] (>= v4.12) IO w/dmcrypt causing audio underruns

[vcaputo]

On Wed, Nov 29, 2017 at 10:39:19AM -0800, vcap...@pengaru.com wrote:
> Hello,
> 
> Recently I noticed substantial audio dropouts when listening to MP3s in
> `cmus` while doing big and churny `git checkout` commands in my linux git
> tree.
> 
> It's not something I've done much of over the last couple months so I
> hadn't noticed until yesterday, but didn't remember this being a problem in
> recent history.
> 
> As there's quite an accumulation of similarly configured and built kernels
> in my grub menu, it was trivial to determine approximately when this began:
> 
> 4.11.0: no dropouts
> 4.12.0-rc7: dropouts
> 4.14.0-rc6: dropouts (seem more substantial as well, didn't investigate)
> 
> Watching top while this is going on in the various kernel versions, it's
> apparent that the kworker behavior changed.  Both the priority and quantity
> of running kworker threads is elevated in kernels experiencing dropouts.
> 
> Searching through the commit history for v4.11..v4.12 uncovered:
> 
> commit a1b89132dc4f61071bdeaab92ea958e0953380a1
> Author: Tim Murray 
> Date:   Fri Apr 21 11:11:36 2017 +0200
> 
> dm crypt: use WQ_HIGHPRI for the IO and crypt workqueues
> 
> Running dm-crypt with workqueues at the standard priority results in IO
> competing for CPU time with standard user apps, which can lead to
> pipeline bubbles and seriously degraded performance.  Move to using
> WQ_HIGHPRI workqueues to protect against that.
> 
> Signed-off-by: Tim Murray 
> Signed-off-by: Enric Balletbo i Serra 
> Signed-off-by: Mike Snitzer 
> 
> ---
> 
> Reverting a1b8913 from 4.14.0-rc6, my current kernel, eliminates the
> problem completely.
> 
> Looking at the diff in that commit, it looks like the commit message isn't
> even accurate; not only is the priority of the dmcrypt workqueues being
> changed - they're also being made "CPU intensive" workqueues as well.
> 
> This combination appears to result in both elevated scheduling priority and
> greater quantity of participant worker threads effectively starving any
> normal priority user task under periods of heavy IO on dmcrypt volumes.
> 
> I don't know what the right solution is here.  It seems to me we're lacking
> the appropriate mechanism for charging CPU resources consumed on behalf of
> user processes in kworker threads to the work-causing process.
> 
> What effectively happens is my normal `git` user process is able to
> greatly amplify what share of CPU it takes from the system by generating IO
> on what happens to be a high-priority CPU-intensive storage volume.
> 
> It looks potentially complicated to fix properly, but I suspect at its core
> this may be a fairly longstanding shortcoming of the page cache and its
> asynchronous design.  Something that has been exacerbated substantially by
> the introduction of CPU-intensive storage subsystems like dmcrypt.
> 
> If we imagine the whole stack simplified, where all the IO was being done
> synchronously in-band, and the dmcrypt kernel code simply ran in the
> IO-causing process context, it would be getting charged to the calling
> process and scheduled accordingly.  The resource accounting and scheduling
> problems all emerge with the page cache, buffered IO, and async background
> writeback in a pool of unrelated worker threads, etc.  That's how it
> appears to me anyways...
> 
> The system used is a X61s Thinkpad 1.8Ghz with 840 EVO SSD, lvm on dmcrypt.
> The kernel .config is attached in case it's of interest.
> 
> Thanks,
> Vito Caputo



Ping...

Could somebody please at least ACK receiving this so I'm not left wondering
if my mails to lkml are somehow winding up flagged as spam, thanks!

Re: [REGRESSION] (>= v4.12) IO w/dmcrypt causing audio underruns

[vcaputo]

On Wed, Nov 29, 2017 at 10:39:19AM -0800, vcap...@pengaru.com wrote:
> Hello,
> 
> Recently I noticed substantial audio dropouts when listening to MP3s in
> `cmus` while doing big and churny `git checkout` commands in my linux git
> tree.
> 
> It's not something I've done much of over the last couple months so I
> hadn't noticed until yesterday, but didn't remember this being a problem in
> recent history.
> 
> As there's quite an accumulation of similarly configured and built kernels
> in my grub menu, it was trivial to determine approximately when this began:
> 
> 4.11.0: no dropouts
> 4.12.0-rc7: dropouts
> 4.14.0-rc6: dropouts (seem more substantial as well, didn't investigate)
> 
> Watching top while this is going on in the various kernel versions, it's
> apparent that the kworker behavior changed.  Both the priority and quantity
> of running kworker threads is elevated in kernels experiencing dropouts.
> 
> Searching through the commit history for v4.11..v4.12 uncovered:
> 
> commit a1b89132dc4f61071bdeaab92ea958e0953380a1
> Author: Tim Murray 
> Date:   Fri Apr 21 11:11:36 2017 +0200
> 
> dm crypt: use WQ_HIGHPRI for the IO and crypt workqueues
> 
> Running dm-crypt with workqueues at the standard priority results in IO
> competing for CPU time with standard user apps, which can lead to
> pipeline bubbles and seriously degraded performance.  Move to using
> WQ_HIGHPRI workqueues to protect against that.
> 
> Signed-off-by: Tim Murray 
> Signed-off-by: Enric Balletbo i Serra 
> Signed-off-by: Mike Snitzer 
> 
> ---
> 
> Reverting a1b8913 from 4.14.0-rc6, my current kernel, eliminates the
> problem completely.
> 
> Looking at the diff in that commit, it looks like the commit message isn't
> even accurate; not only is the priority of the dmcrypt workqueues being
> changed - they're also being made "CPU intensive" workqueues as well.
> 
> This combination appears to result in both elevated scheduling priority and
> greater quantity of participant worker threads effectively starving any
> normal priority user task under periods of heavy IO on dmcrypt volumes.
> 
> I don't know what the right solution is here.  It seems to me we're lacking
> the appropriate mechanism for charging CPU resources consumed on behalf of
> user processes in kworker threads to the work-causing process.
> 
> What effectively happens is my normal `git` user process is able to
> greatly amplify what share of CPU it takes from the system by generating IO
> on what happens to be a high-priority CPU-intensive storage volume.
> 
> It looks potentially complicated to fix properly, but I suspect at its core
> this may be a fairly longstanding shortcoming of the page cache and its
> asynchronous design.  Something that has been exacerbated substantially by
> the introduction of CPU-intensive storage subsystems like dmcrypt.
> 
> If we imagine the whole stack simplified, where all the IO was being done
> synchronously in-band, and the dmcrypt kernel code simply ran in the
> IO-causing process context, it would be getting charged to the calling
> process and scheduled accordingly.  The resource accounting and scheduling
> problems all emerge with the page cache, buffered IO, and async background
> writeback in a pool of unrelated worker threads, etc.  That's how it
> appears to me anyways...
> 
> The system used is a X61s Thinkpad 1.8Ghz with 840 EVO SSD, lvm on dmcrypt.
> The kernel .config is attached in case it's of interest.
> 
> Thanks,
> Vito Caputo



Ping...

Could somebody please at least ACK receiving this so I'm not left wondering
if my mails to lkml are somehow winding up flagged as spam, thanks!

[REGRESSION] (>= v4.12) IO w/dmcrypt causing audio underruns

[vcaputo]

Hello,

Recently I noticed substantial audio dropouts when listening to MP3s in
`cmus` while doing big and churny `git checkout` commands in my linux git
tree.

It's not something I've done much of over the last couple months so I
hadn't noticed until yesterday, but didn't remember this being a problem in
recent history.

As there's quite an accumulation of similarly configured and built kernels
in my grub menu, it was trivial to determine approximately when this began:

4.11.0: no dropouts
4.12.0-rc7: dropouts
4.14.0-rc6: dropouts (seem more substantial as well, didn't investigate)

Watching top while this is going on in the various kernel versions, it's
apparent that the kworker behavior changed.  Both the priority and quantity
of running kworker threads is elevated in kernels experiencing dropouts.

Searching through the commit history for v4.11..v4.12 uncovered:

commit a1b89132dc4f61071bdeaab92ea958e0953380a1
Author: Tim Murray 
Date:   Fri Apr 21 11:11:36 2017 +0200

dm crypt: use WQ_HIGHPRI for the IO and crypt workqueues

Running dm-crypt with workqueues at the standard priority results in IO
competing for CPU time with standard user apps, which can lead to
pipeline bubbles and seriously degraded performance.  Move to using
WQ_HIGHPRI workqueues to protect against that.

Signed-off-by: Tim Murray 
Signed-off-by: Enric Balletbo i Serra 
Signed-off-by: Mike Snitzer 

---

Reverting a1b8913 from 4.14.0-rc6, my current kernel, eliminates the
problem completely.

Looking at the diff in that commit, it looks like the commit message isn't
even accurate; not only is the priority of the dmcrypt workqueues being
changed - they're also being made "CPU intensive" workqueues as well.

This combination appears to result in both elevated scheduling priority and
greater quantity of participant worker threads effectively starving any
normal priority user task under periods of heavy IO on dmcrypt volumes.

I don't know what the right solution is here.  It seems to me we're lacking
the appropriate mechanism for charging CPU resources consumed on behalf of
user processes in kworker threads to the work-causing process.

What effectively happens is my normal `git` user process is able to
greatly amplify what share of CPU it takes from the system by generating IO
on what happens to be a high-priority CPU-intensive storage volume.

It looks potentially complicated to fix properly, but I suspect at its core
this may be a fairly longstanding shortcoming of the page cache and its
asynchronous design.  Something that has been exacerbated substantially by
the introduction of CPU-intensive storage subsystems like dmcrypt.

If we imagine the whole stack simplified, where all the IO was being done
synchronously in-band, and the dmcrypt kernel code simply ran in the
IO-causing process context, it would be getting charged to the calling
process and scheduled accordingly.  The resource accounting and scheduling
problems all emerge with the page cache, buffered IO, and async background
writeback in a pool of unrelated worker threads, etc.  That's how it
appears to me anyways...

The system used is a X61s Thinkpad 1.8Ghz with 840 EVO SSD, lvm on dmcrypt.
The kernel .config is attached in case it's of interest.

Thanks,
Vito Caputo


config-x61s.gz
Description: application/gzip

[REGRESSION] (>= v4.12) IO w/dmcrypt causing audio underruns

[vcaputo]

Hello,

Recently I noticed substantial audio dropouts when listening to MP3s in
`cmus` while doing big and churny `git checkout` commands in my linux git
tree.

It's not something I've done much of over the last couple months so I
hadn't noticed until yesterday, but didn't remember this being a problem in
recent history.

As there's quite an accumulation of similarly configured and built kernels
in my grub menu, it was trivial to determine approximately when this began:

4.11.0: no dropouts
4.12.0-rc7: dropouts
4.14.0-rc6: dropouts (seem more substantial as well, didn't investigate)

Watching top while this is going on in the various kernel versions, it's
apparent that the kworker behavior changed.  Both the priority and quantity
of running kworker threads is elevated in kernels experiencing dropouts.

Searching through the commit history for v4.11..v4.12 uncovered:

commit a1b89132dc4f61071bdeaab92ea958e0953380a1
Author: Tim Murray 
Date:   Fri Apr 21 11:11:36 2017 +0200

dm crypt: use WQ_HIGHPRI for the IO and crypt workqueues

Running dm-crypt with workqueues at the standard priority results in IO
competing for CPU time with standard user apps, which can lead to
pipeline bubbles and seriously degraded performance.  Move to using
WQ_HIGHPRI workqueues to protect against that.

Signed-off-by: Tim Murray 
Signed-off-by: Enric Balletbo i Serra 
Signed-off-by: Mike Snitzer 

---

Reverting a1b8913 from 4.14.0-rc6, my current kernel, eliminates the
problem completely.

Looking at the diff in that commit, it looks like the commit message isn't
even accurate; not only is the priority of the dmcrypt workqueues being
changed - they're also being made "CPU intensive" workqueues as well.

This combination appears to result in both elevated scheduling priority and
greater quantity of participant worker threads effectively starving any
normal priority user task under periods of heavy IO on dmcrypt volumes.

I don't know what the right solution is here.  It seems to me we're lacking
the appropriate mechanism for charging CPU resources consumed on behalf of
user processes in kworker threads to the work-causing process.

What effectively happens is my normal `git` user process is able to
greatly amplify what share of CPU it takes from the system by generating IO
on what happens to be a high-priority CPU-intensive storage volume.

It looks potentially complicated to fix properly, but I suspect at its core
this may be a fairly longstanding shortcoming of the page cache and its
asynchronous design.  Something that has been exacerbated substantially by
the introduction of CPU-intensive storage subsystems like dmcrypt.

If we imagine the whole stack simplified, where all the IO was being done
synchronously in-band, and the dmcrypt kernel code simply ran in the
IO-causing process context, it would be getting charged to the calling
process and scheduled accordingly.  The resource accounting and scheduling
problems all emerge with the page cache, buffered IO, and async background
writeback in a pool of unrelated worker threads, etc.  That's how it
appears to me anyways...

The system used is a X61s Thinkpad 1.8Ghz with 840 EVO SSD, lvm on dmcrypt.
The kernel .config is attached in case it's of interest.

Thanks,
Vito Caputo


config-x61s.gz
Description: application/gzip

Re: [PATCH] net: ethernet: aquantia: default to no in config

[vcaputo]

On Tue, Sep 19, 2017 at 03:52:31PM -0700, David Miller wrote:
> From: Vito Caputo 
> Date: Tue, 19 Sep 2017 15:43:15 -0700
> 
> > NET_VENDOR_AQUANTIA was "default y" for some reason, which seems
> > obviously inappropriate.
> 
> It is appropriate.
> 
> We make all vendor guards default to yes.

Thanks for the quick response.

Out of curiosity, what's the rationale for that decision?

Re: [PATCH] net: ethernet: aquantia: default to no in config

[vcaputo]

On Tue, Sep 19, 2017 at 03:52:31PM -0700, David Miller wrote:
> From: Vito Caputo 
> Date: Tue, 19 Sep 2017 15:43:15 -0700
> 
> > NET_VENDOR_AQUANTIA was "default y" for some reason, which seems
> > obviously inappropriate.
> 
> It is appropriate.
> 
> We make all vendor guards default to yes.

Thanks for the quick response.

Out of curiosity, what's the rationale for that decision?

Re: Detecting page cache trashing state

[vcaputo]

On Fri, Sep 15, 2017 at 04:36:19PM +0200, Michal Hocko wrote:
> On Thu 14-09-17 17:16:27, Taras Kondratiuk wrote:
> > Hi
> > 
> > In our devices under low memory conditions we often get into a trashing
> > state when system spends most of the time re-reading pages of .text
> > sections from a file system (squashfs in our case). Working set doesn't
> > fit into available page cache, so it is expected. The issue is that
> > OOM killer doesn't get triggered because there is still memory for
> > reclaiming. System may stuck in this state for a quite some time and
> > usually dies because of watchdogs.
> > 
> > We are trying to detect such trashing state early to take some
> > preventive actions. It should be a pretty common issue, but for now we
> > haven't find any existing VM/IO statistics that can reliably detect such
> > state.
> > 
> > Most of metrics provide absolute values: number/rate of page faults,
> > rate of IO operations, number of stolen pages, etc. For a specific
> > device configuration we can determine threshold values for those
> > parameters that will detect trashing state, but it is not feasible for
> > hundreds of device configurations.
> > 
> > We are looking for some relative metric like "percent of CPU time spent
> > handling major page faults". With such relative metric we could use a
> > common threshold across all devices. For now we have added such metric
> > to /proc/stat in our kernel, but we would like to find some mechanism
> > available in upstream kernel.
> > 
> > Has somebody faced similar issue? How are you solving it?
> 
> Yes this is a pain point for a _long_ time. And we still do not have a
> good answer upstream. Johannes has been playing in this area [1].
> The main problem is that our OOM detection logic is based on the ability
> to reclaim memory to allocate new memory. And that is pretty much true
> for the pagecache when you are trashing. So we do not know that
> basically whole time is spent refaulting the memory back and forth.
> We do have some refault stats for the page cache but that is not
> integrated to the oom detection logic because this is really a
> non-trivial problem to solve without triggering early oom killer
> invocations.
> 
> [1] http://lkml.kernel.org/r/20170727153010.23347-1-han...@cmpxchg.org

For desktop users running without swap, couldn't we just provide a kernel
setting which marks all executable pages as unevictable when first faulted
in?  Then at least thrashing within the space occupied by executables and
shared libraries before eventual OOM would be avoided, and only the
remaining file-backed non-executable pages would be thrashable.

On my swapless laptops I'd much rather have OOM killer kick in immediately
rather than wait for a few minutes of thrashing to pass while the bogged
down system crawls through depleting what's left of technically reclaimable
memory.  It's much improved on modern SSDs, but still annoying.

Regards,
Vito Caputo

Re: Detecting page cache trashing state

[vcaputo]

On Fri, Sep 15, 2017 at 04:36:19PM +0200, Michal Hocko wrote:
> On Thu 14-09-17 17:16:27, Taras Kondratiuk wrote:
> > Hi
> > 
> > In our devices under low memory conditions we often get into a trashing
> > state when system spends most of the time re-reading pages of .text
> > sections from a file system (squashfs in our case). Working set doesn't
> > fit into available page cache, so it is expected. The issue is that
> > OOM killer doesn't get triggered because there is still memory for
> > reclaiming. System may stuck in this state for a quite some time and
> > usually dies because of watchdogs.
> > 
> > We are trying to detect such trashing state early to take some
> > preventive actions. It should be a pretty common issue, but for now we
> > haven't find any existing VM/IO statistics that can reliably detect such
> > state.
> > 
> > Most of metrics provide absolute values: number/rate of page faults,
> > rate of IO operations, number of stolen pages, etc. For a specific
> > device configuration we can determine threshold values for those
> > parameters that will detect trashing state, but it is not feasible for
> > hundreds of device configurations.
> > 
> > We are looking for some relative metric like "percent of CPU time spent
> > handling major page faults". With such relative metric we could use a
> > common threshold across all devices. For now we have added such metric
> > to /proc/stat in our kernel, but we would like to find some mechanism
> > available in upstream kernel.
> > 
> > Has somebody faced similar issue? How are you solving it?
> 
> Yes this is a pain point for a _long_ time. And we still do not have a
> good answer upstream. Johannes has been playing in this area [1].
> The main problem is that our OOM detection logic is based on the ability
> to reclaim memory to allocate new memory. And that is pretty much true
> for the pagecache when you are trashing. So we do not know that
> basically whole time is spent refaulting the memory back and forth.
> We do have some refault stats for the page cache but that is not
> integrated to the oom detection logic because this is really a
> non-trivial problem to solve without triggering early oom killer
> invocations.
> 
> [1] http://lkml.kernel.org/r/20170727153010.23347-1-han...@cmpxchg.org

For desktop users running without swap, couldn't we just provide a kernel
setting which marks all executable pages as unevictable when first faulted
in?  Then at least thrashing within the space occupied by executables and
shared libraries before eventual OOM would be avoided, and only the
remaining file-backed non-executable pages would be thrashable.

On my swapless laptops I'd much rather have OOM killer kick in immediately
rather than wait for a few minutes of thrashing to pass while the bogged
down system crawls through depleting what's left of technically reclaimable
memory.  It's much improved on modern SSDs, but still annoying.

Regards,
Vito Caputo

Seemingly random performance degradation in 4.13

[vcaputo]

Hello,

I've been working on some graphics hacks which has me paying close
attention to frame rates.  Something I've noticed recently in the 4.13
cycle is a seemingly random loss of ~40% frame rate.  These are purely
software-rendered hacks, and I haven't spent much time trying to figure out
what is changing when the performance drops - typically the performance is
restored on a subsequent boot.

There's been talk of NUMA and scheduler related performance regressions,
and I'm wondering if that's related to what I'm observing.

Is there a fix floating around I can apply and see if it prevents
regressions from occurring?

FYI the machine in question is a 1.8Ghz core2 duo thinkpad x61s, currently
on 4.13-rc7.

If there's anything in particular that would be useful for me to capture
when running one of these tests, a specific perf incantation for example,
I'm happy to test and report the results.

Cheers,
Vito Caputo

Seemingly random performance degradation in 4.13

[vcaputo]

Hello,

I've been working on some graphics hacks which has me paying close
attention to frame rates.  Something I've noticed recently in the 4.13
cycle is a seemingly random loss of ~40% frame rate.  These are purely
software-rendered hacks, and I haven't spent much time trying to figure out
what is changing when the performance drops - typically the performance is
restored on a subsequent boot.

There's been talk of NUMA and scheduler related performance regressions,
and I'm wondering if that's related to what I'm observing.

Is there a fix floating around I can apply and see if it prevents
regressions from occurring?

FYI the machine in question is a 1.8Ghz core2 duo thinkpad x61s, currently
on 4.13-rc7.

If there's anything in particular that would be useful for me to capture
when running one of these tests, a specific perf incantation for example,
I'm happy to test and report the results.

Cheers,
Vito Caputo

[QUESTION] v4.12-rc5 BFQ enabled but unavailable?

[vcaputo]

Hello LKML,

Attempted to play with BFQ but after building with CONFIG_IOSCHED_BFQ=y I'm
still not seeing it:
 # cat 
/sys/devices/pci:00/:00:1f.2/ata1/host0/target0:0:0/0:0:0:0/block/sda/queue/scheduler
 noop deadline [cfq] 
 # 

Even though:
 # dmesg | grep sched
 [0.514536] io scheduler noop registered
 [0.514537] io scheduler deadline registered
 [0.514577] io scheduler cfq registered (default)
 [0.514613] io scheduler bfq registered
 #

Quick glance at block/elevator.c elv_iosched_show() reveals:
 1125 list_for_each_entry(__e, _list, list) {
 1126 if (elv && !strcmp(elv->elevator_name, 
__e->elevator_name)) {
 1127 len += sprintf(name+len, "[%s] ", 
elv->elevator_name);
 1128 continue;
 1129 }
 1130 if (__e->uses_mq && q->mq_ops && elv_support_iosched(q))
 1131 len += sprintf(name+len, "%s ", 
__e->elevator_name);
 1132 else if (!__e->uses_mq && !q->mq_ops)
 1133 len += sprintf(name+len, "%s ", 
__e->elevator_name);
 1134 }

iosched_bfq_mq.uses_mq is true, so it appears bfq can be silently omitted from
elv_iosched_show() output if !q->mq_ops || !elv_support_iosched(q).

What magic incantation is recommended to get this stuff working and how is a
user like myself expected to discover it without digging through the code?

I glanced through Documentation/block/bfq-iosched.txt, but mq isn't mentioned:
 $ grep -ic mq Documentation/block/bfq-iosched.txt 
 0
 $

There are use_blk_mq parameters for dm_mod and scsi_mod, defaulting to N:
 # cd /sys/module && find -iname '*mq*' -print -exec cat {} \;
 ./dm_mod/parameters/use_blk_mq
 N
 ./dm_mod/parameters/dm_mq_nr_hw_queues
 1
 ./dm_mod/parameters/dm_mq_queue_depth
 2048
 ./scsi_mod/parameters/use_blk_mq
 N
 #

Is one expected to force use_blk_mq manually to make BFQ available?

Any advice appreciated, thanks!

Regards,
Vito Caputo

[QUESTION] v4.12-rc5 BFQ enabled but unavailable?

[vcaputo]

Hello LKML,

Attempted to play with BFQ but after building with CONFIG_IOSCHED_BFQ=y I'm
still not seeing it:
 # cat 
/sys/devices/pci:00/:00:1f.2/ata1/host0/target0:0:0/0:0:0:0/block/sda/queue/scheduler
 noop deadline [cfq] 
 # 

Even though:
 # dmesg | grep sched
 [0.514536] io scheduler noop registered
 [0.514537] io scheduler deadline registered
 [0.514577] io scheduler cfq registered (default)
 [0.514613] io scheduler bfq registered
 #

Quick glance at block/elevator.c elv_iosched_show() reveals:
 1125 list_for_each_entry(__e, _list, list) {
 1126 if (elv && !strcmp(elv->elevator_name, 
__e->elevator_name)) {
 1127 len += sprintf(name+len, "[%s] ", 
elv->elevator_name);
 1128 continue;
 1129 }
 1130 if (__e->uses_mq && q->mq_ops && elv_support_iosched(q))
 1131 len += sprintf(name+len, "%s ", 
__e->elevator_name);
 1132 else if (!__e->uses_mq && !q->mq_ops)
 1133 len += sprintf(name+len, "%s ", 
__e->elevator_name);
 1134 }

iosched_bfq_mq.uses_mq is true, so it appears bfq can be silently omitted from
elv_iosched_show() output if !q->mq_ops || !elv_support_iosched(q).

What magic incantation is recommended to get this stuff working and how is a
user like myself expected to discover it without digging through the code?

I glanced through Documentation/block/bfq-iosched.txt, but mq isn't mentioned:
 $ grep -ic mq Documentation/block/bfq-iosched.txt 
 0
 $

There are use_blk_mq parameters for dm_mod and scsi_mod, defaulting to N:
 # cd /sys/module && find -iname '*mq*' -print -exec cat {} \;
 ./dm_mod/parameters/use_blk_mq
 N
 ./dm_mod/parameters/dm_mq_nr_hw_queues
 1
 ./dm_mod/parameters/dm_mq_queue_depth
 2048
 ./scsi_mod/parameters/use_blk_mq
 N
 #

Is one expected to force use_blk_mq manually to make BFQ available?

Any advice appreciated, thanks!

Regards,
Vito Caputo

[REGRESSION] 4.11.0-rc8 ext4 mount randomly returning EBUSY during boot w/systemd?

[vcaputo]

I'm not sure about this one, but never experienced it prior to 4.11.0-rc8.

During boot sometimes the home.mount unit fails and `systemctl status
home.mount` reports that the mount failed due to it already being busy.

I'm able to fix it by simply issuing `systemctl restart home.mount` from the
maintenance shell, ^d and the boot resumes normally.

This is a plain Debian Jessie amd64 system, with dmcrypt and lvm2.

Is this a known kernel regression that got fixed already?  It's possible this
is an unrelated systemd bug, but it's odd that I never saw this before this
kernel.

Unfortunately I've been too busy the last few weeks to spend much time on the
computer in rc8+, so this is all a bit nebulous.

Regards,
Vito Caputo

[REGRESSION] 4.11.0-rc8 ext4 mount randomly returning EBUSY during boot w/systemd?

[vcaputo]

I'm not sure about this one, but never experienced it prior to 4.11.0-rc8.

During boot sometimes the home.mount unit fails and `systemctl status
home.mount` reports that the mount failed due to it already being busy.

I'm able to fix it by simply issuing `systemctl restart home.mount` from the
maintenance shell, ^d and the boot resumes normally.

This is a plain Debian Jessie amd64 system, with dmcrypt and lvm2.

Is this a known kernel regression that got fixed already?  It's possible this
is an unrelated systemd bug, but it's odd that I never saw this before this
kernel.

Unfortunately I've been too busy the last few weeks to spend much time on the
computer in rc8+, so this is all a bit nebulous.

Regards,
Vito Caputo

Re: [PATCH] shmem: fix __shmem_file_setup error path leaks

[vcaputo]

On Mon, Mar 27, 2017 at 10:21:27PM +0100, Al Viro wrote:
> On Mon, Mar 27, 2017 at 10:05:34AM -0700, Vito Caputo wrote:
> > The existing path and memory cleanups appear to be in reverse order, and
> > there's no iput() potentially leaking the inode in the last two error gotos.
> > 
> > Also make put_memory shmem_unacct_size() conditional on !inode since if we
> > entered cleanup at put_inode, shmem_evict_inode() occurs via
> > iput()->iput_final(), which performs the shmem_unacct_size() for us.
> > 
> > Signed-off-by: Vito Caputo 
> > ---
> > 
> > This caught my eye while looking through the memfd_create() implementation.
> > Included patch was compile tested only...
> 
> Obviously so, since you've just introduced a double iput() there.  After
> d_instantiate(path.dentry, inode);
> dropping the reference to path.dentry (done by path_put()) will drop
> the reference to inode transferred into that dentry by d_instantiate().
> NAK.

I see, so it's correct as-is, thanks for the review and apologies for the
noise!

Cheers,
Vito Caputo

Re: [PATCH] shmem: fix __shmem_file_setup error path leaks

[vcaputo]

On Mon, Mar 27, 2017 at 10:21:27PM +0100, Al Viro wrote:
> On Mon, Mar 27, 2017 at 10:05:34AM -0700, Vito Caputo wrote:
> > The existing path and memory cleanups appear to be in reverse order, and
> > there's no iput() potentially leaking the inode in the last two error gotos.
> > 
> > Also make put_memory shmem_unacct_size() conditional on !inode since if we
> > entered cleanup at put_inode, shmem_evict_inode() occurs via
> > iput()->iput_final(), which performs the shmem_unacct_size() for us.
> > 
> > Signed-off-by: Vito Caputo 
> > ---
> > 
> > This caught my eye while looking through the memfd_create() implementation.
> > Included patch was compile tested only...
> 
> Obviously so, since you've just introduced a double iput() there.  After
> d_instantiate(path.dentry, inode);
> dropping the reference to path.dentry (done by path_put()) will drop
> the reference to inode transferred into that dentry by d_instantiate().
> NAK.

I see, so it's correct as-is, thanks for the review and apologies for the
noise!

Cheers,
Vito Caputo

PROBLEM: Namespaced PID 1 ignoring SIG_DFL signals

[vcaputo]

Hello lkml,

According to the comment:

> * Note that if global/container-init sees a sig_kernel_only()
> * signal here, the signal must have been generated internally
> * or must have come from an ancestor namespace. In either
> * case, the signal cannot be dropped.

[https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/kernel/signal.c?id=refs/tags/v3.19#n2290]

Killing a container's PID1 shouldn't be ignored when the default-handled
signal originated from an ancestor namespace or internally.

When executing via `unshare --fork --pid` a trivial "for(;;) pause();" C
program that installs no signal handlers,  then sending SIGTERM to the
namespaced process from the parent namespace, the pause() syscall just
returns EINTR and loops rather than the process terminating.

This does not seem consistent with the intention documented in the code.

Additionally, if using a more comprehensive executor than unshare which
allocates a pty for the child and sets the pty slave as the controlling
tty pre-exec, running the same test the signals delivered by the pty
slave are identically ignored.  It's unclear to me whether signals
originating from the process' own controlling tty would be classified as
"generated internally".

IMHO both of these scenarios should result in the signal being handled,
but if that's inappropriate I'd appreciate any clarification.

Please CC me with any responses, as I'm not subscribed.

Thanks,
Vito Caputo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

PROBLEM: Namespaced PID 1 ignoring SIG_DFL signals

[vcaputo]

Hello lkml,

According to the comment:

 * Note that if global/container-init sees a sig_kernel_only()
 * signal here, the signal must have been generated internally
 * or must have come from an ancestor namespace. In either
 * case, the signal cannot be dropped.

[https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/kernel/signal.c?id=refs/tags/v3.19#n2290]

Killing a container's PID1 shouldn't be ignored when the default-handled
signal originated from an ancestor namespace or internally.

When executing via `unshare --fork --pid` a trivial for(;;) pause(); C
program that installs no signal handlers,  then sending SIGTERM to the
namespaced process from the parent namespace, the pause() syscall just
returns EINTR and loops rather than the process terminating.

This does not seem consistent with the intention documented in the code.

Additionally, if using a more comprehensive executor than unshare which
allocates a pty for the child and sets the pty slave as the controlling
tty pre-exec, running the same test the signals delivered by the pty
slave are identically ignored.  It's unclear to me whether signals
originating from the process' own controlling tty would be classified as
generated internally.

IMHO both of these scenarios should result in the signal being handled,
but if that's inappropriate I'd appreciate any clarification.

Please CC me with any responses, as I'm not subscribed.

Thanks,
Vito Caputo
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: adopt(pid_t pid) syscall proposal [patch included]

[vcaputo]

On Tue, Jun 11, 2013 at 09:48:22AM -0700, Andy Lutomirski wrote:
> On 06/10/2013 06:23 PM, vcap...@gnugeneration.com wrote:
> >+if (!uid_eq(cred->euid, tcred->suid) &&
> >+!uid_eq(cred->euid, tcred->uid)  &&
> >+!uid_eq(cred->uid,  tcred->suid) &&
> >+!uid_eq(cred->uid,  tcred->uid) &&
> >+!ns_capable(cred->user_ns, CAP_KILL)) {
> >+ret = -EPERM;
> >+goto out_unlock;
> >+}
> >+
> 
> That check's far too permissive.

I suspected, but that's easily improved, I just wanted to get this out
there and see what people thought, start the discussion, as well as
get my use case functional.  Although I'm curious, what is your cause
for concern with the existing checks?

> 
> This sounds like it will break anything that uses wait and expects its 
> children to not be stolen out from under it.

This thought crossed my mind, and originally I intended to restrict
adoption to orphans who had become children of init.  It seemed like
more general use might be desirable so I left that out.  If this
really is a possibility worth preventing we could put that restriction
on it.  To the best of my knowledge, nothing informs init of its
becoming parent of an orphan, so we should be able to steal the orphan
back without harm.  This still enables the use case of screen/tmux
reattachment.

> 
> Also, you'll have problems with screen -x or the default tmux shareable 
> configuration.  It sounds like this is better done in userspace.

It just needs some determination of "session leader" applied to which
attach adopts the backend before invoking adopt(), that logic goes in
userspace.  I imagine the implementations of both screen and tmux
already have such determinations happening for other reasons.

Regards,
Vito Caputo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: adopt(pid_t pid) syscall proposal [patch included]

[vcaputo]

On Tue, Jun 11, 2013 at 09:48:22AM -0700, Andy Lutomirski wrote:
 On 06/10/2013 06:23 PM, vcap...@gnugeneration.com wrote:
 +if (!uid_eq(cred-euid, tcred-suid) 
 +!uid_eq(cred-euid, tcred-uid)  
 +!uid_eq(cred-uid,  tcred-suid) 
 +!uid_eq(cred-uid,  tcred-uid) 
 +!ns_capable(cred-user_ns, CAP_KILL)) {
 +ret = -EPERM;
 +goto out_unlock;
 +}
 +
 
 That check's far too permissive.

I suspected, but that's easily improved, I just wanted to get this out
there and see what people thought, start the discussion, as well as
get my use case functional.  Although I'm curious, what is your cause
for concern with the existing checks?

 
 This sounds like it will break anything that uses wait and expects its 
 children to not be stolen out from under it.

This thought crossed my mind, and originally I intended to restrict
adoption to orphans who had become children of init.  It seemed like
more general use might be desirable so I left that out.  If this
really is a possibility worth preventing we could put that restriction
on it.  To the best of my knowledge, nothing informs init of its
becoming parent of an orphan, so we should be able to steal the orphan
back without harm.  This still enables the use case of screen/tmux
reattachment.

 
 Also, you'll have problems with screen -x or the default tmux shareable 
 configuration.  It sounds like this is better done in userspace.

It just needs some determination of session leader applied to which
attach adopts the backend before invoking adopt(), that logic goes in
userspace.  I imagine the implementations of both screen and tmux
already have such determinations happening for other reasons.

Regards,
Vito Caputo
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

adopt(pid_t pid) syscall proposal [patch included]

[vcaputo]

Hello all,

I'd like to propose a new system call, I know I'll probably be flamed,
but here goes.

Today there is no way to get a process which has been orphaned, and thus
become a child of init, back as a child of another process.  This is
annoying me, please allow me to explain why.

The CONFIG_CHECKPOINT_RESTORE feature enables a useful little thing in
proc which I've been taking advantage of to make some interesting
omnipresent desktop monitoring features built into my own composited
window manager.  That thing is the /proc/$pid/task/$pid/children file.

Using this file, I've created scoped per-window process monitors which
can efficiently limit their monitoring to only the client pid of the X
window and its decendants.  These monitors are always running, but
visibility is toggled when desired by a keypress, efficiency is
important and the children file helps tremendously in this regard.

So I'm thrilled with all this and everything is fantastically efficient
and simple, until the first time I detach from my screen session and
reattach.

Initially, I'll have a scenario like:

 xterm
  bash
   screen -S stuff
screen -S stuff
 bash
 bash
  top
 bash
  vi foo.c
 bash
  make
   sh -c cc -Wall ...
cc1 -quiet -I ...
 bash

Behind these rows of text I have user/cpu graphs for each process
sliding by.

After detaching from screen, then reattaching, this is what the above
turns into, as you can probably predict:

 xterm
  bash
   screen -dr stuff

All subsequent processes created through my interactions with this
screen are lost to the monitoring, since the "screen -S stuff" backend
is now stuck being a child of init.

With the attached adopt() syscall patch, and a 1 line change to screen's
attacher function invoking adopt(), I get the following on reattach:

 xterm
  bash
   screen -dr stuff
screen -S stuff
 bash
 bash
  top
 bash
  vi foo.c
 bash
  make
   sh -c cc -Wall ...
cc1 -quiet -I ...


Which makes me very happy, dance around the room happy, and I'm not a
dancer.

Here is a screenshot of the results enabled by the patch:
http://pengaru.com/~swivel/vwm/screen_sys_adopt.png

Thanks for reading, and hopefuly considering this addition, I'm sure
there are other uses as well.  Please CC me in any replies as I'm not
currently subscribed.

Regards,
Vito Caputo
>From 2d9f1781c6cdec8d74f44f98624af2eacbd21810 Mon Sep 17 00:00:00 2001
From: Vito Caputo 
Date: Mon, 10 Jun 2013 17:28:49 -0700
Subject: [PATCH 1/1] 	sched: implement adopt() system call

	This implements a proposed facility for adopting a process
	by another.  The immediate use case is programs like GNU
	screen which lose their parent-child relationship when
	detached.  Using this sytem call on reattach, the relation
	can be restored, which is particularly of use to those
	taking advantage of the /proc/$pid/task/$pid/children list
	provided by CONFIG_CHECKPOINT_RESTORE.

	This implementation applies permission checks similar to
	that of kill(), in addition to preventing the adoption of
	an ancestor process.

	I have tested it and use the change myself to complement
	process subtree monitoring, without which I lose all
	visibility of the descendants of my screen sessions on
	reattach.

Signed-off-by: Vito Caputo 
---
 arch/x86/syscalls/syscall_32.tbl |1 +
 arch/x86/syscalls/syscall_64.tbl |1 +
 include/linux/syscalls.h |2 ++
 kernel/exit.c|   59 ++
 4 files changed, 63 insertions(+)

diff --git a/arch/x86/syscalls/syscall_32.tbl b/arch/x86/syscalls/syscall_32.tbl
index aabfb83..d219781 100644
--- a/arch/x86/syscalls/syscall_32.tbl
+++ b/arch/x86/syscalls/syscall_32.tbl
@@ -357,3 +357,4 @@
 348	i386	process_vm_writev	sys_process_vm_writev		compat_sys_process_vm_writev
 349	i386	kcmp			sys_kcmp
 350	i386	finit_module		sys_finit_module
+351	i386	adopt			sys_adopt
diff --git a/arch/x86/syscalls/syscall_64.tbl b/arch/x86/syscalls/syscall_64.tbl
index 38ae65d..6345ebf 100644
--- a/arch/x86/syscalls/syscall_64.tbl
+++ b/arch/x86/syscalls/syscall_64.tbl
@@ -320,6 +320,7 @@
 311	64	process_vm_writev	sys_process_vm_writev
 312	common	kcmp			sys_kcmp
 313	common	finit_module		sys_finit_module
+314	common	adopt			sys_adopt
 
 #
 # x32-specific system call numbers start at 512 to avoid cache impact
diff --git a/include/linux/syscalls.h b/include/linux/syscalls.h
index 4147d70..3997cde 100644
--- a/include/linux/syscalls.h
+++ b/include/linux/syscalls.h
@@ -841,4 +841,6 @@ asmlinkage long sys_process_vm_writev(pid_t pid,
 asmlinkage long sys_kcmp(pid_t pid1, pid_t pid2, int type,
 			 unsigned long idx1, unsigned long idx2);
 asmlinkage long sys_finit_module(int fd, const char __user *uargs, int flags);
+
+asmlinkage long sys_adopt(pid_t upid);
 #endif
diff --git a/kernel/exit.c b/kernel/exit.c
index af2eb3c..5b554c8 100644
--- a/kernel/exit.c
+++ b/kernel/exit.c
@@ -1681,3 +1681,62 @@ 

adopt(pid_t pid) syscall proposal [patch included]

[vcaputo]

Hello all,

I'd like to propose a new system call, I know I'll probably be flamed,
but here goes.

Today there is no way to get a process which has been orphaned, and thus
become a child of init, back as a child of another process.  This is
annoying me, please allow me to explain why.

The CONFIG_CHECKPOINT_RESTORE feature enables a useful little thing in
proc which I've been taking advantage of to make some interesting
omnipresent desktop monitoring features built into my own composited
window manager.  That thing is the /proc/$pid/task/$pid/children file.

Using this file, I've created scoped per-window process monitors which
can efficiently limit their monitoring to only the client pid of the X
window and its decendants.  These monitors are always running, but
visibility is toggled when desired by a keypress, efficiency is
important and the children file helps tremendously in this regard.

So I'm thrilled with all this and everything is fantastically efficient
and simple, until the first time I detach from my screen session and
reattach.

Initially, I'll have a scenario like:

 xterm
  bash
   screen -S stuff
screen -S stuff
 bash
 bash
  top
 bash
  vi foo.c
 bash
  make
   sh -c cc -Wall ...
cc1 -quiet -I ...
 bash

Behind these rows of text I have user/cpu graphs for each process
sliding by.

After detaching from screen, then reattaching, this is what the above
turns into, as you can probably predict:

 xterm
  bash
   screen -dr stuff

All subsequent processes created through my interactions with this
screen are lost to the monitoring, since the screen -S stuff backend
is now stuck being a child of init.

With the attached adopt() syscall patch, and a 1 line change to screen's
attacher function invoking adopt(), I get the following on reattach:

 xterm
  bash
   screen -dr stuff
screen -S stuff
 bash
 bash
  top
 bash
  vi foo.c
 bash
  make
   sh -c cc -Wall ...
cc1 -quiet -I ...


Which makes me very happy, dance around the room happy, and I'm not a
dancer.

Here is a screenshot of the results enabled by the patch:
http://pengaru.com/~swivel/vwm/screen_sys_adopt.png

Thanks for reading, and hopefuly considering this addition, I'm sure
there are other uses as well.  Please CC me in any replies as I'm not
currently subscribed.

Regards,
Vito Caputo
From 2d9f1781c6cdec8d74f44f98624af2eacbd21810 Mon Sep 17 00:00:00 2001
From: Vito Caputo vcap...@gnugeneration.com
Date: Mon, 10 Jun 2013 17:28:49 -0700
Subject: [PATCH 1/1] 	sched: implement adopt() system call

	This implements a proposed facility for adopting a process
	by another.  The immediate use case is programs like GNU
	screen which lose their parent-child relationship when
	detached.  Using this sytem call on reattach, the relation
	can be restored, which is particularly of use to those
	taking advantage of the /proc/$pid/task/$pid/children list
	provided by CONFIG_CHECKPOINT_RESTORE.

	This implementation applies permission checks similar to
	that of kill(), in addition to preventing the adoption of
	an ancestor process.

	I have tested it and use the change myself to complement
	process subtree monitoring, without which I lose all
	visibility of the descendants of my screen sessions on
	reattach.

Signed-off-by: Vito Caputo vcap...@gnugeneration.com
---
 arch/x86/syscalls/syscall_32.tbl |1 +
 arch/x86/syscalls/syscall_64.tbl |1 +
 include/linux/syscalls.h |2 ++
 kernel/exit.c|   59 ++
 4 files changed, 63 insertions(+)

diff --git a/arch/x86/syscalls/syscall_32.tbl b/arch/x86/syscalls/syscall_32.tbl
index aabfb83..d219781 100644
--- a/arch/x86/syscalls/syscall_32.tbl
+++ b/arch/x86/syscalls/syscall_32.tbl
@@ -357,3 +357,4 @@
 348	i386	process_vm_writev	sys_process_vm_writev		compat_sys_process_vm_writev
 349	i386	kcmp			sys_kcmp
 350	i386	finit_module		sys_finit_module
+351	i386	adopt			sys_adopt
diff --git a/arch/x86/syscalls/syscall_64.tbl b/arch/x86/syscalls/syscall_64.tbl
index 38ae65d..6345ebf 100644
--- a/arch/x86/syscalls/syscall_64.tbl
+++ b/arch/x86/syscalls/syscall_64.tbl
@@ -320,6 +320,7 @@
 311	64	process_vm_writev	sys_process_vm_writev
 312	common	kcmp			sys_kcmp
 313	common	finit_module		sys_finit_module
+314	common	adopt			sys_adopt
 
 #
 # x32-specific system call numbers start at 512 to avoid cache impact
diff --git a/include/linux/syscalls.h b/include/linux/syscalls.h
index 4147d70..3997cde 100644
--- a/include/linux/syscalls.h
+++ b/include/linux/syscalls.h
@@ -841,4 +841,6 @@ asmlinkage long sys_process_vm_writev(pid_t pid,
 asmlinkage long sys_kcmp(pid_t pid1, pid_t pid2, int type,
 			 unsigned long idx1, unsigned long idx2);
 asmlinkage long sys_finit_module(int fd, const char __user *uargs, int flags);
+
+asmlinkage long sys_adopt(pid_t upid);
 #endif
diff --git a/kernel/exit.c b/kernel/exit.c
index af2eb3c..5b554c8 100644
--- a/kernel/exit.c
+++