Re: unknown NMI on AMD Rome

[Adam Borowski]

On Tue, Mar 16, 2021 at 04:45:02PM +0100, Jiri Olsa wrote:
> hi,
> when running 'perf top' on AMD Rome (/proc/cpuinfo below)
> with fedora 33 kernel 5.10.22-200.fc33.x86_64
> 
> we got unknown NMI messages:
> 
> [  226.700160] Uhhuh. NMI received for unknown reason 3d on CPU 90.
> [  226.700162] Do you have a strange power saving mode enabled?
> [  226.700163] Dazed and confused, but trying to continue
> 
> also when discussing ths with Borislav, he managed to reproduce easily
> on his AMD Rome machine

Likewise, 3c on Pinnacle Ridge.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ in the beginning was the boot and root floppies and they were good.
⢿⡄⠘⠷⠚⠋⠀   --  on #linux-sunxi
⠈⠳⣄

Re: [PATCH v2 00/10] fsdax,xfs: Add reflink&dedupe support for fsdax

[Adam Borowski]

On Sat, Mar 13, 2021 at 11:24:00AM -0500, Neal Gompa wrote:
> On Sat, Mar 13, 2021 at 8:09 AM Adam Borowski  wrote:
> >
> > On Wed, Mar 10, 2021 at 02:26:43PM +, Matthew Wilcox wrote:
> > > On Wed, Mar 10, 2021 at 08:21:59AM -0600, Goldwyn Rodrigues wrote:
> > > > DAX on btrfs has been attempted[1]. Of course, we could not
> > >
> > > But why?  A completeness fetish?  I don't understand why you decided
> > > to do this work.
> >
> > * xfs can shapshot only single files, btrfs entire subvolumes
> > * btrfs-send|receive
> > * enumeration of changed parts of a file
> 
> XFS cannot do snapshots since it lacks metadata COW. XFS reflinking is
> primarily for space efficiency.

A reflink is a single-file snapshot.

My work team really wants this very patchset -- reflinks on DAX allow
backups and/or checkpointing, without stopping the world (there's a single
file, "pool", here).

Besides, you can still get poor-man's whole-subvolume(/directory)
snapshots by manually walking the tree and reflinking everything.
That's not atomic -- but rsync isn't atomic either.  That's enough for
eg. dnf/dpkg purposes.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢰⠒⠀⣿⡁
⢿⡄⠘⠷⠚⠋⠀ NADIE anticipa la inquisición de españa!
⠈⠳⣄

Re: [PATCH v2 00/10] fsdax,xfs: Add reflink&dedupe support for fsdax

[Adam Borowski]

On Wed, Mar 10, 2021 at 02:26:43PM +, Matthew Wilcox wrote:
> On Wed, Mar 10, 2021 at 08:21:59AM -0600, Goldwyn Rodrigues wrote:
> > DAX on btrfs has been attempted[1]. Of course, we could not
> 
> But why?  A completeness fetish?  I don't understand why you decided
> to do this work.

* xfs can shapshot only single files, btrfs entire subvolumes
* btrfs-send|receive
* enumeration of changed parts of a file


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ I've read an article about how lively happy music boosts
⣾⠁⢠⠒⠀⣿⡁ productivity.  You can read it, too, you just need the
⢿⡄⠘⠷⠚⠋⠀ right music while doing so.  I recommend Skepticism
⠈⠳⣄ (funeral doom metal).

MaxLinear, please maintain your drivers was Re: [PATCH] leds: lgm: fix gpiolib dependency

[Adam Borowski]

On Tue, Mar 09, 2021 at 08:39:10PM +0100, Pavel Machek wrote:
> > I'd like people from Intel to contact me. There's more to fix there,
> > and AFAICT original author went away.
> 
> The following message to  was
> undeliverable.

> : Recipient
> +address rejected: User unknown in virtual mailbox table'

> commit c3987cd2bca34ddfec69027acedb2fae5ffcf7a0
> Author: Amireddy Mallikarjuna reddy 

I asked around, and got told Mallikarjuna has been "sold" to MaxLinear,
together with the rest of the Connected Home Division.  So he most likely
still works on this stuff, just under a different banner.

> If someone knows how to contact the author, that would be welcome.

Alas, no idea about his MaxLinear address.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ in the beginning was the boot and root floppies and they were good.
⢿⡄⠘⠷⠚⠋⠀   --  on #linux-sunxi
⠈⠳⣄

Re: TLS for 5.10

[Adam Borowski]

On Mon, Feb 08, 2021 at 07:08:05AM +, Wadepohl, Wolfram wrote:
> I'm sad to hear that 5.10 has still an EOL of Dec, 2022. We are in
> development of our 1st GNU/Linux based System,  50k devices for IoT.
[...]
> In general a 2 year support for embedded systems in automation is not a
> reasonable thing. Nevertheless the CIP project has commited to 5.10 as the
> next SLTS kernel.
[...]
> What can we do as a very small development team to support an extended LTS
> period of 5.10.?

The two years are a minimal promise from Greg.  Debian is using 5.10 for
upcoming Bullseye, thus even if Greg won't extend (99% chance he will), Ben
Hutchings will continue the support for as long as Debian Bullseye is alive.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ A dumb species has no way to open a tuna can.
⢿⡄⠘⠷⠚⠋⠀ A smart species invents a can opener.
⠈⠳⣄ A master species delegates.

Re: 5.10 LTS Kernel: 2 or 6 years?

[Adam Borowski]

On Mon, Jan 25, 2021 at 11:55:11AM -0800, Scott Branden wrote:
> The 5.10 LTS kernel being officially LTS supported for 2 years presents a 
> problem:
> why would anyone select a 5.10 kernel with 2 year LTS when 5.4 kernel has a 6 
> year LTS.
> 
> Yet, various unofficial reports indicate it will be supported for 6
> years.  And AOSP has already declared the use of 5.10 kernel in their
> Android S and T releases.

5.10 will also be used for Debian Bullseye.

-- 
⢀⣴⠾⠻⢶⣦⠀ .--[ Makefile ]
⣾⠁⢠⠒⠀⣿⡁ # beware of races
⢿⡄⠘⠷⠚⠋⠀ all: pillage burn
⠈⠳⣄ `

Re: Linux 5.11-rc1

[Adam Borowski]

On Sun, Jan 03, 2021 at 05:45:16PM +, David Laight wrote:
> From: Ilkka Prusi
> > Note that /sbin is now just a symlink to /usr/sbin on Debian since 10 
> > (Buster) as per FHS[1][2].
> > 
> > [1] https://wiki.linuxfoundation.org/lsb/fhs
> > [2] 
> > https://arstechnica.com/information-technology/2019/09/debian-10-playing-catch-up-with-the-rest-
> > of-the-linux-world-thats-a-good-thing/
> 
> Which is exactly 100% backwards :-)

I agree with you that, if all, it's /usr/sbin which should be a symlink,
to reduce typing and to get rid of a vestige of _The_ Unix machine having
/bin spill to a disk that used to have user homes.

But, /sbin is a symlink on only _some_ Debian installations.  There's
currently a discussion whether to go deeper into this scheme, abandon it
or do nothing.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ .--[ Makefile ]
⣾⠁⢠⠒⠀⣿⡁ # beware of races
⢿⡄⠘⠷⠚⠋⠀ all: pillage burn
⠈⠳⣄ `

Re: [PATCH] tty: Remove dead termiox code

[Adam Borowski]

On Fri, Dec 04, 2020 at 10:10:16AM +0100, Greg Kroah-Hartman wrote:
> On Fri, Dec 04, 2020 at 09:51:07AM +0100, Jiri Slaby wrote:
> > > > > On Fri, Dec 04, 2020 at 08:22:41AM +0100, Jiri Slaby wrote:
> > > > > > On 03. 12. 20, 3:03, Jann Horn wrote:
> > > > > > > Delete this dead code; but leave the definition of struct termiox 
> > > > > > > in the
> > > > > > > UAPI headers intact.
[was snipped]
> > > > > > I am thinking -- can/should we mark the structure as deprecated so 
> > > > > > that 
> > > > > > userspace stops using it eventually?   

> > Note this ^. He is talking about _not_ touching the definition in the
> > UAPI header. Does the rest below makes more sense now?
> 
> No, I'm still confused :)
> 
> We can't touch the UAPI definitions, but the fact that this api never
> did anything still is ok as after this patch it continues to not do
> anything.
> 
> I'm confused as to what you are proposing...

The UAPI definition can't be removed, but it would be nice to issue a
compiler _warning_ if it's ever used.

Like eg. __attribute__ ((deprecated))


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀.--[ Makefile ]
⣾⠁⢠⠒⠀⣿⡁# beware of races
⢿⡄⠘⠷⠚⠋⠀all: pillage burn
⠈⠳⣄`

Re: [PATCH 2/2] printk: Make the console flush configurable in hotplug path

[Adam Borowski]

On Fri, Sep 25, 2020 at 11:27:54AM +0200, Greg KH wrote:
> On Thu, Sep 24, 2020 at 08:21:07PM +0200, Thomas Gleixner wrote:
> > On Thu, Sep 24 2020 at 08:33, Greg KH wrote:
> > > On Wed, Sep 23, 2020 at 05:08:32PM -0700, Prasad Sodagudi wrote:
> > >> +config CONSOLE_FLUSH_ON_HOTPLUG
> > >> +bool "Enable console flush configurable in hot plug code path"

> > CPU hotplug is not meant to be a high speed operation and if people
> > think they need it to be fast then its pretty much guaranteed that they
> > want it for the completely wrong reasons.
> 
> Odds are, it's the big/little systems that are trying to use cpu hotplug
> for this type of thing :(

Just a bit of info:
My MT6797X (10 core: 4×A53 + 4×A53 + 2×A72), flickers its cores this way:
the right-hand piece is CPUs, one character per core: bars show utilization,
"o" stands for offline; every line is 0.1 second interval.

topline -i 0.1
mmcblk(⠀) (▄▆oo▅o)
mmcblk(⠀) (▅▄)
mmcblk(⠀) (▆▆)
mmcblk(⠀) (▅ooo▆o)
mmcblk(⠀) (▆▆oo▄o)
mmcblk(⠀) (▆▇)
mmcblk(⠀) (▇ooo▅o)
mmcblk(⠀) (▆ooo█o)
mmcblk(⠀) (▆ooo▄o)
mmcblk(⠀) (▅ooo▆o)
mmcblk(⠀) (▆ooo▅o)
mmcblk(⠀) (▄ooo▇o)
mmcblk(⠀) (▇▆oo▆o)
mmcblk(⠀) (▆ooo▅o)
mmcblk(⠀) (▅▆)
mmcblk(⠀) (▆█)
mmcblk(⠀) (▆▇)
mmcblk(⠀) (▆▆)
mmcblk(⠀) (▅▆)
mmcblk(⠀) (▆▅)
mmcblk(⠀) (▆ooo▆o)
mmcblk(⠀) (▆ooo▇o)
mmcblk(⢀) (▆▇oo▆o)
mmcblk(⠀) (▄ooo▆o)
mmcblk(⠀) (▆ooo█o)
mmcblk(⠀) (▄ooo▇o)
mmcblk(⠀) (▄▆)
mmcblk(⠀) (▆▆)

So it's on the order of a few ons/offs per second.

The offline CPUs are "present" and "offline"; not sure if this means hotplug
or not (I'd expect dropping from "present" to "possible", but I don't know
these parts).


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ in the beginning was the boot and root floppies and they were good.
⢿⡄⠘⠷⠚⠋⠀   --  on #linux-sunxi
⠈⠳⣄

Re: fbcon: remove soft scrollback code (missing Doc. patch)

[Adam Borowski]

On Wed, Sep 16, 2020 at 10:54:34PM +0200, Pavel Machek wrote:
> On Mon 2020-09-14 18:28:34, Linus Torvalds wrote:
> > Note that scrollback hasn't actually gone away entirely - the original
> > scrollback supported by _hardware_ still exists.
> > 
> > Of course, that's really just the old-fashioned text VGA console, but
> > that one actually scrolls not by moving any bytes around, but by
> > moving the screen start address. And the scrollback similarly isn't
> > about any software buffering, but about the ability of moving back
> > that screen start address.

> Could we pause this madness? Scrollback is still useful. I needed it
> today... it was too small, so command results I was looking for
> already scrolled away, but... life will be really painful with 0 scrollback.
> 
> You'll need it, too... as soon as you get oops and will want to see
> errors just prior to that oops.

I concur -- this a serious usability regression for regular users.  Linus:
you have a serial cable on your main dev machine, so do I, but hardly any
regular people do -- that's restricted to mostly IPMI and such.

And without some kind of scrollback, there's no way of knowing why eg.
your rootfs failed to mount (there was some oops, but its reason was at
the beginning...).  Or, any other problem the user would be able to solve,
or pass the error messages to someone more knowledgeable.

I also wonder why did you choose to remove softscrollback which is actually
useful, yet leave hardscrollback which doesn't come to use on any
non-ancient hardware:
* on !x86 there's no vgacon at all
* on x86, in-tree drivers for GPUs by Intel, nVidia and AMD (others are
  dead) default to switching away from vgacon
* EFI wants its own earlycon
... thus, the only niche left is nVidia proprietary drivers which, the last
time I looked, still used CGA text mode.

> If it means I get to maintain it... I'm not happy about it but that's
> better than no scrollback.

That'd be greatly appreciated.  There are also some simplifications/rewrites
that could be done, like getting rid of redundant 1-byte/4-byte storage (or
even the code for 1-byte...).  Hard scrollback could be axed altogether (it
provides only a small amount of scroll).  Etc...

>  Kernel is now very verbose, so important messages
>  during bootup scroll away. It is way bigger deal when you can no
>  longer get to them using shift-pageup.

Thus hard scrollback is inadequate in the rare cases it's even present.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ in the beginning was the boot and root floppies and they were good.
⢿⡄⠘⠷⠚⠋⠀   --  on #linux-sunxi
⠈⠳⣄

build failure: aicasm: renamed yaccage

[Adam Borowski]

Hi!
My randconfig builds notoriously fail on this:

[~/linux/drivers/scsi/aic7xxx/aicasm](vanilla)$ make -j1
bison -d -b aicasm_gram aicasm_gram.y
mv aicasm_gram.tab.c .//aicasm_gram.c
mv aicasm_gram.tab.h .//aicasm_gram.h
bison -d -b aicasm_macro_gram -p mm aicasm_macro_gram.y
mv aicasm_macro_gram.tab.c .//aicasm_macro_gram.c
mv aicasm_macro_gram.tab.h .//aicasm_macro_gram.h
flex  -o aicasm_scan.c aicasm_scan.l
flex  -Pmm -o aicasm_macro_scan.c aicasm_macro_scan.l
cc -I/usr/include -I. -I./ aicasm.c aicasm_symbol.c .//aicasm_gram.c 
.//aicasm_macro_gram.c .//aicasm_scan.c .//aicasm_macro_scan.c -o .//aicasm -ldb
aicasm_symbol.c: In function ‘aic_print_reg_dump_end’:
aicasm_symbol.c:393:13: warning: implicit declaration of function ‘tolower’ 
[-Wimplicit-function-declaration]
  393 |   *letter = tolower(*letter);
  | ^~~
aicasm_gram.tab.c:204:10: fatal error: aicasm_gram.tab.h: No such file or 
directory
compilation terminated.
aicasm_macro_gram.tab.c:167:10: fatal error: aicasm_macro_gram.tab.h: No such 
file or directory
compilation terminated.

And the generated yaccage has:
#include "aicasm_gram.tab.h"
which tries to refer to the just renamed file.

As the files in question are generated, with the filename coming from $YACC
rather than source, it'd take some after-processing with sed or a similar
hack.  Thus, instead of sending a patch, I thought it'd better to ask:
what the renames are for?


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁
⢿⡄⠘⠷⠚⠋⠀ It's time to migrate your Imaginary Protocol from version 4i to 6i.
⠈⠳⣄

Re: [PATCH] x86/pci: don't set acpi stuff if !CONFIG_ACPI

[Adam Borowski]

On Fri, Aug 21, 2020 at 10:13:25PM +0200, Thomas Gleixner wrote:
> On Thu, Aug 20 2020 at 14:53, Adam Borowski wrote:
> > Found by randconfig builds.
> >
> >  arch/x86/pci/intel_mid_pci.c | 2 ++
> >  arch/x86/pci/xen.c   | 2 ++

> > --- a/arch/x86/pci/intel_mid_pci.c
> > +++ b/arch/x86/pci/intel_mid_pci.c
> > @@ -299,8 +299,10 @@ int __init intel_mid_pci_init(void)
> > +#ifdef CONFIG_ACPI
> > /* Continue with standard init */
> > acpi_noirq_set();
> > +#endif

> If CONFIG_ACPI=n then acpi_noirq_set() is an empty stub inline. So I'm
> not sure what you are trying to solve here.
> 
> Ah, I see with CONFIG_ACPI=n linux/acpi.h does not include asm/acpi.h so
> the stubs are unreachable. So that needs to be fixed and not papered
> over with #ifdeffery

If I understand Randy Dunlap correctly, he already sent a pair of patches
that do what you want.


Meow.
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁
⢿⡄⠘⠷⠚⠋⠀ It's time to migrate your Imaginary Protocol from version 4i to 6i.
⠈⠳⣄

Re: [PATCH v10 4/8] usr: add support for zstd compressed initramfs

[Adam Borowski]

On Tue, Aug 04, 2020 at 09:25:23AM +0200, Sedat Dilek wrote:
> On Tue, Aug 4, 2020 at 8:52 AM Geert Uytterhoeven  
> wrote:
> > On Thu, Jul 30, 2020 at 9:13 PM Nick Terrell  wrote:
> > > From: Nick Terrell 
> > > * Add support for a zstd compressed initramfs.
> > > * Add compression for compressing built-in initramfs with zstd.

> > > --- a/usr/Kconfig
> > > +++ b/usr/Kconfig
> > > @@ -100,6 +100,15 @@ config RD_LZ4
> > >   Support loading of a LZ4 encoded initial ramdisk or cpio buffer
> > >   If unsure, say N.
> > >
> > > +config RD_ZSTD
> > > +   bool "Support initial ramdisk/ramfs compressed using ZSTD"
> > > +   default y
> > > +   depends on BLK_DEV_INITRD
> > > +   select DECOMPRESS_ZSTD
> > > +   help
> > > + Support loading of a ZSTD encoded initial ramdisk or cpio 
> > > buffer.
> > > + If unsure, say N.
> >
> > I'm aware you copied this from the other entries, but IMHO "default y",
> > and "If unsure, say N" are not a good combination.

> you are right - for new stuff it should be "default n".

It got already applied to Linus' tree with "y", and I think it'd be nice
to have it as a default.  Let's disable other compressors instead.

On the other hand, having an unsupported rd compressor results in a boot
failure that's not immediately obvious, so that's a reason for keeping
the setting as "y".

On the third hand, distributions default to either gz or xz, thus I'd say:
* let's have gz xz zstd default to y, all others to n
* drop bzip2 lzma1 completely
* distros can't switch the mkinitramfs default yet, but if RD_ZSTD=y now,
  they'll be able to once they drop support for old kernels in a few years

> What I am missing - still - is a note - that your user-space should
> have the correct bits to support zstd-initramfs.
> Unsure where to place such an information.

Looks like INITRAMFS_COMPRESSION_* have lengthy prose but are not shown in
menuconfig, while RD_*, with no such prose, are shown.

The prose itself is grossly obsolete, too.  I have some updates in:
https://github.com/kilobyte/linux/commits/nobz2-v3
but that patchset needs rebasing and refreshing.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁
⢿⡄⠘⠷⠚⠋⠀ It's time to migrate your Imaginary Protocol from version 4i to 6i.
⠈⠳⣄

Re: [PATCH v8 6/7] x86: Add support for ZSTD compressed kernel

[Adam Borowski]

On Fri, Jul 24, 2020 at 02:26:40PM +0200, Ingo Molnar wrote:
> > -#ifdef CONFIG_KERNEL_BZIP2
> > +#if defined(CONFIG_KERNEL_BZIP2)
> >  # define BOOT_HEAP_SIZE0x40
> > -#else /* !CONFIG_KERNEL_BZIP2 */
> > +#elif defined(CONFIG_KERNEL_ZSTD)
> > +# define BOOT_HEAP_SIZE 0x3
> > +#else
> >  # define BOOT_HEAP_SIZE 0x1
> >  #endif
> 
> So the other patches explain why the decompression buffer extra space 
> was increased from 64k to 128k, but is there a similar 
> calculation/estimate for bumping BOOT_HEAD_SIZE from 64k to 192k?
> 
> Admittedly the BZ2 exception doesn't set a good example, but maybe we 
> can do this for ZSTD?

By the way, I have a patchset on top of this, to drop BZ2 and LZMA(1)
support, that should clean up this code somewhat.  And bring a lot of
lines of Linus happiness, as both bzip2 and lzma code are not used by
anything else in the kernel, unlike lzma2 (xz).

If you draw a speed-vs-size graph, at no point bzip2 or lzma are a good
choice, while zstd wins by a large margin for most of the range.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁
⢿⡄⠘⠷⠚⠋⠀ It's time to migrate your Imaginary Protocol from version 4i to 6i.
⠈⠳⣄

Re: [PATCH] kbuild: fix broken builds because of GZIP,BZIP2,LZOP variables

[Adam Borowski]

On Mon, Jun 08, 2020 at 12:59:44PM +0300, Denis Efremov wrote:
> Redefine GZIP, BZIP2, LZOP variables as KGZIP, KBZIP2, KLZOP resp.
> GZIP, BZIP2, LZOP env variables are reserved by the tools. The original
> attempt to redefine them internally doesn't work in makefiles/scripts
> intercall scenarios, e.g., "make GZIP=gzip bindeb-pkg" and results in
> broken builds. There can be other broken build commands because of this,
> so the universal solution is to use non-reserved env variables for the
> compression tools.
> 
> Fixes: 8dfb61dcbace ("kbuild: add variables for compression tools")

Same said my bisect before I noticed your fix. :)

> Signed-off-by: Denis Efremov 

However, I run just basic "make bindeb-pkg" without forcing any variables,
thus the commit message is wrong.

Yet, your patch fixes the functionality.  Thanks!

> ---
>  Makefile  | 24 +---
>  arch/arm/boot/deflate_xip_data.sh |  2 +-
>  arch/ia64/Makefile|  2 +-
>  arch/m68k/Makefile|  8 
>  arch/parisc/Makefile  |  2 +-
>  scripts/Makefile.lib  |  6 +++---
>  scripts/Makefile.package  |  6 +++---
>  scripts/package/buildtar  |  4 ++--
>  8 files changed, 20 insertions(+), 34 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index 839f9fee22cb..e43d193bb3b2 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -458,27 +458,13 @@ PYTHON  = python
>  PYTHON3  = python3
>  CHECK= sparse
>  BASH = bash
> -GZIP = gzip
> -BZIP2= bzip2
> -LZOP = lzop
> +KGZIP= gzip
> +KBZIP2   = bzip2
> +KLZOP= lzop
>  LZMA = lzma
>  LZ4  = lz4c
>  XZ   = xz
>  
> -# GZIP, BZIP2, LZOP env vars are used by the tools. Support them as the 
> command
> -# line interface, but use _GZIP, _BZIP2, _LZOP internally.
> -_GZIP  := $(GZIP)
> -_BZIP2 := $(BZIP2)
> -_LZOP  := $(LZOP)
> -
> -# Reset GZIP, BZIP2, LZOP in this Makefile
> -override GZIP=
> -override BZIP2=
> -override LZOP=
> -
> -# Reset GZIP, BZIP2, LZOP in recursive invocations
> -MAKEOVERRIDES += GZIP= BZIP2= LZOP=
> -
>  CHECKFLAGS := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ \
> -Wbitwise -Wno-return-void -Wno-unknown-attribute $(CF)
>  NOSTDINC_FLAGS :=
> @@ -526,7 +512,7 @@ CLANG_FLAGS :=
>  export ARCH SRCARCH CONFIG_SHELL BASH HOSTCC KBUILD_HOSTCFLAGS CROSS_COMPILE 
> LD CC
>  export CPP AR NM STRIP OBJCOPY OBJDUMP OBJSIZE READELF PAHOLE LEX YACC AWK 
> INSTALLKERNEL
>  export PERL PYTHON PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX
> -export _GZIP _BZIP2 _LZOP LZMA LZ4 XZ
> +export KGZIP KBZIP2 KLZOP LZMA LZ4 XZ
>  export KBUILD_HOSTCXXFLAGS KBUILD_HOSTLDFLAGS KBUILD_HOSTLDLIBS 
> LDFLAGS_MODULE
>  
>  export KBUILD_CPPFLAGS NOSTDINC_FLAGS LINUXINCLUDE OBJCOPYFLAGS 
> KBUILD_LDFLAGS
> @@ -1047,7 +1033,7 @@ export mod_strip_cmd
>  mod_compress_cmd = true
>  ifdef CONFIG_MODULE_COMPRESS
>ifdef CONFIG_MODULE_COMPRESS_GZIP
> -mod_compress_cmd = $(_GZIP) -n -f
> +mod_compress_cmd = $(KGZIP) -n -f
>endif # CONFIG_MODULE_COMPRESS_GZIP
>ifdef CONFIG_MODULE_COMPRESS_XZ
>  mod_compress_cmd = $(XZ) -f
> diff --git a/arch/arm/boot/deflate_xip_data.sh 
> b/arch/arm/boot/deflate_xip_data.sh
> index 739f0464321e..304495c3c2c5 100755
> --- a/arch/arm/boot/deflate_xip_data.sh
> +++ b/arch/arm/boot/deflate_xip_data.sh
> @@ -56,7 +56,7 @@ trap 'rm -f "$XIPIMAGE.tmp"; exit 1' 1 2 3
>  # substitute the data section by a compressed version
>  $DD if="$XIPIMAGE" count=$data_start iflag=count_bytes of="$XIPIMAGE.tmp"
>  $DD if="$XIPIMAGE"  skip=$data_start iflag=skip_bytes |
> -$_GZIP -9 >> "$XIPIMAGE.tmp"
> +$KGZIP -9 >> "$XIPIMAGE.tmp"
>  
>  # replace kernel binary
>  mv -f "$XIPIMAGE.tmp" "$XIPIMAGE"
> diff --git a/arch/ia64/Makefile b/arch/ia64/Makefile
> index f817f3d5e758..2876a7df1b0a 100644
> --- a/arch/ia64/Makefile
> +++ b/arch/ia64/Makefile
> @@ -40,7 +40,7 @@ $(error Sorry, you need a newer version of the assember, 
> one that is built from
>  endif
>  
>  quiet_cmd_gzip = GZIP$@
> -cmd_gzip = cat $(real-prereqs) | $(_GZIP) -n -f -9 > $@
> +cmd_gzip = cat $(real-prereqs) | $(KGZIP) -n -f -9 > $@
>  
>  quiet_cmd_objcopy = OBJCOPY $@
>  cmd_objcopy = $(OBJCOPY) $(OBJCOPYFLAGS) $(OBJCOPYFLAGS_$(@F)) $< $@
> diff --git a/arch/m68k/Makefile b/arch/m68k/Makefile
> index ce6db5e5a5a3..0415d28dbe4f 100644
> --- a/arch/m68k/Makefile
> +++ b/arch/m68k/Makefile
> @@ -135,10 +135,10 @@ vmlinux.gz: vmlinux
>  ifndef CONFIG_KGDB
>   cp vmlinux vmlinux.tmp
>   $(STRIP) vmlinux.tmp
> - $(_GZIP) -9c vmlinux.tmp >vmlinux.gz
> + $(KGZIP) -9c vmlinux.tmp >vmlinux.gz
>   rm vmlinux.tmp
>  else
> - $(_GZIP) -9c vmlinux >vmlinux.gz
> + $(KGZIP) -9c vmlinux >vmlinux.gz
>  endif
>  
>  bzImage: vmlinux.bz2
> @@ -148,10 +148,10 @@ vmlinux.bz2: vmlinux
>  ifndef CONFIG_KGDB

Re: boot failure: stack-protector: Kernel stack is corrupted in: start_secondary

[Adam Borowski]

On Tue, Apr 28, 2020 at 09:45:27AM +0200, Borislav Petkov wrote:
> On Tue, Apr 21, 2020 at 03:32:34AM +0200, Adam Borowski wrote:
> > Hi!
> > With kernels compiled with gcc-10, on two different machines (AMD Phenom2,
> > AMD 2990WX) I get the following panic during boot:
> 
> Welcome to the party:
> 
> https://git.kernel.org/tip/f670269a42bfdd2c83a1118cc3d1b475547eac22
> 
> Try this branch to check whether it works for ya:
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git/log/?h=x86/build

✓


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ in the beginning was the boot and root floppies and they were good.
⢿⡄⠘⠷⠚⠋⠀   --  on #linux-sunxi
⠈⠳⣄

Re: Stop breaking the CSRNG

[Adam Borowski]

On Thu, Oct 03, 2019 at 10:13:39AM +, David Laight wrote:
> From: Kurt Roeckx
> > Sent: 02 October 2019 17:56
> > As OpenSSL, we want cryptograhic secure random numbers. Before
> > getrandom(), Linux never provided a good API for that, both
> > /dev/random and /dev/urandom have problems. getrandom() fixed
> > that, so we switched to it were available.
> 
> The fundamental problem is that you can't always get ' cryptograhic secure
> random numbers'. No API changes are ever going to change that.
> 
> The system can either return an error or sleep (possibly indefinitely)
> until some 'reasonably random' numbers are available.
> 
> A RISC-V system running on an FGPA (I've only used Altera NIOS cpu)
> may have absolutely no sources of randomness at boot time.

I'd say this is a hardware security vulnerability; no different from eg.
having no or faulty MMU, speculation that allows exfiltrating data, etc.
We did not understand the seriousness of lacking hardware sources of
randomness, but that's a common thing to many other security
vulnerabilities.

Machines that lack any sources of entropy have their uses, but they're akin
to processors with no MMU.  You should never run a world-accessible ssh
daemon on either of them.

> Saying the architecture must include a random number instruction
> doesn't help!

It won't fix existing systems, and is irrelevant to deeply embedded, but
communicating this requirement to SoC designers sounds like a good idea to
me.  IoTrash appliance makers won't care but their security is already so
atrocious that lack of entropy is nowhere near the easiest way to get in,
while anyone else will at least notice the warning.

Any real-silicon hardware can include an entropy source, and if it doesn't,
shaming the maker is the way to go.  Calling the problem a security
vulnerability (which I say it is) sends a stronger message.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ A MAP07 (Dead Simple) raspberry tincture recipe: 0.5l 95% alcohol,
⣾⠁⢠⠒⠀⣿⡁ 1kg raspberries, 0.4kg sugar; put into a big jar for 1 month.
⢿⡄⠘⠷⠚⠋⠀ Filter out and throw away the fruits (can dump them into a cake,
⠈⠳⣄ etc), let the drink age at least 3-6 months.

[PATCH] Documentation: sysrq: don't recommend 'S' 'U' before 'B'

[Adam Borowski]

This advice is obsolete and slightly harmful for filesystems from this
millenium: any modern filesystem can handle unexpected crashes without
requiring fsck -- and on the other hand, trying to write to the disk when
the kernel is in a bad state risks introducing corruption.

For ext2, any unsafe shutdown meant widespread breakage, but it's no longer
a reasonable filesystem for any non-special use.

Signed-off-by: Adam Borowski 
---
 Documentation/admin-guide/sysrq.rst | 20 +---
 1 file changed, 9 insertions(+), 11 deletions(-)

diff --git a/Documentation/admin-guide/sysrq.rst 
b/Documentation/admin-guide/sysrq.rst
index 7b9035c01a2e..72b2cfb066f4 100644
--- a/Documentation/admin-guide/sysrq.rst
+++ b/Documentation/admin-guide/sysrq.rst
@@ -171,22 +171,20 @@ It seems others find it useful as (System Attention Key) 
which is
 useful when you want to exit a program that will not let you switch consoles.
 (For example, X or a svgalib program.)
 
-``reboot(b)`` is good when you're unable to shut down. But you should also
-``sync(s)`` and ``umount(u)`` first.
+``reboot(b)`` is good when you're unable to shut down, it is an equivalent
+of pressing the "reset" button.
 
 ``crash(c)`` can be used to manually trigger a crashdump when the system is 
hung.
 Note that this just triggers a crash if there is no dump mechanism available.
 
-``sync(s)`` is great when your system is locked up, it allows you to sync your
-disks and will certainly lessen the chance of data loss and fscking. Note
-that the sync hasn't taken place until you see the "OK" and "Done" appear
-on the screen. (If the kernel is really in strife, you may not ever get the
-OK or Done message...)
+``sync(s)`` is handy before yanking removable medium or after using a rescue
+shell that provides no graceful shutdown -- it will ensure your data is
+safely written to the disk. Note that the sync hasn't taken place until you see
+the "OK" and "Done" appear on the screen.
 
-``umount(u)`` is basically useful in the same ways as ``sync(s)``. I generally
-``sync(s)``, ``umount(u)``, then ``reboot(b)`` when my system locks. It's saved
-me many a fsck. Again, the unmount (remount read-only) hasn't taken place until
-you see the "OK" and "Done" message appear on the screen.
+``umount(u)`` can be used to mark filesystems as properly unmounted. From the
+running system's point of view, they will be remounted read-only. The remount
+isn't complete until you see the "OK" and "Done" message appear on the screen.
 
 The loglevels ``0``-``9`` are useful when your console is being flooded with
 kernel messages you do not want to see. Selecting ``0`` will prevent all but
-- 
2.23.0

Re: [PATCH v3 0/8] AMD64 EDAC fixes

[Adam Borowski]

On Thu, Aug 22, 2019 at 10:35:48AM +0200, Borislav Petkov wrote:
> On Thu, Aug 22, 2019 at 02:50:20AM +0200, Adam Borowski wrote:
> > While you're editing that code, could you please also cut the spam if ECC is
> > actually disabled?  For example, a 2990WX with non-ECC RAM gets 1024 lines;
> 
> Patch is in there. I'll give you extra points if you spot it.

Yeah, some of messages are no longer emitted for memory-less nodes (NUMA 1
and 3).  Your patch set also overhauls the messages.

But, the amount of redundant messages I'm complaining about has actually
increased:

dmesg|grep EDAC|cut -c 16-|sort|uniq -c
256 EDAC MC: UMC0 chip selects:
256 EDAC MC: UMC1 chip selects:
  1 EDAC MC: Ver: 3.0.0
128 EDAC amd64: ECC disabled in the BIOS or no ECC capability, module will 
not load.
^ three lines each
 64 EDAC amd64: F17h detected (node 0).
 64 EDAC amd64: F17h detected (node 1).
 64 EDAC amd64: F17h detected (node 2).
 64 EDAC amd64: F17h detected (node 3).
512 EDAC amd64: MC: 0: 0MB 1: 0MB
256 EDAC amd64: MC: 2: 0MB 3: 0MB
256 EDAC amd64: MC: 2:  8192MB 3: 0MB
 64 EDAC amd64: Node 0: DRAM ECC disabled.
 64 EDAC amd64: Node 2: DRAM ECC disabled.
256 EDAC amd64: using x4 syndromes.

(Full dmesg at http://ix.io/1T1o)

While on 5.3-rc5 without the patchset I get:

  1 EDAC MC: Ver: 3.0.0
256 EDAC amd64: ECC disabled in the BIOS or no ECC capability, module will 
not load.
^ three lines each
 64 EDAC amd64: Node 0: DRAM ECC disabled.
 64 EDAC amd64: Node 1: DRAM ECC disabled.
 64 EDAC amd64: Node 2: DRAM ECC disabled.
 64 EDAC amd64: Node 3: DRAM ECC disabled.

So I wonder if we could deduplicate those.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢠⠒⠀⣿⡁ A dumb species has no way to open a tuna can.
⢿⡄⠘⠷⠚⠋⠀ A smart species invents a can opener.
⠈⠳⣄ A master species delegates.

Re: [PATCH v3 0/8] AMD64 EDAC fixes

[Adam Borowski]

On Wed, Aug 21, 2019 at 11:59:53PM +, Ghannam, Yazen wrote:
> I've also added RFC patches to avoid the "ECC disabled" message for
> nodes without memory. I haven't fully tested these, but I wanted to get
> your thoughts. Here's an earlier discussion:
> https://lkml.kernel.org/r/20180321191335.7832-1-yazen.ghan...@amd.com

While you're editing that code, could you please also cut the spam if ECC is
actually disabled?  For example, a 2990WX with non-ECC RAM gets 1024 lines;
64 copies of:

[8.186164] EDAC amd64: Node 0: DRAM ECC disabled.
[8.188364] EDAC amd64: ECC disabled in the BIOS or no ECC capability, 
module will not load.
Either enable ECC checking or force module loading by setting 
'ecc_enable_override'.
(Note that use of the override may cause unknown side effects.)
[8.194762] EDAC amd64: Node 1: DRAM ECC disabled.
[8.196307] EDAC amd64: ECC disabled in the BIOS or no ECC capability, 
module will not load.
Either enable ECC checking or force module loading by setting 
'ecc_enable_override'.
(Note that use of the override may cause unknown side effects.)
[8.199840] EDAC amd64: Node 2: DRAM ECC disabled.
[8.200963] EDAC amd64: ECC disabled in the BIOS or no ECC capability, 
module will not load.
Either enable ECC checking or force module loading by setting 
'ecc_enable_override'.
(Note that use of the override may cause unknown side effects.)
[8.204326] EDAC amd64: Node 3: DRAM ECC disabled.
[8.205436] EDAC amd64: ECC disabled in the BIOS or no ECC capability, 
module will not load.
Either enable ECC checking or force module loading by setting 
'ecc_enable_override'.
(Note that use of the override may cause unknown side effects.)


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁
⢿⡄⠘⠷⠚⠋  The root of a real enemy is an imaginary friend.
⠈⠳⣄

Re: Device to write to all (serial) consoles

[Adam Borowski]

On Sat, Aug 03, 2019 at 03:55:37PM +0200, Greg Kroah-Hartman wrote:
> On Sat, Aug 03, 2019 at 03:23:23PM +0200, Adam Borowski wrote:
> > On Fri, Aug 02, 2019 at 09:59:06PM +0200, Paul Menzel wrote:
> > > Because the cable is always connected to the port on the back side, and
> > > sometimes the port in the front has ID 0, and the one in the back 1, and
> > > other times vice versa. We do not want to track that, and it would be
> > > convenient to just write to both ports.
> > 
> > Sounds like an XY problem then: what you want is not writing to all ports,
> > but to have the port assignments stable (see also: disk device reordering).
> 
> You can get that information from the symlinks in /dev/serial/ which
> udev creates.

Doesn't seem to work for me, for any ttyS0 ttyS1 ttySAC1 device:

Box one, PCIe card + one USB dongle:
07:00.0 Serial controller: MosChip Semiconductor Technology Ltd. MCS9922 PCIe 
Multi-I/O Controller
07:00.1 Serial controller: MosChip Semiconductor Technology Ltd. MCS9922 PCIe 
Multi-I/O Controller
Bus 003 Device 004: ID 1a86:7523 QinHeng Electronics HL-340 USB-Serial adapter

/dev/serial/by-id/usb-1a86_USB2.0-Serial-if00-port0
/dev/serial/by-path/pci-:0b:00.3-usb-0:4:1.0-port0

Only ttyUSB0 is there.


Box two, on-board + one USB dongle:
[3.404340] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[3.431287] 00:01: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 
16550A
Bus 001 Device 002: ID 10c4:ea60 Cygnal Integrated Products, Inc. CP2102/CP2109 
UART Bridge Controller [CP210x family]

/dev/serial/by-id/usb-Silicon_Labs_CP2104_USB_to_UART_Bridge_Controller_00DB1604-if00-port0
/dev/serial/by-path/pci-:00:14.0-usb-0:1:1.0-port0


Box three: RockPro64, euler + USB dongle, kernel 4.4.
Box four: Pine64, euler.
Box five: Odroid-U2, something GPIOish (ttySAC1), kernel 5.0.


Most are running kernel 5.2, Debian unstable.

And indeed, in /lib/udev/rules.d/60-serial.rules :
# /dev/serial/by-path/, /dev/serial/by-id/ for USB devices
KERNEL!="ttyUSB[0-9]*|ttyACM[0-9]*", GOTO="serial_end"


Like me, Paul is using ttyS0 for server-side.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢰⠒⠀⣿⡁ 10 people enter a bar: 1 who understands binary,
⢿⡄⠘⠷⠚⠋⠀ 1 who doesn't, D who prefer to write it as hex,
⠈⠳⣄ and 1 who narrowly avoided an off-by-one error.

Re: Device to write to all (serial) consoles

[Adam Borowski]

On Fri, Aug 02, 2019 at 09:59:06PM +0200, Paul Menzel wrote:
> On 02.08.19 18:02, Greg Kroah-Hartman wrote:
> > On Fri, Aug 02, 2019 at 03:23:08PM +0200, Paul Menzel wrote:
> > > On a lot of devices, like servers, you have more than one serial console,
> > > and you do not always know, how they are numbered. Therefore, we start a
> > > console on ttyS0 and ttyS1.
> 
> Because the cable is always connected to the port on the back side, and
> sometimes the port in the front has ID 0, and the one in the back 1, and
> other times vice versa. We do not want to track that, and it would be
> convenient to just write to both ports.

Sounds like an XY problem then: what you want is not writing to all ports,
but to have the port assignments stable (see also: disk device reordering).


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ Latin:   meow 4 characters, 4 columns,  4 bytes
⣾⠁⢠⠒⠀⣿⡁ Greek:   μεου 4 characters, 4 columns,  8 bytes
⢿⡄⠘⠷⠚⠋  Runes:   ᛗᛖᛟᚹ 4 characters, 4 columns, 12 bytes
⠈⠳⣄ Chinese: 喵   1 character,  2 columns,  3 bytes <-- best!

Re: [RFC] remove arch/sh?

[Adam Borowski]

On Tue, Jun 25, 2019 at 11:02:36AM +0200, John Paul Adrian Glaubitz wrote:
> On 6/25/19 10:56 AM, Christoph Hellwig wrote:
> > arch/sh seems pretty much unmaintained these days.  The last time I got
> > any reply to sh patches from the list maintainers, and the last maintainer
> > pull request was over a year ago, and even that has been rather sporadic.
> > 
> > In the meantime we've not really seen any updates for new kernel features
> > and code seems to be bitrotting.
> 
> We're still using sh4 in Debian

I wouldn't call it "used": it has popcon of 1, and despite watching many
Debian channels, I don't recall hearing a word about sh4 in quite a while.

Hardware development is dead: we were promised modern silicon by j-core
after original patents expired, but after J2 nothing happened, there was
silence from their side, and now https://j-core.org is down.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Packager's rule #1: upstream _always_ screws something up.  This
⢿⡄⠘⠷⠚⠋⠀ is true especially if you're packaging your own project.
⠈⠳⣄ 

Re: [RESEND PATCH v2 0/2] Add support for ZSTD-compressed kernel

[Adam Borowski]

On Fri, Jun 07, 2019 at 07:20:46PM +, Nick Terrell wrote:
> We'd love to get this mainlined as well!
> 
> We're using these patches internally as well. We're seeing an improvement on 
> an
> Intel Atom N3710, where boot time is reduced by one second over using an xz
> compressed kernel. It looks like Ubuntu just switched to a lz4 compressed 
> kernel,
> but zstd is likely a better trade off, because it compresses much better and 
> still has
> excellent decompression speed.
> 
> Since its been nearly a year since I sent these out, I will take some time to 
> rebase
> and retest the patches in case anything changed, and then then resend the 
> patches
> in the next weeks.

Hi!
After the ping, I intended to resend the patch-set (with removals included)
after I return from miniDebconf Hamburg, but you 1. are the author of the
non-trivial part, 2. you have a better test machinery, and 3. I have a
deeply seated preference for effort to be done by people who are not me.

A rebased and working version is at 
https://github.com/kilobyte/linux/tree/nobz2-v3
but there are no real improvements beyond rebases, a typo fix, and Paul Burton's
ACK for mips.

There's an unaddressed comment by Ingo Molnar
https://lore.kernel.org/lkml/20181112042200.ga96...@gmail.com/
for your part of the code.

So what do you suggest?


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ Latin:   meow 4 characters, 4 columns,  4 bytes
⣾⠁⢠⠒⠀⣿⡁ Greek:   μεου 4 characters, 4 columns,  8 bytes
⢿⡄⠘⠷⠚⠋  Runes:   ᛗᛖᛟᚹ 4 characters, 4 columns, 12 bytes
⠈⠳⣄ Chinese: 喵   1 character,  2 columns,  3 bytes <-- best!

Re: [PATCH RFC 0/5] mm: process_vm_mmap() -- syscall for duplication a process mapping

[Adam Borowski]

On Thu, May 16, 2019 at 04:10:07PM +0300, Kirill Tkhai wrote:
> On 15.05.2019 22:38, Adam Borowski wrote:
> > On Wed, May 15, 2019 at 06:11:15PM +0300, Kirill Tkhai wrote:
> >> This patchset adds a new syscall, which makes possible
> >> to clone a mapping from a process to another process.
> >> The syscall supplements the functionality provided
> >> by process_vm_writev() and process_vm_readv() syscalls,
> >> and it may be useful in many situation.
> >>
> >> For example, it allows to make a zero copy of data,
> >> when process_vm_writev() was previously used:
> > 
> > I wonder, why not optimize the existing interfaces to do zero copy if
> > properly aligned?  No need for a new syscall, and old code would immediately
> > benefit.
> 
> Because, this is just not possible. You can't zero copy anonymous pages
> of a process to pages of a remote process, when they are different pages.

fork() manages that, and so does KSM.  Like KSM, you want to make a page
shared -- you just skip the comparison step as you want to overwrite the old
contents.

And there's no need to touch the page, as fork() manages that fine no matter
if the page is resident, anonymous in swap, or file-backed, all without
reading from swap.

> >> There are several problems with process_vm_writev() in this example:
> >>
> >> 1)it causes pagefault on remote process memory, and it forces
> >>   allocation of a new page (if was not preallocated);
> >>
> >> 2)amount of memory for this example is doubled in a moment --
> >>   n pages in current and n pages in remote tasks are occupied
> >>   at the same time;
> >>
> >> 3)received data has no a chance to be properly swapped for
> >>   a long time.
> > 
> > That'll handle all of your above problems, except for making pages
> > subject to CoW if written to.  But if making pages writeably shared is
> > desired, the old functions have a "flags" argument that doesn't yet have a
> > single bit defined.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ Latin:   meow 4 characters, 4 columns,  4 bytes
⣾⠁⢠⠒⠀⣿⡁ Greek:   μεου 4 characters, 4 columns,  8 bytes
⢿⡄⠘⠷⠚⠋  Runes:   ᛗᛖᛟᚹ 4 characters, 4 columns, 12 bytes
⠈⠳⣄ Chinese: 喵   1 character,  2 columns,  3 bytes <-- best!

Re: [PATCH RFC 0/5] mm: process_vm_mmap() -- syscall for duplication a process mapping

[Adam Borowski]

On Wed, May 15, 2019 at 06:11:15PM +0300, Kirill Tkhai wrote:
> This patchset adds a new syscall, which makes possible
> to clone a mapping from a process to another process.
> The syscall supplements the functionality provided
> by process_vm_writev() and process_vm_readv() syscalls,
> and it may be useful in many situation.
> 
> For example, it allows to make a zero copy of data,
> when process_vm_writev() was previously used:

I wonder, why not optimize the existing interfaces to do zero copy if
properly aligned?  No need for a new syscall, and old code would immediately
benefit.

> There are several problems with process_vm_writev() in this example:
> 
> 1)it causes pagefault on remote process memory, and it forces
>   allocation of a new page (if was not preallocated);
> 
> 2)amount of memory for this example is doubled in a moment --
>   n pages in current and n pages in remote tasks are occupied
>   at the same time;
> 
> 3)received data has no a chance to be properly swapped for
>   a long time.

That'll handle all of your above problems, except for making pages
subject to CoW if written to.  But if making pages writeably shared is
desired, the old functions have a "flags" argument that doesn't yet have a
single bit defined.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ Latin:   meow 4 characters, 4 columns,  4 bytes
⣾⠁⢠⠒⠀⣿⡁ Greek:   μεου 4 characters, 4 columns,  8 bytes
⢿⡄⠘⠷⠚⠋  Runes:   ᛗᛖᛟᚹ 4 characters, 4 columns, 12 bytes
⠈⠳⣄ Chinese: 喵   1 character,  2 columns,  3 bytes <-- best!

Re: [PATCH RESEND v5 0/5] namei: vfs flags to restrict path resolution

[Adam Borowski]

On Thu, Apr 25, 2019 at 01:38:06AM +1000, Aleksa Sarai wrote:
>  * openat(2) ignores unknown flags, meaning that old kernels will ignore
>new programs trying to use O_THISROOT and might end up causing
>security issues. Yes, it'd be trivial to check whether the new O_*
>flags are supported at start-up, but I think a security feature
>shouldn't have a foot-gun associated with it. In fact, I didn't know
>openat(2) ignored unknown flags until I wrote this patchset -- I
>doubt many other userspace developers do either.

For this reason, I propose every new syscall that has flags to follow a
bitmask scheme, where any flag assigned a bit in the upper half returns
EOPNOTSUPP when called on an old kernel.  That would allow defining which
flags can be safely ignored and which can't.

It otherwise takes major hacks to implement a fail-if-not-supported flag
while keeping compat with old kernels.  For example, for mmap(), MAP_SHARED
has been duplicated as MAP_SHARED_VALIDATE just to allow an unrelated flag
(MAP_SYNC) to fail on old kernels.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢰⠒⠀⣿⡁ Imagine there are bandits in your house, your kid is bleeding out,
⢿⡄⠘⠷⠚⠋⠀ the house is on fire, and seven giant trumpets are playing in the
⠈⠳⣄ sky.  Your cat demands food.  The priority should be obvious...

Re: How to turn off IPv4 without disabling IPv6

[Adam Borowski]

On Thu, Apr 25, 2019 at 11:32:38AM +0200, Nico Schottelius wrote:
> running some IPv6 only
> networks. The systems in the IPv6 only networks do not need any IPv4
> support anymore and thus for switches/routers we turned the support off.

> Today we tried to turn off IPv4 in the Linux kernel at compile time.
> But it seems that as soon as we turn off CONFIG_INET, CONFIG_IPV6 is
> automatically turned off as well.

Even if you don't want global nor even link-scope IPv4, way too many
programs assume that at least 127.0.0.1 (ie, lo) is working.  They can't be
reconfigured to use ::1 without patching and rebuilding.

> Coming back to my original question: is there a way or how would we turn
> off IPv4 support in the Linux kernel?

I believe this is not worth your time for today.

Just do what IPv6-haters do on stock modern distros: have no routes for the
other IP version configured; non-buggy programs will do the right thing.

This seems to work well.  Heck, I had a busy dev server with broken IPv4, I
didn't notice that for 1.5 years until I tried to pull something directly
from Github (which is still v4 only).

You're a network admin so you know far more than me wrt anything that goes
over the wire -- but as as a distro developer/user, I'd say there's a
considerable cost to have every of tens of thousands programs shipped by a
distro, and many more that are private to a company/university/etc, updated
to autodetect how to access "localhost" on a particular box.

That's an extra moving part where there was none before.  Complexity is bad.
Having the IPv4 stack built just for the lo interface simplifies things.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Did ya know that typing "test -j8" instead of "ctest -j8"
⢿⡄⠘⠷⠚⠋⠀ will make your testsuite pass much faster, and fix bugs?
⠈⠳⣄

Re: Staging status of speakup

[Adam Borowski]

On Tue, Mar 19, 2019 at 04:31:21PM +, Alan Cox wrote:
> On Sat, 16 Mar 2019 10:35:43 +0100
> Samuel Thibault  wrote:
> > Chris Brannon, le ven. 15 mars 2019 18:19:39 -0700, a ecrit:
> > > What kind of reproducer do you need here?  It's straightforward to
> > > reproduce in casual use, at least with a software synthesizer.  
> > 
> > we need a walk-through of the kind of operation that
> > produces the issue. It does not have to be reproducible each time it is
> > done. Perhaps (I really don't know what that bug is about actually) it
> > is a matter of putting text in the selection buffer, and try to paste it
> > 100 times, and once every 10 times it will be garbled, for instance.
> 
> paste_selection still says
> 
> /* Insert the contents of the selection buffer into the
>  * queue of the tty associated with the current console.
>  * Invoked by ioctl().
>  *
>  * Locking: called without locks. Calls the ldisc wrongly with
>  * unsafe methods,
>  */
> 
> from which I deduce that with everyone using X nobody ever bothered to
> fix it. So before you look too hard at the speakup code you might want to
> review the interaction with selection.c too.

This looks like https://bugs.debian.org/849474 which causes a lockup, and
for which Bill Allombert wrote a nice reproducer.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Did ya know that typing "test -j8" instead of "ctest -j8"
⢿⡄⠘⠷⠚⠋⠀ will make your testsuite pass much faster, and fix bugs?
⠈⠳⣄

Re: [PATCH] selftests/ftrace: Make the coloring POSIX compliant

[Adam Borowski]

On Wed, Feb 20, 2019 at 09:20:20PM +0100, Juerg Haefliger wrote:
> On Wed, 20 Feb 2019 14:49:34 -0500
> Steven Rostedt  wrote:
> 
> > On Wed, 20 Feb 2019 17:13:33 +0100
> > Juerg Haefliger  wrote:
> > 
> > > echo -e and \e are not POSIX. Depending on what /bin/sh is, we can get
> > > incorrect output like:  
> > 
> > I'm curious to which shell this is.
> 
> Quite frankly I don't know but that's the output we get when we run it in
> Jenkins. I'll try to find out.

The only shell that did not support \e was dash -- I fixed it myself on
2017-01-24; thus I expect any Ubuntus prior to Zesty to require \033.  This
means your Jenkins likely runs Xenial.

\e has been supported since ages in at least: bash zsh mksh sash posh ksh
busybox:sh; also in perl python ruby lua php, gcc clang tcc -- MSVC being
the only other exception I know about.

Indeed POSIX doesn't specify \e, but as it pretends there are charsets other
than ASCII and EBCDIC, it can't.  There's no escape in its "portable
character set".


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁
⢿⡄⠘⠷⠚⠋⠀ Have you accepted Khorne as your lord and saviour?
⠈⠳⣄

Re: GRSec is vital to Linux security

[Adam Borowski]

On Thu, Jan 24, 2019 at 04:31:10PM +0100, Enrico Weigelt, metux IT consult 
wrote:
> On 23.01.19 21:46, Ivan Ivanov wrote:
> 
> > Linux really needs to stop adding new features and
> > refactor itself to a smaller and more secure codebase before going
> > forward. Maybe 1 year break would be nice.
> 
> Do you have some actual proposals / patches ?

Enrico, you're responding to a notorious troll.  If you haven't noticed,
this "Ivan Ivanov" sock puppet is a persona of some bastard who talks to
him/herself while tarnishing the name of our dear friend MikeeUSA (a true
pillar of the community!).  His/her methods evolve, but the gist is the
same.  Expect bringing up a bogus but semi-plausible controversy in order
to start as big a thread as possible, then once people who this bastard
wants to attack have joined, try to equate their position in the public view
with statements such as:

(Excuse the quotation, please wipe your monitor afterwards.)

# But from a man?
#
# Well, goes to show you. White men ain't men. Best they are is 40 year
# old bois. Faggots to say for short in American parlance.
#
# Same reason they won't hold it down when a bunch of fucking cunts CoC
# them. You build the whole edifice, then you let a bunch of do-nothing
# white women rule over the thing you built and you.

And this has been going for quite a while.

Connecting to systemd threads doesn't seem to work any longer, as people on
debian-user vs dng have wisened up.  Same with license rescinsion threads. 
What you read is just a yet another attempt to stir up some excrement.
Don't let any of it spray on you.  Because that's the fake-Mikee's way.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Remember, the S in "IoT" stands for Security, while P stands
⢿⡄⠘⠷⠚⠋⠀ for Privacy.
⠈⠳⣄

[PATCH] doc: process: GPL -> GPL-compatible

[Adam Borowski]

Drivers under MIT, BSD-17-clause, or uncle-Bob's-newest-take-on-PD are
all fine, not just GPL.

Signed-off-by: Adam Borowski 
---
Not reformatting to fill lines, it'll semi-conflict with another patch
that's been acked but not yet pushed.

 Documentation/process/stable-api-nonsense.rst | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Documentation/process/stable-api-nonsense.rst 
b/Documentation/process/stable-api-nonsense.rst
index 24f5aeecee91..9b69fb08b65e 100644
--- a/Documentation/process/stable-api-nonsense.rst
+++ b/Documentation/process/stable-api-nonsense.rst
@@ -170,7 +170,8 @@ nightmare, and trying to keep up with an ever changing 
kernel interface
 is also a rough job.
 
 Simple, get your kernel driver into the main kernel tree (remember we
-are talking about GPL released drivers here, if your code doesn't fall
+are talking about drivers released under a GPL-compatible license here,
+if your code doesn't fall
 under this category, good luck, you are on your own here, you leech
 .)  If your
 driver is in the tree, and a kernel interface changes, it will be fixed
-- 
2.20.1

Re: [PATCH] mm/mmu_notifier: mm/rmap.c: Fix a mmu_notifier range bug in try_to_unmap_one

[Adam Borowski]

On Wed, Jan 09, 2019 at 04:51:17PM -0800, Sean Christopherson wrote:
> Manifests as KVM use-after-free WARNINGs and subsequent "BUG: Bad page
> state in process X" errors when reclaiming from a KVM guest due to KVM
> removing the wrong pages from its own mappings.

With your patch, no badness happened so far.  Thanks!

> Reported-by: Adam Borowski 
> Fixes: ac46d4f3c432 ("mm/mmu_notifier: use structure for 
> invalidate_range_start/end calls v2")

> --- a/mm/rmap.c
> +++ b/mm/rmap.c
> - mmu_notifier_range_init(&range, vma->vm_mm, vma->vm_start,
> - min(vma->vm_end, vma->vm_start +
> + mmu_notifier_range_init(&range, vma->vm_mm, address,
> + min(vma->vm_end, address +


Meow.
-- 
⢀⣴⠾⠻⢶⣦⠀ Hans 1 was born and raised in Johannesburg, then moved to Boston,
⣾⠁⢠⠒⠀⣿⡁ and has just became a naturalized citizen.  Hans 2's grandparents
⢿⡄⠘⠷⠚⠋⠀ came from Melanesia to Düsseldorf, and he hasn't ever been outside
⠈⠳⣄ Germany until yesterday.  Which one is an African-American?

Re: 5.0-rc1 KVM inspired "BUG: Bad page state in process" spew

[Adam Borowski]

On Wed, Jan 09, 2019 at 06:38:58AM +0100, Mike Galbraith wrote:
> KVM seems to be busted in master ATM.  All I have to do to have host
> start screaming and maybe exploding (if the guest doesn't do so first)
> is to try to install a (obese in this case) kernel over nfs mount of
> the host in a guest.
> 
> Kernel producing the spew below is 3bd6e94, config attached.

I get same, except that the BUGs were preceded by a bunch of warnings,

> homer: # grep BUG: /netconsole.log
> [ 1531.909703] BUG: Bad page state in process X  pfn:100491
> [ 1531.958141] BUG: Bad page state in process systemd-journal  pfn:100412
> [ 1532.662359] BUG: Bad page state in process X  pfn:10043f
> [ 1532.664033] BUG: Bad page state in process X  pfn:10044d
> [ 1532.686433] BUG: Bad page state in process systemd-journal  pfn:1027b0

the first one being:

Jan  9 00:41:22 umbar kernel: [74122.790461] WARNING: CPU: 2 PID: 26769 at 
arch/x86/kvm/mmu.c:830 mmu_spte_clear_track_bits+0x7e/0x100
Jan  9 00:41:22 umbar kernel: [74122.799676] Modules linked in: tun dm_mod 
cdc_acm rndis_wlan rndis_host cdc_ether usbnet sha256_generic cfg80211 rfkill 
mii nfnetlink snd_usb_audio snd_usbmidi_lib cp210x usbserial radeon ttm 
pcc_cpufreq
Jan  9 00:41:22 umbar kernel: [74122.817764] CPU: 2 PID: 26769 Comm: 
qemu-system-x86 Not tainted 5.0.0-rc1-debug-00035-gdce22716f1b4 #1
Jan  9 00:41:22 umbar kernel: [74122.827069] Hardware name: System manufacturer 
System Product Name/M4A77T, BIOS 240105/18/2011
Jan  9 00:41:22 umbar kernel: [74122.836025] RIP: 
0010:mmu_spte_clear_track_bits+0x7e/0x100
Jan  9 00:41:22 umbar kernel: [74122.841528] Code: 48 89 e8 48 ba 00 00 00 00 
00 ea ff ff 48 c1 e0 06 48 01 d0 48 8b 50 08 48 8d 4a ff 83 e2 01 48 0f 45 c1 
8b 40 34 85 c0 75 02 <0f> 0b 48 0f ba e3 3e 73 34 48 85 1d d2 32 26 01 75 5a 48 
d1 eb 83
Jan  9 00:41:22 umbar kernel: [74122.860290] RSP: 0018:c900028634d0 EFLAGS: 
00010246
Jan  9 00:41:22 umbar kernel: [74122.865516] RAX:  RBX: 
00010198bc67 RCX: dead00ff
Jan  9 00:41:22 umbar kernel: [74122.872649] RDX:  RSI: 
 RDI: ea00040662c0
Jan  9 00:41:22 umbar kernel: [74122.879790] RBP: 0010198b R08: 
28f5c28f5c28f5c3 R09: 8882264dd000
Jan  9 00:41:22 umbar kernel: [74122.886912] R10: 88822fffa000 R11: 
88822fffa000 R12: 
Jan  9 00:41:22 umbar kernel: [74122.894046] R13: c90002863580 R14: 
 R15: c90002863580
Jan  9 00:41:22 umbar kernel: [74122.901170] FS:  7f19b9953700() 
GS:888227a8() knlGS:
Jan  9 00:41:22 umbar kernel: [74122.909249] CS:  0010 DS:  ES:  CR0: 
80050033
Jan  9 00:41:22 umbar kernel: [74122.914994] CR2: 102630e4 CR3: 
0001c80ee000 CR4: 06e0
Jan  9 00:41:22 umbar kernel: [74122.922135] Call Trace:
Jan  9 00:41:22 umbar kernel: [74122.924590]  drop_spte+0x1b/0xc0
Jan  9 00:41:22 umbar kernel: [74122.927822]  mmu_page_zap_pte+0xb2/0xe0
Jan  9 00:41:22 umbar kernel: [74122.931661]  
kvm_mmu_prepare_zap_page+0x4f/0x2b0
Jan  9 00:41:22 umbar kernel: [74122.936297]  mmu_shrink_scan+0x199/0x240
Jan  9 00:41:22 umbar kernel: [74122.940223]  do_shrink_slab+0x11e/0x1a0
Jan  9 00:41:22 umbar kernel: [74122.944054]  shrink_slab+0x220/0x2b0
Jan  9 00:41:22 umbar kernel: [74122.947632]  shrink_node+0x168/0x460
Jan  9 00:41:22 umbar kernel: [74122.951203]  do_try_to_free_pages+0xc1/0x370
Jan  9 00:41:22 umbar kernel: [74122.955467]  try_to_free_pages+0xb0/0xe0
Jan  9 00:41:22 umbar kernel: [74122.959385]  __alloc_pages_slowpath+0x37d/0xb60
Jan  9 00:41:22 umbar kernel: [74122.963917]  __alloc_pages_nodemask+0x255/0x270
Jan  9 00:41:22 umbar kernel: [74122.968441]  
do_huge_pmd_anonymous_page+0x13c/0x5d0
Jan  9 00:41:22 umbar kernel: [74122.973322]  __handle_mm_fault+0x984/0xfb0
Jan  9 00:41:22 umbar kernel: [74122.977438]  handle_mm_fault+0xc2/0x200
Jan  9 00:41:22 umbar kernel: [74122.981278]  __get_user_pages+0x258/0x6c0
Jan  9 00:41:22 umbar kernel: [74122.985290]  
get_user_pages_unlocked+0x153/0x1d0
Jan  9 00:41:22 umbar kernel: [74122.989954]  __gfn_to_pfn_memslot+0x149/0x410
Jan  9 00:41:22 umbar kernel: [74122.994320]  try_async_pf+0x96/0x1b0
Jan  9 00:41:22 umbar kernel: [74122.997900]  ? kvm_host_page_size+0x81/0xa0
Jan  9 00:41:22 umbar kernel: [74123.002093]  tdp_page_fault+0x168/0x2b0
Jan  9 00:41:22 umbar kernel: [74123.005927]  ? svm_vcpu_run+0x294/0x680
Jan  9 00:41:22 umbar kernel: [74123.009765]  kvm_mmu_page_fault+0x89/0x3f0
Jan  9 00:41:22 umbar kernel: [74123.013865]  ? kvm_set_cr8.part.87+0xa/0x30
Jan  9 00:41:22 umbar kernel: [74123.018049]  ? svm_vcpu_run+0x4fd/0x680
Jan  9 00:41:22 umbar kernel: [74123.021888]  ? kvm_fast_pio+0x140/0x190
Jan  9 00:41:22 umbar kernel: [74123.025729]  
kvm_arch_vcpu_ioctl_run+0x59e/0x19c0
Jan  9 00:41:22 umbar kernel: [74123.030435]  ? _copy_to_user+0x26/0x30
Jan  9 00:41:22 umbar kernel: [74123.034187]  ? kvm_vm_ioctl+0x69a/0x950
Jan  9 00:41:22 umb

Re: Can we drop upstream Linux x32 support?

[Adam Borowski]

On Thu, Dec 13, 2018 at 10:37:31AM +0100, Sven Hartrumpf wrote:
> Will the proposed patch ("only") remove the possibility to build x32 kernels
> or will it make impossible to compile and run any x32 binaries?

There's no such thing as x32 kernels.  It's an ABI atop amd64 kernels; the
kernel is always 64-bit -- short pointers apply only to userspace and
syscalls.

It could be interesting to block regular amd64 syscalls and leave x32 only
-- for "security through rarity", but the kernel itself would still look the
same.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢠⠒⠀⣿⡁ Ivan was a worldly man: born in St. Petersburg, raised in
⢿⡄⠘⠷⠚⠋⠀ Petrograd, lived most of his life in Leningrad, then returned
⠈⠳⣄ to the city of his birth to die.

Re: Official Linux system wrapper library?

[Adam Borowski]

On Sun, Nov 11, 2018 at 12:46:35PM +0100, Florian Weimer wrote:
> A lot of multi-threaded applications assume that most high-level
> functionality remains usable even after fork in a multi-threaded
> process.

How would this be even possible?  Currently fork kills all threads
(save for the caller).

Glibc's manpage also warns:

# After a fork() in a multithreaded program, the child can safely call only
# async-signal-safe functions (see signal-safety(7)) until such time as it
# calls execve(2).

Which makes sense as its malloc uses a mutex, and you can't take a breath
without a library call using malloc somewhere (or in C++, the language
itself).

So any functionality remaining usable after fork is pretty strictly
limited...


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ I've read an article about how lively happy music boosts
⣾⠁⢰⠒⠀⣿⡁ productivity. You can read it, too, you just need the
⢿⡄⠘⠷⠚⠋⠀ right music while doing so. I recommend Skepticism
⠈⠳⣄ (funeral doom metal).

[PATCH 01/17] lib: Add support for ZSTD-compressed kernel

[Adam Borowski]

From: Nick Terrell 

Add support for extracting ZSTD-compressed kernel images, as well as
ZSTD-compressed ramdisk images in the kernel boot process.

When neither `fill' nor `flush' are used, the decompression function
requires a constant amount of memory (192 KB is sufficient). When either
is used the decompression function requires memory proportional to the
window size used during compression, which is limited to 8 MB. The maximum
memory usage is just over 8 MB.

Fix up lib/zstd and lib/xxhash.c for the preboot environment. They avoid
declaring themselves as modules. A memcpy() call needs to be a
__builtin_memcpy() for performance. The gcc-7.1 bug in ZSTD_wildcopy() was
fixed in gcc-7.2, so it can be gated, since it hurts performance.

Signed-off-by: Nick Terrell 
---
 include/linux/decompress/unzstd.h |  26 +++
 init/Kconfig  |  14 +-
 lib/Kconfig   |   4 +
 lib/Makefile  |   1 +
 lib/decompress.c  |   5 +
 lib/decompress_unzstd.c   | 341 ++
 lib/xxhash.c  |  21 +-
 lib/zstd/decompress.c |   2 +
 lib/zstd/fse_decompress.c |   9 +-
 lib/zstd/zstd_internal.h  |  10 +-
 scripts/Makefile.lib  |  15 ++
 usr/Kconfig   |  22 ++
 12 files changed, 451 insertions(+), 19 deletions(-)
 create mode 100644 include/linux/decompress/unzstd.h
 create mode 100644 lib/decompress_unzstd.c

diff --git a/include/linux/decompress/unzstd.h 
b/include/linux/decompress/unzstd.h
new file mode 100644
index ..6f3022cd0955
--- /dev/null
+++ b/include/linux/decompress/unzstd.h
@@ -0,0 +1,26 @@
+/*
+ * Copyright (C) 2017 Facebook
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see .
+ */
+
+#ifndef LINUX_DECOMPRESS_UNZSTD_H
+#define LINUX_DECOMPRESS_UNZSTD_H
+
+int unzstd(unsigned char *inbuf, long len,
+  long (*fill)(void*, unsigned long),
+  long (*flush)(void*, unsigned long),
+  unsigned char *output,
+  long *pos,
+  void (*error_fn)(char *x));
+#endif
diff --git a/init/Kconfig b/init/Kconfig
index a4112e95724a..ffa5ae4abc88 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -134,13 +134,16 @@ config HAVE_KERNEL_LZO
 config HAVE_KERNEL_LZ4
bool
 
+config HAVE_KERNEL_ZSTD
+   bool
+
 config HAVE_KERNEL_UNCOMPRESSED
bool
 
 choice
prompt "Kernel compression mode"
default KERNEL_GZIP
-   depends on HAVE_KERNEL_GZIP || HAVE_KERNEL_BZIP2 || HAVE_KERNEL_LZMA || 
HAVE_KERNEL_XZ || HAVE_KERNEL_LZO || HAVE_KERNEL_LZ4 || HAVE_KERNEL_UNCOMPRESSED
+   depends on HAVE_KERNEL_GZIP || HAVE_KERNEL_BZIP2 || HAVE_KERNEL_LZMA || 
HAVE_KERNEL_XZ || HAVE_KERNEL_LZO || HAVE_KERNEL_LZ4 || HAVE_KERNEL_ZSTD || 
HAVE_KERNEL_UNCOMPRESSED
help
  The linux kernel is a kind of self-extracting executable.
  Several compression algorithms are available, which differ
@@ -219,6 +222,15 @@ config KERNEL_LZ4
  is about 8% bigger than LZO. But the decompression speed is
  faster than LZO.
 
+config KERNEL_ZSTD
+   bool "ZSTD"
+   depends on HAVE_KERNEL_ZSTD
+   help
+ ZSTD is a compression algorithm targeting intermediate compression
+ with fast decompression speed. It will compress better than GZIP and
+ decompress around the same speed as LZO, but slower than LZ4. You
+ will need at least 192 KB RAM or more for booting.
+
 config KERNEL_UNCOMPRESSED
bool "None"
depends on HAVE_KERNEL_UNCOMPRESSED
diff --git a/lib/Kconfig b/lib/Kconfig
index a9965f4af4dd..e7ab43fd5461 100644
--- a/lib/Kconfig
+++ b/lib/Kconfig
@@ -304,6 +304,10 @@ config DECOMPRESS_LZ4
select LZ4_DECOMPRESS
tristate
 
+config DECOMPRESS_ZSTD
+   select ZSTD_DECOMPRESS
+   tristate
+
 #
 # Generic allocator support is selected if needed
 #
diff --git a/lib/Makefile b/lib/Makefile
index db06d1237898..58b48993f48a 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -139,6 +139,7 @@ lib-$(CONFIG_DECOMPRESS_LZMA) += decompress_unlzma.o
 lib-$(CONFIG_DECOMPRESS_XZ) += decompress_unxz.o
 lib-$(CONFIG_DECOMPRESS_LZO) += decompress_unlzo.o
 lib-$(CONFIG_DECOMPRESS_LZ4) += decompress_unlz4.o
+lib-$(CONFIG_DECOMPRESS_ZSTD) += decompress_unzstd.o
 
 obj-$(CONFIG_TEXTSEARCH) += textsearch.o
 obj-$(CONFIG_TEXTSEARCH_KMP) += ts_kmp.o
diff --git a/lib/decompress.c b/lib/decompress.c
index

[PATCH 16/17] Kconfig: Update the prose for selection of compression algorithm

[Adam Borowski]

It was really obsolete, and some entries contradicted each other.

Let's not recommend ZSTD for kernel compression yet as it's available
only on x86, and some distros might not have the tool installed.
Proposing ZSTD for initrd is safer but let's test it first.

Signed-off-by: Adam Borowski 
---
 init/Kconfig | 25 +
 usr/Kconfig  | 24 +++-
 2 files changed, 24 insertions(+), 25 deletions(-)

diff --git a/init/Kconfig b/init/Kconfig
index 412ba93673fa..7c0180c41a3c 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -153,26 +153,27 @@ choice
  version of this functionality (bzip2 only), for 2.4, was
  supplied by Christian Ludwig)
 
- High compression options are mostly useful for users, who
- are low on disk space (embedded systems), but for whom ram
- size matters less.
+ High compression options tend to be more useful in most cases,
+ as bootloaders are often egregiously slow to read the kernel
+ from the disk/SD card/network/etc, overcoming any boot time
+ savings you would get from faster decompression.
 
- If in doubt, select 'gzip'
+ If in doubt, select 'xz'
 
 config KERNEL_GZIP
bool "Gzip"
depends on HAVE_KERNEL_GZIP
help
- The old and tried gzip compression. It provides a good balance
- between compression ratio and decompression speed.
+ The old and tried gzip compression. You generally want it if
+ some tool you use doesn't support more modern compressors.
 
 config KERNEL_LZMA
bool "LZMA"
depends on HAVE_KERNEL_LZMA
help
- This compression algorithm's ratio is best.  Decompression speed
- is between gzip and bzip2.  Compression is slowest.
- The kernel size is about 33% smaller with LZMA in comparison to gzip.
+ An old version of xz, like it providing strong compression at slow
+ speed. It lacks a header and support for filters or uncompressed
+ blocks, thus it's usually better to pick xz.
 
 config KERNEL_XZ
bool "XZ"
@@ -193,9 +194,9 @@ config KERNEL_LZO
bool "LZO"
depends on HAVE_KERNEL_LZO
help
- Its compression ratio is the poorest among the choices. The kernel
- size is about 10% bigger than gzip; however its speed
- (both compression and decompression) is the fastest.
+ Its compression ratio is pretty poor (but still better than
+ LZ4). You want to pick ZSTD (similar speed but much better
+ compression) or LZ4 (much better speed) instead.
 
 config KERNEL_LZ4
bool "LZ4"
diff --git a/usr/Kconfig b/usr/Kconfig
index 8d99edacabc9..f6e871585f05 100644
--- a/usr/Kconfig
+++ b/usr/Kconfig
@@ -131,17 +131,15 @@ config INITRAMFS_COMPRESSION_NONE
  on those architectures that support this. However, not compressing the
  initramfs may lead to slightly higher memory consumption during a
  short time at boot, while both the cpio image and the unpacked
- filesystem image will be present in memory simultaneously
+ filesystem image will be present in memory simultaneously.
 
 config INITRAMFS_COMPRESSION_GZIP
bool "Gzip"
depends on RD_GZIP
help
- Use the old and well tested gzip compression algorithm. Gzip provides
- a good balance between compression ratio and decompression speed and
- has a reasonable compression speed. It is also more likely to be
- supported by your build system as the gzip tool is present by default
- on most distros.
+ Use the old and well tested gzip compression algorithm. Gzip doesn't
+ provide very good compression nor speed, but it's the safest choice,
+ with wide support.
 
 config INITRAMFS_COMPRESSION_XZ
bool "XZ"
@@ -150,20 +148,20 @@ config INITRAMFS_COMPRESSION_XZ
  XZ uses the LZMA2 algorithm and has a large dictionary which may cause
  problems on memory constrained systems. The initramfs size is about
  30% smaller with XZ in comparison to gzip. Decompression speed is
- better than that of bzip2 but worse than gzip and LZO. Compression is
- slow.
+ okayish but still slowest of all currently available algorithms.
+ Compression is slow.
 
- If you choose this, keep in mind that you may need to install the xz
- tool to be able to compress the initram.
+ Any modern distro provides xz in its default install, but on
+ minimal build systems you might need to install xz-utils to be
+ able to compress the initram.
 
 config INITRAMFS_COMPRESSION_LZO
bool "LZO"
depends on RD_LZO
help
  It's compression ratio is the second poorest amongst

drivers by default (was Re: Another HID problem this merge window..)

[Adam Borowski]

On Sat, Oct 27, 2018 at 11:13:17AM -0700, Linus Torvalds wrote:
> Ok, so this is a much smaller issue than the i2c one that cause boot
> problems, but it's annoying.
> 
> We do *not* enable new random drivers by default. And we most
> *definitely* don't do it when they are odd-ball ones that most people
> have never heard of.
> 
> Yet the new "BigBen Interactive" driver that was added this merge
> window did exactly that.
> 
> Just don't do it.

Amen to that.  But, perhaps you could encourage people to do enable drivers
once they become very popular?  For example, I just (72a9c673636) got hit by
USB 3.0 being off in defconfigs, and not having keyboard is not that cool.

So what about a policy that says "almost all drivers are to be =n initially,
but patches enabling popular stuff by default are welcome"?  Preferably if
obsolete crap gets mass-disabled at least once per decade (think of the
floppy driver's importance on day one vs now).

> Yes, yes, every developer always thinks that _their_ driver is so
> special and so magically important that it should be enabled by
> default. But no. When we have thousands of drivers, we don't randomly
> pick one new driver to be enabled by default just because some
> developer thinks it is special. It's not.

We don't know which drivers will become important and which won't, only time
can tell.  But not having such drivers on by default is also a problem.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢰⠒⠀⣿⡁ 10 people enter a bar: 1 who understands binary,
⢿⡄⠘⠷⠚⠋⠀ 1 who doesn't, D who prefer to write it as hex,
⠈⠳⣄ and 1 who narrowly avoided an off-by-one error.

[tip:x86/build] x86/defconfig: Enable CONFIG_USB_XHCI_HCD=y

[tip-bot for Adam Borowski]

Commit-ID:  72a9c673636b779e370983fea08e40f97039b981
Gitweb: https://git.kernel.org/tip/72a9c673636b779e370983fea08e40f97039b981
Author: Adam Borowski 
AuthorDate: Tue, 9 Oct 2018 08:28:03 +0200
Committer:  Ingo Molnar 
CommitDate: Wed, 10 Oct 2018 08:29:51 +0200

x86/defconfig: Enable CONFIG_USB_XHCI_HCD=y

A spanking new machine I just got has all but one USB ports wired as 3.0.
Booting defconfig resulted in no keyboard or mouse, which was pretty
uncool.  Let's enable that -- USB3 is ubiquitous rather than an oddity.
As 'y' not 'm' -- recovering from initrd problems needs a keyboard.

Also add it to the 32-bit defconfig.

Signed-off-by: Adam Borowski 
Cc: Greg Kroah-Hartman 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: linux-...@vger.kernel.org
Link: http://lkml.kernel.org/r/20181009062803.4332-1-kilob...@angband.pl
Signed-off-by: Ingo Molnar 
---
 arch/x86/configs/i386_defconfig   | 1 +
 arch/x86/configs/x86_64_defconfig | 1 +
 2 files changed, 2 insertions(+)

diff --git a/arch/x86/configs/i386_defconfig b/arch/x86/configs/i386_defconfig
index 0eb9f92f3717..6c3ab05c231d 100644
--- a/arch/x86/configs/i386_defconfig
+++ b/arch/x86/configs/i386_defconfig
@@ -247,6 +247,7 @@ CONFIG_USB_HIDDEV=y
 CONFIG_USB=y
 CONFIG_USB_ANNOUNCE_NEW_DEVICES=y
 CONFIG_USB_MON=y
+CONFIG_USB_XHCI_HCD=y
 CONFIG_USB_EHCI_HCD=y
 CONFIG_USB_EHCI_TT_NEWSCHED=y
 CONFIG_USB_OHCI_HCD=y
diff --git a/arch/x86/configs/x86_64_defconfig 
b/arch/x86/configs/x86_64_defconfig
index e32fc1f274d8..ac9ae487cfeb 100644
--- a/arch/x86/configs/x86_64_defconfig
+++ b/arch/x86/configs/x86_64_defconfig
@@ -243,6 +243,7 @@ CONFIG_USB_HIDDEV=y
 CONFIG_USB=y
 CONFIG_USB_ANNOUNCE_NEW_DEVICES=y
 CONFIG_USB_MON=y
+CONFIG_USB_XHCI_HCD=y
 CONFIG_USB_EHCI_HCD=y
 CONFIG_USB_EHCI_TT_NEWSCHED=y
 CONFIG_USB_OHCI_HCD=y

Re: Contributors can not just rescind the license (Legal Opinion)

[Adam Borowski]

On Fri, Sep 28, 2018 at 11:31:32AM +0200, www.Advocati.org wrote:
> CONTRIBUTORS CAN NOT JUST RESCIND THE LICENSE
> THEY GRANTED UNDER GPLv2. IT IS *COPYLEFTED*

> A letter from 17.09.2018 by @observerofaffairs
> titled 'GPL version 2 is a bare license. Recind. (Regarding (future)
> Code of Conduct Bannings)'
> https://lkml.org/lkml/2018/9/17/1174,

> repeated by another letter from 20.09.2018 by @unconditionedwitness
> titled 'Re: A Plea to Unfuck our Codes of Conduct'
> https://lulz.com/linux-devs-threaten-killswitch-coc-controversy-1252/

You're responding to trolling by our dear MikeeUSA, whom most of us long
since learned to recognize and ignore.  There are better uses for your time
than responding to a notorious cartooney.

He's as nasty a troll as CoralineAda, and about as harmful to the community.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 10 people enter a bar:
⣾⠁⢰⠒⠀⣿⡁ • 1 who understands binary,
⢿⡄⠘⠷⠚⠋⠀ • 1 who doesn't,
⠈⠳⣄ • and E who prefer to write it as hex.

Re: POSIX violation by writeback error

[Adam Borowski]

On Tue, Sep 25, 2018 at 11:46:27AM -0400, Theodore Y. Ts'o wrote:
> P.S.  One thought: it might be cool if there was some way for
> userspace applications to mark files with "nuke if not closed" flag,
> such that if the system crashes, the file systems would automatically
> unlink the file after a reboot or if the process was killed or exits
> without an explicit close(2).  For networked/remote file systems that
> supported this flag, after the client comes back up after a reboot, it
> could notify the server that all files created previously from that
> client should be unlinked.
> 
> Unlike O_TMPFILE, this would require file system changes to support,
> so maybe it's not worth having something which automatically cleans up
> files that were in the middle of being written at the time of a system
> crash.

Isn't this what the snippet for O_TMPFILE in "man 2 open" does?:

  char path[PATH_MAX];
  fd = open("/path/to/dir", O_TMPFILE | O_RDWR,
  S_IRUSR | S_IWUSR);

  /* File I/O on 'fd'... */

  snprintf(path, PATH_MAX,  "/proc/self/fd/%d", fd);
  linkat(AT_FDCWD, path, AT_FDCWD, "/path/for/file",
  AT_SYMLINK_FOLLOW);


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 10 people enter a bar:
⣾⠁⢰⠒⠀⣿⡁ • 1 who understands binary,
⢿⡄⠘⠷⠚⠋⠀ • 1 who doesn't,
⠈⠳⣄ • and E who prefer to write it as hex.

Re: [...] an apology, and a maintainership note

[Adam Borowski]

On Sun, Sep 16, 2018 at 12:22:43PM -0700, Linus Torvalds wrote:
> This is my reality.  I am not an emotionally empathetic kind of person
> and that probably doesn't come as a big surprise to anybody.  Least of
> all me.  The fact that I then misread people and don't realize (for
> years) how badly I've judged a situation and contributed to an
> unprofessional environment is not good.
> 
> This week people in our community confronted me about my lifetime of
> not understanding emotions.  My flippant attacks in emails have been
> both unprofessional and uncalled for.  Especially at times when I made
> it personal.  In my quest for a better patch, this made sense to me.
> I know now this was not OK and I am truly sorry.
> 
> The above is basically a long-winded way to get to the somewhat
> painful personal admission that hey, I need to change some of my
> behavior, and I want to apologize to the people that my personal
> behavior hurt and possibly drove away from kernel development
> entirely.

Despite me being just among bottom-rung popcorn of kernel contributors, let
me says this:

No.  Just no.  You're so successful because you're one of few people who
don't waste time beating around the bush.  You call a spade a spade instead
of polite "professional" bullshit.

You often use rude words, but you don't do so without a reason.  IMO your
most striking quality is not technical ability (pretty high...) but the
ratio of times you open your mouth to the times you're right.  And even
if you're not right, you don't take offense at getting corrected and
immediately admit someone else was right.  

Sure, there are cases when both choices are right, but your approach avoids
wasting time making a decision.  For example: recently, you forced disabling
string truncation warnings despite many people feeling otherwise.  I for one
believe GCC's warnings even though sounding bogus are good for eliminating
strncpy -- what I would have done would be giving it an aliased version
named "fixedfieldncpy" or such that disables the warning, and fixing the
whole rest.  But what you did instead deprioritizes the issue: the kernel
doesn't work any worse than it did with gcc-7, thus there are indeed more
urgent matters elsewhere.  So even if I don't fully agree with you, you are
the boss and as long as your version is acceptable, let's stick to it.

And, it's _you_ who has proven merit, not me.

> I am going to take time off and get some assistance on how to
> understand people’s emotions and respond appropriately.
> 
> Put another way: When asked at conferences, I occasionally talk about
> how the pain-points in kernel development have generally not been
> about the _technical_ issues, but about the inflection points where
> development flow and behavior changed.

Too many projects get detracted by prolonged crap about social things, don't
let this pull you down.  There's a problem when people _without merit_ are
rude -- those indeed need to get a spanking.  A spanking not ADHD meds.
Short and to the point, letting them learn.

But you, you _earned_ the right to be rude to get your point across.
I watched a video about you getting shamed on a DebConf because of breaching
some "code of conduct" by using a naughty word.  I didn't like that and
believe it was you who was right (I don't recall the details though).

> I've talked to Greg to ask him if he'd mind finishing up 4.19 for me, so
> that I can take a break, and try to at least fix my own behavior.
> 
> This is not some kind of "I'm burnt out, I need to just go away"
> break.  I'm not feeling like I don't want to continue maintaining
> Linux. Quite the reverse.  I very much *do* want to continue to do
> this project that I've been working on for almost three decades.
> 
> This is more like the time I got out of kernel development for a while
> because I needed to write a little tool called "git".  I need to take
> a break to get help on how to behave differently and fix some issues
> in my tooling and workflow.

You do deserve a vacation.  By all means, do take a break and let the
community rehearse for "Linus got mauled by a bear".  But we want you back.

> And yes, some of it might be "just" tooling.  Maybe I can get an email
> filter in place so at when I send email with curse-words, they just
> won't go out.  Because hey, I'm a big believer in tools, and at least
> _some_ problems going forward might be improved with simple
> automation.

Please don't.  When you use curse words, they're _warranted_.  They're a
tool which, in my opinion, you don't overuse.

And it's fun to listen to a true master of words.  An example: how many
pages would https://lkml.org/lkml/2016/8/2/2062 take to say politely yet get
the same effect?

> I know when I really look “myself in the mirror” it will be clear it's
> not the only change that has to happen, but hey...  You can send me
> suggestions in email.

When you look yourself in the mirror, I want you to see that guy who codes
in a bathrobe instea

Re: Kernel-only deployments?

[Adam Borowski]

On Thu, Aug 23, 2018 at 10:43:59AM -0700, Paul E. McKenney wrote:
> The mkinitramfs approach results in about 40MB of initrd, and dracut
> about 10MB.  Most of this is completely useless for rcutorture, which
> isn't interested in mounting filesystems, opening devices, and almost
> all of the other interesting things that mkinitramfs and dracut enable.
> 
> Those who know me will not be at all surprised to learn that I went
> overboard making the resulting initrd as small as possible.  I started
> by throwing out everything not absolutely needed by the dash and sleep
> binaries, which got me down to about 2.5MB, 1.8MB of which was libc.
> This situation of course prompted me to create an initrd containing
> a statically linked binary named "init" and absolutely nothing else
> (not even /dev or /tmp directories), which weighs in at not quite 800KB.
> This is a great improvement over 10MB, to say nothing of 40MB, but 800KB
> for a C-language "for" loop containing nothing more than a single call to
> sleep()?

.globl _start
.data
req:.8byte 9, 9
.text
_start:
mov $35, %rax   # syscall: nanosleep
mov $req, %rdi
xor %rsi, %rsi
syscall
jmp _start


as sl.s -o sl.o
ld sl.o -o init

'Ere you go, no libc needed.  If your arch is not amd64, just say so.

If you want to do anything more complex, though -- you really want musl
or another lightweight libc instead.  Glibc is utterly unfit for static
linking.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ .globl _start↵.data↵rc: .ascii "/etc/init.d/rcS\0"↵.text↵_start
⣾⠁⢰⠒⠀⣿⡁ mov $57,%rax↵syscall↵cmp $0,%rax↵jne child↵parent:↵mov $61,%rax
⢿⡄⠘⠷⠚⠋⠀ mov $-1,%rdi↵xor %rsi,%rsi↵xor %rdx,%rdx↵syscall↵jmp parent↵child:
⠈⠳⣄ mov $59,%rax↵mov $rc,%rdi↵xor %rsi,%rsi↵xor %rdx,%rdx↵syscall

Re: linux-next: build warnings from Linus' tree

[Adam Borowski]

On Sun, Aug 19, 2018 at 04:21:57PM -0700, Linus Torvalds wrote:
> On Sun, Aug 19, 2018 at 3:13 PM Stephen Rothwell  
> wrote:
> >
> > Today's linux-next build (powerpc ppc64_defconfig) produced these
> > warnings:
> >
> > fs/cifs/cifssmb.c:605:3: warning: 'strncpy' writing 16 bytes into a region 
> > of size 1 overflows the destination [-Wstringop-overflow=]
> >strncpy(pSMB->DialectsArray+count, protocols[i].name, 16);
> >
> > Presumably caused by my update to gcc 8.2.0.
> 
> Yeah. There are some patches to mark some arrays as non-strings to get
> rid of these, but we'll see. Maybe we'll just disable the new gcc
> warning if it causes more pain than it is worth.

Every single use of strncpy() for a C string is either a bug, inefficiency,
or both.  In this particular case the code:

count = 0;
for (i = 0; i < CIFS_NUM_PROT; i++) {
strncpy(pSMB->DialectsArray+count, protocols[i].name, 16);
count += strlen(protocols[i].name) + 1;
/* null at end of source and target buffers anyway */
}

* pointlessly clears 16 bytes in every iteration
* calculates the string's length twice
* there's no protection against buffer overflow anyway

So what is the strncpy() there for, when an unbounded copy would be just as
good?  For other cases, there's a bunch of better functions: strlcpy(),
snprintf(), even strlen()+memcpy(), etc.

Valid uses of strncpy() do exist (such as SCSI structs), but those deal with
fixed-width fields.  Thus, gcc is right for warning for at least some of
misuse of strncpy() for C strings.  The function wasn't designed for them.

(Skipped analysis why strncpy is always a bad choice for C strings.)


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ What Would Jesus Do, MUD/MMORPG edition:
⣾⠁⢰⠒⠀⣿⡁ • multiplay with an admin char to benefit your mortal [Mt3:16-17]
⢿⡄⠘⠷⠚⠋⠀ • abuse item cloning bugs [Mt14:17-20, Mt15:34-37]
⠈⠳⣄ • use glitches to walk on water [Mt14:25-26]

Re: [RESEND PATCH v2 0/2] Add support for ZSTD-compressed kernel

[Adam Borowski]

On Fri, Aug 17, 2018 at 12:22:44PM -0700, Andi Kleen wrote:
> On Fri, Aug 17, 2018 at 07:57:46PM +0200, Adam Borowski wrote:
> > > The "favourite compressor" seems to roughly change every year, so if
> > > we keep adding new ones things will get more and more convoluted.
> > 
> > The above patchset drops just bzip2.  It is the only one that's strictly
> > beaten in every way (ratio, time, memory usage), there are also no other
> 
> Does time include build time? I've been reverting back to gzip recently
> because I care very much about that.

Too lazy to benchmark a kernel image (IIRC Nick Terrell posted that a while
ago), here's copypasta of a random 16824672 byte executable, in userspace,
with default level setting:

compdecomp  size
xz  8.038s  0.356s  4320292
bz2 2.265s  0.730s  5234516
zst 0.274s  0.102s  5657626
gz  0.880s  0.152s  6515505
Z   0.499s  0.133s  8932459
lzo 0.100s  0.095s  9198874

As you can see, zstd's compression time is drastically better than gzip,
while ratio is better.  The default level is very low (-3 on -1..-22 scale)
but you can crank it up for stronger compression.

The defaults fit your use case.

> > uses of bzip2 anywhere in the kernel so we'd get to drop its code
> > completely: 900 lines of Linus' happiness.
> 
> Great!
> 
> > Other candidates are lzo and bare lzma (you want lz4, zstd or xz instead),
> > but those are used elsewhere thus there's hardly any gain.  If you want them
> > gone, please say so -- I'll include their droppage.
> 
> Yes would be good to remove the kernel image support for those too
> just to simplify the config process, even if it doesn't save much code.

There's one caveat: fast choices are quite new:
* lz4 userspace tools are not even in current Debian stable (just unstable)
* uncompressed kernel got in only this merge window
* zstd has userspace tools in Debian stable but is not merged into the
  kernel yet
(other dists are probably similar)

Thus, it might be a good idea to keep lzo for a while longer.

Bare lzma can probably go -- xz filters are nice for binaries of archs it
knows (disabled otherwise), and lack of header requires hacks to find out
the payload's size.

So it's up to you guys: do you want me to drop lzo and/or lzma?
We can also drop them just for vmlinuz but not initrd.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ What Would Jesus Do, MUD/MMORPG edition:
⣾⠁⢰⠒⠀⣿⡁ • multiplay with an admin char to benefit your mortal [Mt3:16-17]
⢿⡄⠘⠷⠚⠋⠀ • abuse item cloning bugs [Mt14:17-20, Mt15:34-37]
⠈⠳⣄ • use glitches to walk on water [Mt14:25-26]

Re: [RESEND PATCH v2 0/2] Add support for ZSTD-compressed kernel

[Adam Borowski]

On Fri, Aug 17, 2018 at 09:54:03AM -0700, Andi Kleen wrote:
> On Fri, Aug 17, 2018 at 06:15:45PM +0200, René Rebe wrote:
> > Hey,
> > 
> > is there any mainline future for this zstd support?
> > Currently my most favourite compressor for this, and for what it’s worth
> > zstd/initrd now even tested on #t2sde / hp/pa-risc
> 
> Can you remove some other compressor for the kernel image if you merge this?

Funny you say this.  I want to post this after the merge window is over:
https://github.com/kilobyte/linux/commits/nobz2-next-20180731

Sorry for a tree atop next, but there are conflicts that needed to be
adressed.  The patchset is also undertested (works for me, but not even
compile-tested on archs I don't have a toolchain for at hand).

> The "favourite compressor" seems to roughly change every year, so if
> we keep adding new ones things will get more and more convoluted.

The above patchset drops just bzip2.  It is the only one that's strictly
beaten in every way (ratio, time, memory usage), there are also no other
uses of bzip2 anywhere in the kernel so we'd get to drop its code
completely: 900 lines of Linus' happiness.

Other candidates are lzo and bare lzma (you want lz4, zstd or xz instead),
but those are used elsewhere thus there's hardly any gain.  If you want them
gone, please say so -- I'll include their droppage.

> Surely zstd is universally better than some existing compressor
> the kernel already uses for itself for a particular sweet spot?
> If it isn't it's likely not needed.

zstd is better in the balanced niche: unless you want extreme speed (lz4) or
best compression (xz), zstd handily beats algorithms in the middle.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ What Would Jesus Do, MUD/MMORPG edition:
⣾⠁⢰⠒⠀⣿⡁ • multiplay with an admin char to benefit your mortal [Mt3:16-17]
⢿⡄⠘⠷⠚⠋⠀ • abuse item cloning bugs [Mt14:17-20, Mt15:34-37]
⠈⠳⣄ • use glitches to walk on water [Mt14:25-26]

Re: Linux 4.18.1

[Adam Borowski]

On Thu, Aug 16, 2018 at 05:11:21PM +0200, Greg KH wrote:
> On Thu, Aug 16, 2018 at 03:59:58PM +0200, Sven Joachim wrote:
> > On 2018-08-16 15:05 +0200, Adam Borowski wrote:
> > > I'm afraid that I get a build failure; v4.18 is ok, v4.18.1 fails with:
> > >
> > > ld: arch/x86/kvm/x86.o: in function `kvm_get_arch_capabilities':
> > > (.text+0x43b2): undefined reference to `l1tf_vmx_mitigation'
> > 
> > Same here and also in 4.17.15, but not in Linus' tree.
> > 
> > > CONFIG_KVM=y
> > > # CONFIG_KVM_INTEL is not set
> > > CONFIG_KVM_AMD=y
> > 
> > I have CONFIG_KVM{,AMD}=m and CONFIG_KVM_INTEL is not set either.
> 
> This is fixed in my queue, if it really bothers you, apply commit
> 1eb46908b35d ("x86/l1tf: Fix build error seen if CONFIG_KVM_INTEL is
> disabled") to your tree, it will be in the next round of stable kernel
> releases.

Probably obvious, but: cherry-picking that commit fixes the problem, it
builds, boots and works ok.

Thanks!


Meow.
-- 
⢀⣴⠾⠻⢶⣦⠀ What Would Jesus Do, MUD/MMORPG edition:
⣾⠁⢰⠒⠀⣿⡁ • multiplay with an admin char to benefit your mortal [Mt3:16-17]
⢿⡄⠘⠷⠚⠋⠀ • abuse item cloning bugs [Mt14:17-20, Mt15:34-37]
⠈⠳⣄ • use glitches to walk on water [Mt14:25-26]

Re: Linux 4.18.1

[Adam Borowski]

Hi!

On Thu, Aug 16, 2018 at 12:14:29PM +0200, Greg KH wrote:
> I'm announcing the release of the 4.18.1 kernel.

I'm afraid that I get a build failure; v4.18 is ok, v4.18.1 fails with:

ld: arch/x86/kvm/x86.o: in function `kvm_get_arch_capabilities':
(.text+0x43b2): undefined reference to `l1tf_vmx_mitigation'

gcc 8.2.0-4, as shipped in Debian unstable.
.config attached.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ What Would Jesus Do, MUD/MMORPG edition:
⣾⠁⢰⠒⠀⣿⡁ • multiplay with an admin char to benefit your mortal [Mt3:16-17]
⢿⡄⠘⠷⠚⠋⠀ • abuse item cloning bugs [Mt14:17-20, Mt15:34-37]
⠈⠳⣄ • use glitches to walk on water [Mt14:25-26]


config.xz
Description: application/xz

Re: get_maintainer.pl and change of email

[Adam Borowski]

On Wed, Aug 01, 2018 at 11:36:23AM -0700, Nick Desaulniers wrote:
> It seems that get_maintainer.pl will make recommendations based on
> commit history to a file, but over time, people change emails that
> they commit from, then get_maintainer.pl recommends the possibly now
> invalid email address.
> 
> I feel like we should have a look up table / file like MAINTAINERS
> that tracks previous and new email addresses, which get_maintainer.pl
> would parse, that way we have a record of movement between committer
> email addresses allowing get_maintainer.pl to make more accurate
> recommendations.  Thoughts?

.mailmap

-- 
// If you believe in so-called "intellectual property", please immediately
// cease using counterfeit alphabets.  Instead, contact the nearest temple
// of Amon, whose priests will provide you with scribal services for all
// your writing needs, for Reasonable And Non-Discriminatory prices.

Re: [PATCH 0/6] vt: no blinking on console, 256/24-bit color improvements

[Adam Borowski]

On Mon, Jul 23, 2018 at 10:53:29AM +0200, Geert Uytterhoeven wrote:
> On Sat, Jul 21, 2018 at 11:39 PM Adam Borowski  wrote:
> > Technically, every console can be made to blink by drawing/clearing affected
> > characters a few times per second, but that'd be quite a waste of coding
> > time and kernel size.  There's a reason browsers dropped support for 
> > and text-decoration:blink.
> 
> It's very simple and fast to implement in fbcon for FB_TYPE_PLANES or
> FB_TYPE_INTERLEAVED_PLANES and FB_VISUAL_PSEUDOCOLOR ;-)

Interesting...  I'm still not going to do the effort to implement that
(which would require learning fbdev internals first), though.  But while I
dislike this feature, someone else might want it.

The main problem here is that there are only 8 or 7 bits available for
attributes, thus it's better to use them for something more useful.  And
here, fbcon already interprets this bit as bright background, thus this
patchset makes vt use it instead of non-existant blink.

There'll be more bits available once attributes get migrated into uniscr --
either 11 or 32 bits depending on chosen implementation.  But I still
wouldn't go too wild with them: the console is meant for recovery tasks as
on any properly working system you can have an X terminal configured for
pixel-to-pixel identical behaviour as anything console can do.  Thus, only
cheap improvements to attributes make sense.  This patchset is currently at
+3 net lines, this certainly counts as cheap.

On the other hand, displaying the _glyph_ better would be worth some effort.
Nicolas' uniscr that's already in Greg's tree did most of the work, what
remains is a way to load and store a font with more glyphs (needs a sparse
representation).


Meow!
-- 
// If you believe in so-called "intellectual property", please immediately
// cease using counterfeit alphabets.  Instead, contact the nearest temple
// of Amon, whose priests will provide you with scribal services for all
// your writing needs, for Reasonable And Non-Discriminatory prices.

Re: [PATCH 0/6] vt: no blinking on console, 256/24-bit color improvements

[Adam Borowski]

On Sat, Jul 21, 2018 at 09:43:19AM +0200, Greg Kroah-Hartman wrote:
> On Wed, Jul 18, 2018 at 05:01:52AM +0200, Adam Borowski wrote:
> > Here's a patchset with two entangled improvements:
> > 
> > * it'd be good to get rid of blinking where possible.  Even CGA (thus VGA)
> >   allows disabling it, rendering such characters with a bright background
> >   instead.
> > * due to my error, 256-color mode uses a much darker palette for conversion,
> >   resulting in behaving inconsistently with 24-bit mode.
> > 
> > The new code uses bright backgrounds when possible, enabled with \e[100m or
> > \e[48;m.
> > 
> > Despite the whole idea following a VGA capability, this patchset doesn't
> > change vgacon yet, just fbcon.  [Not breaking nvidia-proprietary.]
> > 
> > Thus, let's enable unblinking on fbcon for now.  We can flip that bit (in
> > register 0x10) later.
> > 
> > This fixes the display of catimg and similar tools.
> 
> I've applied the first patch, as it was obvious :)
> 
> For the rest, can you make it a config option as Alan said?  And I
> agree, we don't care about breaking nvidia systems, go ahead :)

The only thing such an option would be able to set is disabling blinking on
vgacon, which is not yet implemented.  fbcon never blinks, and the only case
it can't display bright background is 1-bit black&white.

I probably should change the line in fbcon:
+   vc->vc_unblinking = vc->vc_can_do_color;
to always use 1, but it doesn't matter either way.  Only difference would be
matching the variable name ("unblinking" vs "shows_bright_bg").

Console drivers can interpret that bit of attribute as:
* blinking (vc_unblinking = 0)
* bright background (vc_unblinking = 1)
* neither (result is ignored down the pipeline)

The matrix is:
color b&w
---+---+--
fbbright bg neither
mda  N/A blink
newport  ?bright bg? ?N/A?
sti   ?neither??
vga  CGA/VGA: hardware switch (write to port)  MDA: blink
sisusb??

Technically, every console can be made to blink by drawing/clearing affected
characters a few times per second, but that'd be quite a waste of coding
time and kernel size.  There's a reason browsers dropped support for 
and text-decoration:blink.

This patchset doesn't drop blink in any case that was supported before,
though.  I did the easy part used by most people first (fbcon), so it can be
reviewed/merged.  Doing the hard part is quite pointless if this is NAKed
(but obviously doesn't require actual merging).

But perhaps by "option" you mean letting the user drop blink with no
replacement, which might also be a good idea if configurable?


Meow!
-- 
// If you believe in so-called "intellectual property", please immediately
// cease using counterfeit alphabets.  Instead, contact the nearest temple
// of Amon, whose priests will provide you with scribal services for all
// your writing needs, for Reasonable And Non-Discriminatory prices.
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 0/6] vt: no blinking on console, 256/24-bit color improvements

[Adam Borowski]

On Thu, Jul 19, 2018 at 11:47:49AM +0100, Alan Cox wrote:
> On Wed, 18 Jul 2018 05:01:52 +0200
> Adam Borowski  wrote:
> 
> > Hi!
> > Here's a patchset with two entangled improvements:
> > 
> > * it'd be good to get rid of blinking where possible.  Even CGA (thus VGA)
> >   allows disabling it, rendering such characters with a bright background
> >   instead.
> 
> That's a matter of taste so needs to configurable. Changing the default
> ought I think to be its own patch as it's a separate discussion to having
> the choice there and easy to make.

For vgacon yeah, you have a good point, as it can support either (and to be
exact, exactly one of the two as they share a hardware bit).  Only reason to
not have this configurable would be avoiding bloating the kernel with knobs
hardly anyone flips -- but you can already set minutiae like replacement
color for underline/italic/dim.  Thus you're right that if/when killing
blink on vgacon is implemented, it probably should be settable.

This is not an option on fbcon, though -- it can't blink (doable but I'm not
seeing anyone wanting to implement that) and already interprets that
attribute bit the way VGA would.  Thus, this patch merely makes vt behave a
way to match what the driver does.  This fixes some visual corruption in
certain user programs.

> For the palette why does it needs changing and exactly what standards
> document defines 'right', especially given we don't do ICC in console
> mode ? Have you tested the values used against multiple monitor types and
> cards with a light meter ?

You don't need a light meter for a difference of 53-out-of-256.  Most
desktop environments don't do ICC out of the box either, but there's an
assumption that whatever your monitor does, it gives the same result for the
same input (in the same lighting conditions).

That patch's purpose is:
* behave consistently between two APIs to set the same thing (\e[38;5;m vs
  \e[38;2;m) in a way that matches other terminals
* in case a future driver has better color handling we'd be different from
  other terminals

By the way, you can have ICC on console: "apt install vtgamma"
(https://github.com/kilobyte/vtgamma).

> BTW visibly breaking the Nvidia crud is also fine. They'll then actually
> bother to fix it and uually quite soon.

Yeah, but they haven't fixed 512-glyph yet (at least the last time I
looked), broken since day one.  In any case, this patchset doesn't support
vgacon yet, thus this is moot for now.  I picked the easy case (fbcon which
is always unblinking) over writing stuff to ports, we can update other
drivers later.

And vgacon has code that looks like it can do CGA and MDA (redundant with
mdacon?), either of which I haven't used in... quite a while.  Might be
tricky getting access to such hardware to test...


Meow!
-- 
// If you believe in so-called "intellectual property", please immediately
// cease using counterfeit alphabets.  Instead, contact the nearest temple
// of Amon, whose priests will provide you with scribal services for all
// your writing needs, for Reasonable And Non-Discriminatory prices.

[PATCH 3/6] vt: let \e[100m use bright background if unblinking

[Adam Borowski]

Let's keep \e[5m setting this bit, it's a nice way to convey the
information, and it preserves old behaviour.  Some other terminals
that can't or don't want to blink do so as well.

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/vt.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
index 45057bbf6f74..4096093c8cd2 100644
--- a/drivers/tty/vt/vt.c
+++ b/drivers/tty/vt/vt.c
@@ -1709,6 +1709,8 @@ static void csi_m(struct vc_data *vc)
if (vc->vc_par[i] >= 90 && vc->vc_par[i] <= 107) {
if (vc->vc_par[i] < 100)
vc->vc_intensity = 2;
+   else if (vc->vc_unblinking)
+   vc->vc_blink = 1;
vc->vc_par[i] -= 60;
}
if (vc->vc_par[i] >= 30 && vc->vc_par[i] <= 37)
-- 
2.18.0

--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 1/6] vt: drop unused struct vt_struct

[Adam Borowski]

Hasn't been ever used within historic (ie, git) times.

Signed-off-by: Adam Borowski 
---
 include/linux/console_struct.h | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/include/linux/console_struct.h b/include/linux/console_struct.h
index 2c8d3239899b..fea64f2692a0 100644
--- a/include/linux/console_struct.h
+++ b/include/linux/console_struct.h
@@ -17,7 +17,6 @@
 #include 
 #include 
 
-struct vt_struct;
 struct uni_pagedir;
 struct uni_screen;
 
@@ -150,7 +149,7 @@ struct vc {
struct vc_data *d;
struct work_struct SAK_work;
 
-   /* might add  scrmem, vt_struct, kbd  at some time,
+   /* might add  scrmem, kbd  at some time,
   to have everything in one place - the disadvantage
   would be that vc_cons etc can no longer be static */
 };
-- 
2.18.0

[PATCH 2/6] vt: add console flag "unblinking"

[Adam Borowski]

Marks consoles that interpret the blink bit by showing bright background
instead.  Doesn't matter if the console can't do either.

For now, turn it on for fbcon when in color mode.  Vgacon can also do so
but requires setting appropriate VGA register (bit 3 of AMCR).  I don't
know other consoles: newport looks like it shows bright bg, sti can't do
either, mda appears to blink, etc -- but confirmation would be needed.

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/vt.c  | 1 +
 drivers/video/fbdev/core/fbcon.c | 1 +
 include/linux/console_struct.h   | 1 +
 3 files changed, 3 insertions(+)

diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
index 846dfedb657d..45057bbf6f74 100644
--- a/drivers/tty/vt/vt.c
+++ b/drivers/tty/vt/vt.c
@@ -998,6 +998,7 @@ static void visual_init(struct vc_data *vc, int num, int 
init)
vc->vc_hi_font_mask = 0;
vc->vc_complement_mask = 0;
vc->vc_can_do_color = 0;
+   vc->vc_unblinking = 0;
vc->vc_panic_force_write = false;
vc->vc_cur_blink_ms = DEFAULT_CURSOR_BLINK_MS;
vc->vc_sw->con_init(vc, init);
diff --git a/drivers/video/fbdev/core/fbcon.c b/drivers/video/fbdev/core/fbcon.c
index c910e74d46ff..4c67254f1ec4 100644
--- a/drivers/video/fbdev/core/fbcon.c
+++ b/drivers/video/fbdev/core/fbcon.c
@@ -1092,6 +1092,7 @@ static void fbcon_init(struct vc_data *vc, int init)
 
vc->vc_panic_force_write = !!(info->flags & FBINFO_CAN_FORCE_OUTPUT);
vc->vc_can_do_color = (fb_get_color_depth(&info->var, &info->fix)!=1);
+   vc->vc_unblinking = vc->vc_can_do_color;
vc->vc_complement_mask = vc->vc_can_do_color ? 0x7700 : 0x0800;
if (charcnt == 256) {
vc->vc_hi_font_mask = 0;
diff --git a/include/linux/console_struct.h b/include/linux/console_struct.h
index fea64f2692a0..f94b28a6bd2d 100644
--- a/include/linux/console_struct.h
+++ b/include/linux/console_struct.h
@@ -122,6 +122,7 @@ struct vc_data {
unsigned intvc_ques : 1;
unsigned intvc_need_wrap: 1;
unsigned intvc_can_do_color : 1;
+   unsigned intvc_unblinking   : 1;/* shows bright bg for blink */
unsigned intvc_report_mouse : 2;
unsigned char   vc_utf  : 1;/* Unicode UTF-8 encoding */
unsigned char   vc_utf_count;
-- 
2.18.0

[PATCH 5/6] vt: compensate for brightening the 256-color palette

[Adam Borowski]

The algorithm for 256-to-16 conversion was designed with wrong input
palette but actually tuned on mainstream GUI terminals.  This resulted in
something that works well only for data we convert ourselves (ie, 256 not
24-bit).

As the change is non-linear, I did not bother replicating it exactly, thus
there are some differences, among others:
* values very close to black go to 0 (black) rather than 8 (dark grey)
* grayscale ramp is more even

A comparison of the old vs new vs FreeBSD's teken is at:
https://github.com/kilobyte/colorkernel

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/vt.c | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
index 8c61caafdf3c..c777f4c91df0 100644
--- a/drivers/tty/vt/vt.c
+++ b/drivers/tty/vt/vt.c
@@ -1559,17 +1559,17 @@ static void rgb_foreground(struct vc_data *vc, const 
struct rgb *c)
 {
u8 hue = 0, max = max3(c->r, c->g, c->b);
 
-   if (c->r > max / 2)
+   if (c->r > max / 2 + 32)
hue |= 4;
-   if (c->g > max / 2)
+   if (c->g > max / 2 + 32)
hue |= 2;
-   if (c->b > max / 2)
+   if (c->b > max / 2 + 32)
hue |= 1;
 
-   if (hue == 7 && max <= 0x55) {
+   if (hue == 7 && max <= 0x70) {
hue = 0;
vc->vc_intensity = 2;
-   } else if (max > 0xaa)
+   } else if (max > 0xc0)
vc->vc_intensity = 2;
else
vc->vc_intensity = 1;
-- 
2.18.0

[PATCH 4/6] vt: change 256-color palette to match all(?) modern terminals

[Adam Borowski]

Turns out that osso-xterm which I based upon uses something a lot different
from apparently any other terminal -- they all use identical shades, much
brighter than what I copied:

Old: 00 2a 55 7f aa d4
New: 00 5f 87 af d7 ff

This did hardly matter as we immediately shoehorn the colors into only 16
values, but recently 24-bit codes turned from an oddity to something
widespread, thus it's better to handle 256 vs 24-bit consistently.

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/vt.c | 9 ++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
index 4096093c8cd2..8c61caafdf3c 100644
--- a/drivers/tty/vt/vt.c
+++ b/drivers/tty/vt/vt.c
@@ -1545,9 +1545,12 @@ static void rgb_from_256(int i, struct rgb *c)
c->g = i&2 ? 0xff : 0x55;
c->b = i&4 ? 0xff : 0x55;
} else if (i < 232) {   /* 6x6x6 colour cube. */
-   c->r = (i - 16) / 36 * 85 / 2;
-   c->g = (i - 16) / 6 % 6 * 85 / 2;
-   c->b = (i - 16) % 6 * 85 / 2;
+   int r = (i - 16) / 36;
+   int g = (i - 16) / 6 % 6;
+   int b = (i - 16) % 6;
+   c->r = r ? r * 0x28 + 0x37 : 0;
+   c->g = g ? g * 0x28 + 0x37 : 0;
+   c->b = b ? b * 0x28 + 0x37 : 0;
} else  /* Grayscale ramp. */
c->r = c->g = c->b = i * 10 - 2312;
 }
-- 
2.18.0

--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 6/6] vt: support bright backgrounds for \e[48m if unblinking

[Adam Borowski]

This improves schemes used by fancy new programs.  For example, it lets
bright powerline segments match, and fixes images shown by catimg being
striped to the point of unreadability.

Handling of 8-color backgrounds uses stripped 16-color value instead of a
dedicated formula; this works worse for dark and better for bright inputs.

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/vt.c | 38 +-
 1 file changed, 17 insertions(+), 21 deletions(-)

diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
index c777f4c91df0..7fcb0ff2dccf 100644
--- a/drivers/tty/vt/vt.c
+++ b/drivers/tty/vt/vt.c
@@ -1555,7 +1555,7 @@ static void rgb_from_256(int i, struct rgb *c)
c->r = c->g = c->b = i * 10 - 2312;
 }
 
-static void rgb_foreground(struct vc_data *vc, const struct rgb *c)
+static u8 rgb_to_16(const struct rgb *c)
 {
u8 hue = 0, max = max3(c->r, c->g, c->b);
 
@@ -1566,22 +1566,12 @@ static void rgb_foreground(struct vc_data *vc, const 
struct rgb *c)
if (c->b > max / 2 + 32)
hue |= 1;
 
-   if (hue == 7 && max <= 0x70) {
-   hue = 0;
-   vc->vc_intensity = 2;
-   } else if (max > 0xc0)
-   vc->vc_intensity = 2;
+   if (hue == 7 && max <= 0x70)
+   return 8;
+   if (max > 0xc0)
+   return hue | 8;
else
-   vc->vc_intensity = 1;
-
-   vc->vc_color = (vc->vc_color & 0xf0) | hue;
-}
-
-static void rgb_background(struct vc_data *vc, const struct rgb *c)
-{
-   /* For backgrounds, err on the dark side. */
-   vc->vc_color = (vc->vc_color & 0x0f)
-   | (c->r&0x80) >> 1 | (c->g&0x80) >> 2 | (c->b&0x80) >> 3;
+   return hue;
 }
 
 /*
@@ -1593,10 +1583,10 @@ static void rgb_background(struct vc_data *vc, const 
struct rgb *c)
  * Subcommands 3 (CMY) and 4 (CMYK) are so insane there's no point in
  * supporting them.
  */
-static int vc_t416_color(struct vc_data *vc, int i,
-   void(*set_color)(struct vc_data *vc, const struct rgb *c))
+static int vc_t416_color(struct vc_data *vc, int i, int bgshift)
 {
struct rgb c;
+   u8 v;
 
i++;
if (i > vc->vc_npar)
@@ -1615,7 +1605,13 @@ static int vc_t416_color(struct vc_data *vc, int i,
} else
return i;
 
-   set_color(vc, &c);
+   v = rgb_to_16(&c);
+
+   vc->vc_color = (vc->vc_color & (0xf0 >> bgshift)) | v << bgshift;
+   if (!bgshift)
+   vc->vc_intensity = (v & 8 >> 4) + 1;
+   else if (vc->vc_unblinking)
+   vc->vc_blink = v & 8 >> 4;
 
return i;
 }
@@ -1695,10 +1691,10 @@ static void csi_m(struct vc_data *vc)
vc->vc_reverse = 0;
break;
case 38:
-   i = vc_t416_color(vc, i, rgb_foreground);
+   i = vc_t416_color(vc, i, 0);
break;
case 48:
-   i = vc_t416_color(vc, i, rgb_background);
+   i = vc_t416_color(vc, i, 4);
break;
case 39:
vc->vc_color = (vc->vc_def_color & 0x0f) |
-- 
2.18.0

--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 0/6] vt: no blinking on console, 256/24-bit color improvements

[Adam Borowski]

Hi!
Here's a patchset with two entangled improvements:

* it'd be good to get rid of blinking where possible.  Even CGA (thus VGA)
  allows disabling it, rendering such characters with a bright background
  instead.
* due to my error, 256-color mode uses a much darker palette for conversion,
  resulting in behaving inconsistently with 24-bit mode.

The new code uses bright backgrounds when possible, enabled with \e[100m or
\e[48;m.

Despite the whole idea following a VGA capability, this patchset doesn't
change vgacon yet, just fbcon.  The reason being: ~80% of x86 users have an
nVidia chip, which means nouveau or nvidia-proprietary.  Nouveau implies
fbcon, nvidia-proprietary fails to properly restore text flags (as evidenced
by 512 glyph mode turning to 256 on switch from graphics).  You don't care
about the proprietary driver, but let's not break it pointlessly, and as
both nVidia cards I own work only with nouveau, I don't want to touch what I
can't test.

Thus, let's enable unblinking on fbcon for now.  We can flip that bit (in
register 0x10) later.

This fixes the display of catimg and similar tools.


Diffstat:
 drivers/tty/vt/vt.c  | 56 
+---
 drivers/video/fbdev/core/fbcon.c |  1 +
 include/linux/console_struct.h   |  4 ++--
 3 files changed, 32 insertions(+), 29 deletions(-)

-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢠⠒⠀⣿⡁ A dumb species has no way to open a tuna can.
⢿⡄⠘⠷⠚⠋⠀ A smart species invents a can opener.
⠈⠳⣄ A master species delegates.

[PATCH 3/3] vt: selection: take screen contents from uniscr if available

[Adam Borowski]

This preserves whatever was written even if we can't currently display the
given glyph.  Mouse paste won't corrupt any character of wcwidth() == 1
anymore.

Note that for now uniscr doesn't get allocated until something reads
/dev/vcsuN for that console, making this code dormant for most users.

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/selection.c | 11 +++
 drivers/tty/vt/vt.c| 10 ++
 include/linux/selection.h  |  1 +
 3 files changed, 18 insertions(+), 4 deletions(-)

diff --git a/drivers/tty/vt/selection.c b/drivers/tty/vt/selection.c
index 69ca337d3220..07496c711d7d 100644
--- a/drivers/tty/vt/selection.c
+++ b/drivers/tty/vt/selection.c
@@ -57,11 +57,13 @@ static inline void highlight_pointer(const int where)
complement_pos(sel_cons, where);
 }
 
-static u16
+static u32
 sel_pos(int n)
 {
+   if (use_unicode)
+   return screen_glyph_unicode(sel_cons, n / 2);
return inverse_translate(sel_cons, screen_glyph(sel_cons, n),
-   use_unicode);
+   0);
 }
 
 /**
@@ -90,7 +92,8 @@ static u32 inwordLut[]={
   0x07FE, /* lowercase */
 };
 
-static inline int inword(const u16 c) {
+static inline int inword(const u32 c)
+{
return c > 0x7f || (( inwordLut[c>>5] >> (c & 0x1F) ) & 1);
 }
 
@@ -167,7 +170,7 @@ int set_selection(const struct tiocl_selection __user *sel, 
struct tty_struct *t
struct tiocl_selection v;
char *bp, *obp;
int i, ps, pe, multiplier;
-   u16 c;
+   u32 c;
int mode;
 
poke_blanked_console();
diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
index 7fcb0ff2dccf..19da4c0b4b8e 100644
--- a/drivers/tty/vt/vt.c
+++ b/drivers/tty/vt/vt.c
@@ -4545,6 +4545,16 @@ u16 screen_glyph(struct vc_data *vc, int offset)
 }
 EXPORT_SYMBOL_GPL(screen_glyph);
 
+u32 screen_glyph_unicode(struct vc_data *vc, int n)
+{
+   struct uni_screen *uniscr = get_vc_uniscr(vc);
+
+   if (uniscr)
+   return uniscr->lines[n / vc->vc_cols][n % vc->vc_cols];
+   return inverse_translate(vc, screen_glyph(vc, n * 2), 1);
+}
+EXPORT_SYMBOL_GPL(screen_glyph_unicode);
+
 /* used by vcs - note the word offset */
 unsigned short *screen_pos(struct vc_data *vc, int w_offset, int viewed)
 {
diff --git a/include/linux/selection.h b/include/linux/selection.h
index 067d2e99c79f..a8f5b97b216f 100644
--- a/include/linux/selection.h
+++ b/include/linux/selection.h
@@ -32,6 +32,7 @@ extern unsigned char default_blu[];
 
 extern unsigned short *screen_pos(struct vc_data *vc, int w_offset, int 
viewed);
 extern u16 screen_glyph(struct vc_data *vc, int offset);
+extern u32 screen_glyph_unicode(struct vc_data *vc, int offset);
 extern void complement_pos(struct vc_data *vc, int offset);
 extern void invert_screen(struct vc_data *vc, int offset, int count, int 
shift);
 
-- 
2.18.0

--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/3] vt: selection: handle storing of characters above U+FFFF

[Adam Borowski]

Those above U+10 get replaced with U+FFFD.

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/selection.c | 23 ++-
 1 file changed, 18 insertions(+), 5 deletions(-)

diff --git a/drivers/tty/vt/selection.c b/drivers/tty/vt/selection.c
index 34e7110f310d..69ca337d3220 100644
--- a/drivers/tty/vt/selection.c
+++ b/drivers/tty/vt/selection.c
@@ -116,8 +116,8 @@ static inline int atedge(const int p, int size_row)
return (!(p % size_row) || !((p + 2) % size_row));
 }
 
-/* stores the char in UTF8 and returns the number of bytes used (1-3) */
-static int store_utf8(u16 c, char *p)
+/* stores the char in UTF8 and returns the number of bytes used (1-4) */
+static int store_utf8(u32 c, char *p)
 {
if (c < 0x80) {
/*  0*** */
@@ -128,13 +128,26 @@ static int store_utf8(u16 c, char *p)
p[0] = 0xc0 | (c >> 6);
p[1] = 0x80 | (c & 0x3f);
return 2;
-   } else {
+   } else if (c < 0x1) {
/* 1110 10** 10** */
p[0] = 0xe0 | (c >> 12);
p[1] = 0x80 | ((c >> 6) & 0x3f);
p[2] = 0x80 | (c & 0x3f);
return 3;
-   }
+   } else if (c < 0x11) {
+   /* 0*** 10** 10** 10** */
+   p[0] = 0xf0 | (c >> 18);
+   p[1] = 0x80 | ((c >> 12) & 0x3f);
+   p[2] = 0x80 | ((c >> 6) & 0x3f);
+   p[3] = 0x80 | (c & 0x3f);
+   return 4;
+   } else {
+   /* outside Unicode, replace with U+FFFD */
+   p[0] = 0xef;
+   p[1] = 0xbf;
+   p[2] = 0xbd;
+   return 3;
+   }
 }
 
 /**
@@ -273,7 +286,7 @@ int set_selection(const struct tiocl_selection __user *sel, 
struct tty_struct *t
sel_end = new_sel_end;
 
/* Allocate a new buffer before freeing the old one ... */
-   multiplier = use_unicode ? 3 : 1;  /* chars can take up to 3 bytes */
+   multiplier = use_unicode ? 4 : 1;  /* chars can take up to 4 bytes */
bp = kmalloc_array((sel_end - sel_start) / 2 + 1, multiplier,
   GFP_KERNEL);
if (!bp) {
-- 
2.18.0

[PATCH 1/3] vt: don't reinvent min()

[Adam Borowski]

All the helper function saved us was a cast.

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/selection.c | 14 --
 1 file changed, 4 insertions(+), 10 deletions(-)

diff --git a/drivers/tty/vt/selection.c b/drivers/tty/vt/selection.c
index 90ea1cc52b7a..34e7110f310d 100644
--- a/drivers/tty/vt/selection.c
+++ b/drivers/tty/vt/selection.c
@@ -116,12 +116,6 @@ static inline int atedge(const int p, int size_row)
return (!(p % size_row) || !((p + 2) % size_row));
 }
 
-/* constrain v such that v <= u */
-static inline unsigned short limit(const unsigned short v, const unsigned 
short u)
-{
-   return (v > u) ? u : v;
-}
-
 /* stores the char in UTF8 and returns the number of bytes used (1-3) */
 static int store_utf8(u16 c, char *p)
 {
@@ -167,10 +161,10 @@ int set_selection(const struct tiocl_selection __user 
*sel, struct tty_struct *t
if (copy_from_user(&v, sel, sizeof(*sel)))
return -EFAULT;
 
-   v.xs = limit(v.xs - 1, vc->vc_cols - 1);
-   v.ys = limit(v.ys - 1, vc->vc_rows - 1);
-   v.xe = limit(v.xe - 1, vc->vc_cols - 1);
-   v.ye = limit(v.ye - 1, vc->vc_rows - 1);
+   v.xs = min_t(u16, v.xs - 1, vc->vc_cols - 1);
+   v.ys = min_t(u16, v.ys - 1, vc->vc_rows - 1);
+   v.xe = min_t(u16, v.xe - 1, vc->vc_cols - 1);
+   v.ye = min_t(u16, v.ye - 1, vc->vc_rows - 1);
ps = v.ys * vc->vc_size_row + (v.xs << 1);
pe = v.ye * vc->vc_size_row + (v.xe << 1);
 
-- 
2.18.0

--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 0/3] use unicode for vt mouse paste

[Adam Borowski]

Hi!
Based on Nicolas' nice work (in tty-next), let's avoid corrupting characters
that have been copy+pasted via mouse selection.  The uniscr array holds
their original identity even if they got mangled by glyph conversion.
The glyph conversion lossily turns similar-looking characters into a
representation, and everyone else into a replacement character.

There's no proper handling for CJK (yet?) but anything of wcwidth()==1 will
work fine.

The whole thing doesn't get enabled until something reads from /dev/vcsu for
that console, but let's test this code first before enabling it widely.


Diffstat:
 drivers/tty/vt/selection.c | 48 
+---
 drivers/tty/vt/vt.c| 10 ++
 include/linux/selection.h  |  1 +
 3 files changed, 40 insertions(+), 19 deletions(-)


-- 
// If you believe in so-called "intellectual property", please immediately
// cease using counterfeit alphabets.  Instead, contact the nearest temple
// of Amon, whose priests will provide you with scribal services for all
// your writing needs, for Reasonable And Non-Discriminatory prices.

Re: [PATCH 1/3] vt: avoid a VLA in the unicode screen scroll function

[Adam Borowski]

On Tue, Jul 17, 2018 at 09:02:40PM -0400, Nicolas Pitre wrote:
> The nr argument is typically small: most often nr == 1. However this
> could be abused with a very large explicit scroll in a resized screen.
> Make the code scroll lines one at a time in all cases to avoid the VLA.
> Anything smarter is most likely not warranted here.

Even though nr can be 32767 at most, your new version is O(nr*nr) for no
reason.  Instead of O(n) memory or O(n²) time, a variant of the original
that copies values one at a time would be shorter and faster.

> Requested-by: Kees Cook 
> Signed-off-by: Nicolas Pitre 
> ---
>  drivers/tty/vt/vt.c | 18 ++
>  1 file changed, 10 insertions(+), 8 deletions(-)
> 
> diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
> index 2d14bb195d..03e79f7787 100644
> --- a/drivers/tty/vt/vt.c
> +++ b/drivers/tty/vt/vt.c
> @@ -433,20 +433,22 @@ static void vc_uniscr_scroll(struct vc_data *vc, 
> unsigned int t, unsigned int b,
>  
>   if (uniscr) {
>   unsigned int s, d, rescue, clear;
> - char32_t *save[nr];
>  
>   s = clear = t;
> - d = t + nr;
> - rescue = b - nr;
> + d = t + 1;
> + rescue = b - 1;
>   if (dir == SM_UP) {
>   swap(s, d);
>   swap(clear, rescue);
>   }
> - memcpy(save, uniscr->lines + rescue, nr * sizeof(*save));
> - memmove(uniscr->lines + d, uniscr->lines + s,
> - (b - t - nr) * sizeof(*uniscr->lines));
> - memcpy(uniscr->lines + clear, save, nr * sizeof(*save));
> - vc_uniscr_clear_lines(vc, clear, nr);
> + while (nr--) {
> + char32_t *tmp;
> + tmp = uniscr->lines[rescue];
> + memmove(uniscr->lines + d, uniscr->lines + s,
> + (b - t - 1) * sizeof(*uniscr->lines));
> + uniscr->lines[clear] = tmp;
> + vc_uniscr_clear_lines(vc, clear, 1);
> + }
>   }
>  }

What the function does is rotating an array (slice [t..b) here), by nr if
SM_DOWN or by -nr ie (b - t - nr) if SM_UP.  A nice problem that almost every
"code interview questions" book includes :)

Please say if you don't have time for such games, I've just refreshed what's
a good answer. :þ


Meow.
-- 
// If you believe in so-called "intellectual property", please immediately
// cease using counterfeit alphabets.  Instead, contact the nearest temple
// of Amon, whose priests will provide you with scribal services for all
// your writing needs, for Reasonable And Non-Discriminatory prices.
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] .gitignore: add ZSTD-compressed files

[Adam Borowski]

For now, that's arch/x86/boot/compressed/vmlinux.bin.zst but probably more
will come, thus let's be consistent with all other compressors.

Signed-off-by: Adam Borowski 
---
 .gitignore | 1 +
 1 file changed, 1 insertion(+)

diff --git a/.gitignore b/.gitignore
index 97ba6b79834c..0d09cf1c053c 100644
--- a/.gitignore
+++ b/.gitignore
@@ -42,6 +42,7 @@
 *.tab.[ch]
 *.tar
 *.xz
+*.zst
 Module.symvers
 modules.builtin
 
-- 
2.18.0

Re: [PATCH v3 1/3] vt: preserve unicode values corresponding to screen characters

[Adam Borowski]

On Wed, Jul 11, 2018 at 01:39:56PM -0700, Kees Cook wrote:
> On Wed, Jul 11, 2018 at 2:18 AM, Greg Kroah-Hartman
>  wrote:
> > On Tue, Jul 10, 2018 at 05:52:01PM -0700, Kees Cook wrote:
> >> On Tue, Jun 26, 2018 at 8:56 PM, Nicolas Pitre  
> >> wrote:
> >> > +++ b/drivers/tty/vt/vt.c
> >> > [...]
> >> > +static void vc_uniscr_scroll(struct vc_data *vc, unsigned int t, 
> >> > unsigned int b,
> >> > +enum con_scroll dir, unsigned int nr)
> >> > +{
> >> > +   struct uni_screen *uniscr = get_vc_uniscr(vc);
> >> > +
> >> > +   if (uniscr) {
> >> > +   unsigned int s, d, rescue, clear;
> >> > +   char32_t *save[nr];
> >>
> >> Can you adjust this to avoid the VLA here? I've almost gotten all VLAs
> >> removed from the kernel[1], and this is introducing a new one. :)
> >
> Yup, that's fine. (It's how I noticed it: linux-next VLA build tests.)
> I'm just hoping it can get solved before the merge window opens. :)

This one is actually nasty: max console height is 32767, if you resize it
that big then issue a large scroll request, boom it goes.

> There are still a bunch of VLAs I'm chipping away at, but this was a
> newly added one, so I was hoping Nicolas (when he's back from
> vacation) will have ideas on how to best avoid it.

Nicolas: what about just moving line pointers one at a time?  Rotating an
array slice in-place isn't that slow -- and optimizing that much when
reasonable sizes don't exceed 100ish (depending on how good your eyes are)
is quite ridiculous.  Thanks to your change, we don't need to move actual
contents, just line pointers -- that's fast enough.


Meow!
-- 
// If you believe in so-called "intellectual property", please immediately
// cease using counterfeit alphabets.  Instead, contact the nearest temple
// of Amon, whose priests will provide you with scribal services for all
// your writing needs, for Reasonable And Non-Discriminatory prices.
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH resend] scripts: teach extract-vmlinux about LZ4 and ZSTD

[Adam Borowski]

Note that the LZ4 signature is different than that of modern LZ4 as we
use the "legacy" format which suffers from some downsides like inability
to disable compression.

Signed-off-by: Adam Borowski 
---
The first time this was sent I managed to screw up both the subject and
scissors line, so it's likely whatever filter you're using didn't notice
the patch inside a thread.

ZSTD compression for vmlinux is not yet in, but being able to unpack it
can't hurt.  LZ4 is in for quite a while.


 scripts/extract-vmlinux | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/scripts/extract-vmlinux b/scripts/extract-vmlinux
index 5061abcc2540..e6239f39abad 100755
--- a/scripts/extract-vmlinux
+++ b/scripts/extract-vmlinux
@@ -57,6 +57,8 @@ try_decompress '\3757zXZ\000' abcde unxz
 try_decompress 'BZh'  xybunzip2
 try_decompress '\135\0\0\0'   xxx   unlzma
 try_decompress '\211\114\132' xy'lzop -d'
+try_decompress '\002!L\030'   xxx   'lz4 -d'
+try_decompress '(\265/\375'   xxx   unzstd
 
 # Bail out:
 echo "$me: Cannot find vmlinux." >&2
-- 
2.18.0

Re: [PATCH v2 0/4] have the vt console preserve unicode characters

[Adam Borowski]

On Wed, Jun 20, 2018 at 10:59:08PM -0400, Nicolas Pitre wrote:
> On Thu, 21 Jun 2018, Adam Borowski wrote:
> 
> > On Tue, Jun 19, 2018 at 11:34:34AM -0400, Nicolas Pitre wrote:
> > > On Tue, 19 Jun 2018, Adam Borowski wrote:
> > > > Thus, it'd be nice to use the structure you add to implement full 
> > > > Unicode
> > > > range for the vast majority of people.  This includes even U+2800..FF.  
> > > > :)

> > > If the core console code makes the switch to full unicode then yes, that 
> > > would be the way to go to maintain backward compatibility. However 
> > > vgacon users would see a performance drop when switching between VT's 
> > > and we used to brag about how fast the Linux console used to be 20 years 
> > > ago. Does it still matter today?
> 
> > * VT switch
> > * scrollback
> > 
> > The last two cases are initiated by the user, and within human reaction time
> > you need to convert usually 2000 -- up to 20k-ish -- characters.  The
> > conversion is done by a 3-level array.  I think a ZX Spectrum can handle
> > this fine without a visible slowdown.
> 
> In the scrollback case, currently each driver is doing its own thing. 
> The vgacon driver is probably the most efficient as it only moves the 
> base memory register around without copying anything at all. And that 
> part doesn't have to change.

As long as the data is still in video memory, yeah.  Soft scrollback is not
yet the default, because some userspace tools assume vt switch clears
scrollback and do so for security reasons.  All known tools that do so have
been fixed (at least in Debian), but as you can run new kernels with
arbitrarily old userspace, it's better to wait a bit longer before switching
to something effectively identical to soft scrollback.  Failure mode: after
logout, you can scroll back to the supposedly cleared content of the old
session.

Your code avoids this, at the cost of losing data about anything
representable by the currently loaded charset for anything inside
scrollback.

But in the near future, it'd be good to have soft scrollback work the same
for all drivers.

> > Right, let's see if your patchset gets okayed before building atop it.
> 
> May I add your ACK to it?

I don't believe I'm knowledgeful nor active enough in this part for my ACKs
to be meaningful.  On the other hand, I've analyzed your patchset long
enough to see no problems with it, thus if you have an use for my tags, then
sure, you have my ACK.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ There's an easy way to tell toy operating systems from real ones.
⣾⠁⢰⠒⠀⣿⡁ Just look at how their shipped fonts display U+1F52B, this makes
⢿⡄⠘⠷⠚⠋⠀ the intended audience obvious.  It's also interesting to see OSes
⠈⠳⣄ go back and forth wrt their intended target.

Re: [PATCH v2 0/4] have the vt console preserve unicode characters

[Adam Borowski]

On Wed, Jun 20, 2018 at 10:21:37PM -0400, Dave Mielke wrote:
> [quoted lines by Adam Borowski on 2018/06/21 at 03:43 +0200]
> 
> >It's meant for displaying braille to _sighted_ people.  And in real world,
> >the main [ab]use is a way to show images that won't get corrupted by
> >proportional fonts. :-þ
> 
> It's not abuse at all. I often use U+28xx to show sighted people what the
> braille for something looks like. I often need to do this when, for example, I
> need them to comapre what I'm showing them to what's on an actual braille
> display. U+28xx is the only way for me to do this without a lengthy 
> description
> containing sequences of dot number combinations.

What you describe is the intended use.  Abuse is when people use these
glyphs to write text like this:

⡎⠉⠂⠠⠤⡀⣄⠤⡀⠀⠀⠀⡄⠀⡄⡠⠤⡀⡄⠀⡄⠀⠀⠀⣄⠤⡀⡠⠤⡀⠠⠤⡀⡠⠤⡇⠀⠀⠀⠤⡧⠄⣇⠤⡀⠠⡅⠀⡠⠤⠄⠎⢉⠆
⠣⠤⠂⠪⠭⠇⠇⠀⠇⠀⠀⠀⠨⠭⠃⠣⠤⠃⠣⠤⠃⠀⠀⠀⠇⠀⠀⠫⠭⠁⠪⠭⠇⠣⠤⠇⠣⠄⠇⠀⠇⠀⠣⠀⠬⠭⠂⠀⠅⠀

(Not sure if you're completely blind or merely very weakly sighted; if the
former, this is my way to show you how actual Latin letters look like,
without a lengthy description of letter shapes. :) )

or for graphs.  Here's commits per UTC hour of day:

⡀⠀⠀⢀⣴⣤⣴⣶⣶⣶⣾⣦
⣿⣷⣾⣿

git log --pretty=format:'%at'|
perl -pe 'use integer;/^(\d+)$/ or die;$_=$1/3600%24 ."\n"'|
sort -n|uniq -c|cut -c3-7|braillegraph -y 8

or arbitrary images, like my .sig in all my mails in this thread.

But your patch set doesn't special-case braille in any way, thus allowing
such abuse to work on the console is merely an unintended side effect.

> >The primary users would be:
> >* people who want symbols uncorrupted (especially if their language uses a
> >  non-latin script)
> >* CJK people (as discussed below)
> 
> Again, that's not true. Why aren't braille users included in this list? After
> all, it's we who motivated this enhancement. I guess actual blind people
> mustn't count just because there are relatively fewer of us. :-(

Well, I meant users of Unicode display fonts, ie, _additional_ functionality
that's not yet coded but would rely on this patchset.  What you guys want is
already included.

The reason I'm raising this issue now is because if the Unicode struct would
be the primary one, there's no point in keeping vc_data in addition to
uni_screen.  And that would require designing the struct well from the
start, to avoid unnecessary changes in the future.

But then, taking a bitmask from that 32-bit value wouldn't be a big change
-- you already take variously 8 or 9 bits out of a 16-bit field, depending
on 256 vs 512 glyph mode.

The other point is a quite pointless assumption that existing scrollback is
"optimized".  Even vgacon mostly uses software scrollback these days, as the
amount of VGA display memory is really small.

I don't know much about console display drivers in general, though, and it
looks like most of them are unmaintained (just noticed that sisusb for
example hasn't seen a maintainer action for 13 years, and that person's
domain expired in 2006).


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ There's an easy way to tell toy operating systems from real ones.
⣾⠁⢰⠒⠀⣿⡁ Just look at how their shipped fonts display U+1F52B, this makes
⢿⡄⠘⠷⠚⠋⠀ the intended audience obvious.  It's also interesting to see OSes
⠈⠳⣄ go back and forth wrt their intended target.

Re: [PATCH v2 0/4] have the vt console preserve unicode characters

[Adam Borowski]

On Tue, Jun 19, 2018 at 11:34:34AM -0400, Nicolas Pitre wrote:
> On Tue, 19 Jun 2018, Adam Borowski wrote:
> > Thus, it'd be nice to use the structure you add to implement full Unicode
> > range for the vast majority of people.  This includes even U+2800..FF.  :)
> 
> Be my guest if you want to use this structure. As for U+2800..FF, like I 
> said earlier, this is not what most people use when communicating, so it 
> is of little interest even to blind users except for displaying native 
> braille documents, or showing off. ;-)

It's meant for displaying braille to _sighted_ people.  And in real world,
the main [ab]use is a way to show images that won't get corrupted by
proportional fonts. :-þ

> If the core console code makes the switch to full unicode then yes, that 
> would be the way to go to maintain backward compatibility. However 
> vgacon users would see a performance drop when switching between VT's 
> and we used to brag about how fast the Linux console used to be 20 years 
> ago. Does it still matter today?

I've seen this slowness.  A long time ago, on a server that someone gave an
_ISA_ graphics card (it was an old machine, and it was 1.5 decades ago). 
Indeed, switching VTs took around a second.  But this was drawing speed, not
Unicode conversion.

There are three cases when a character can enter the screen:
* being printed by the tty.  This is the only case not sharply rate-limited.
  It already has to do the conversion.  If we eliminate the old struct, it
  might even be a speed-up when lots of text gets blasted to a non-active
  VT.
* VT switch
* scrollback

The last two cases are initiated by the user, and within human reaction time
you need to convert usually 2000 -- up to 20k-ish -- characters.  The
conversion is done by a 3-level array.  I think a ZX Spectrum can handle
this fine without a visible slowdown.

> > > I'm a prime user of this feature, as well as the BRLTTY maintainer Dave 
> > > Mielke
> > > who implemented support for this in BRLTTY. There is therefore a vested
> > > interest in maintaining this feature as necessary. And this received
> > > extensive testing as well at this point.
> > 
> > So, you care only about people with faulty wetware.  Thus, it sounds like
> > work that benefits sighted people would need to be done by people other than
> > you. 
> 
> Hard for me to contribute more if I can't enjoy the result.

Obviously.

The primary users would be:
* people who want symbols uncorrupted (especially if their language uses a
  non-latin script)
* CJK people (as discussed below)

It could also simplify the life for distros -- less required configuration:
a single font needed for currently supported charsets together has mere
~1000 glyphs, at 8x16 that's 16000 bytes (+ mapping).  Obviously for CJK
that's more.
 
> > So I'm only mentioning possible changes; they could possibly go after
> > your patchset goes in:
> > 
> > A) if memory is considered to be at premium, what about storing only one
> >32-bit value, masked 21 bits char 11 bits attr?  On non-vgacon, there's
> >no reason to keep the old structures.
> 
> Absolutely. As soon as vgacon is officially relegated to second class 
> citizen i.e. perform the glyph translation each time it requires 
> a refresh instead of dictating how the core console code works then the 
> central glyph buffer can go.

Per the analysis above, on-the-fly translation is so unobtrusive that it
shouldn't be a problem.

> > B) if being this frugal wrt memory is ridiculous today, what about instead
> >going for 32 bits char (wasteful) 32 bits attr?  This would be much nicer
> >15 bit fg color + 15 bit bg color + underline + CJK or something.
> > You already triple memory use; variant A) above would reduce that to 2x,
> > variant B) to 4x.
> 
> You certainly won't find any objections from me.

Right, let's see if your patchset gets okayed before building atop it.
 
> In the mean time, both systems may work in parallel for a smooth 
> transition.

Sounds like a good idea.


WRT support for fonts >512 glyphs: I talked to a Chinese hacker (log
starting at 15:32 on https://irclog.whitequark.org/linux-sunxi/2018-06-19),
she said there are multiple popular non-mainline patchsets implementing CJK
on console.  None of them got accepted because of pretty bad code like
https://github.com/Gentoo-zh/linux-cjktty/commit/b6160f85ef5bc5c2cae460f6c0a1aba3e417464f
but getting this done cleanly would require just:
* your patchset here
* console driver using the Unicode structure
* loading such larger fonts (the one in cjktty is built-in)
* double-width characters in vt.c


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ There's an easy way to tell toy operating systems from real ones.
⣾⠁⢰⠒⠀⣿⡁ Just look at how their shipped fonts display U+1F52B, this makes
⢿⡄⠘⠷⠚⠋⠀ the intended audience obvious.  It's also interesting to see OSes
⠈⠳⣄ go back and forth wrt their intended target.

Re: [PATCH v2 0/4] have the vt console preserve unicode characters

[Adam Borowski]

On Tue, Jun 19, 2018 at 09:52:13AM -0400, Dave Mielke wrote:
> [quoted lines by Adam Borowski on 2018/06/19 at 15:09 +0200]
> 
> >You're thinking small.  That 256 possible values for Braille are easily
> >encodable within the 512-glyph space (256 char + stolen fg brightness bit,
> >another CGA peculiarity).  
> 
> Not at all. We braille users, especially when working with languages other 
> than
> English, need more than 256 non-braille characters. Even for those who can 
> live
> with just 256 non-braille characters, it's still a major pain having to come 
> up
> with a usable braille-capable font for every needed 256 non-braille characters
> set. I can assure you, as an actual braille user, that the limitation has been
> a very long-standing problem and it's a great relief that it's finally been
> resolved.

Ok, I thought Braille is limited to 2x3 dots, recently extended to 2x4;
thanks for the explanation!

But those of us who are sighted, are greatly annoyed by characters that are
usually taken for granted being randomly missing.  For example, no console
font+mapping shipped with Debian supports ░▒▓▄▀ (despite them being a
commonly used part of the BIOS charset), so unless you go out of your way to
beat them back they'll be corrupted (usually into ♦).  Then Perl6 wants ｢｣⚛,
and so on.  All these problems would instantly disappear the moment console
sheds the limit of 256/512 glyphs.

So I'm pretty happy seeing this patch set.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ There's an easy way to tell toy operating systems from real ones.
⣾⠁⢰⠒⠀⣿⡁ Just look at how their shipped fonts display U+1F52B, this makes
⢿⡄⠘⠷⠚⠋⠀ the intended audience obvious.  It's also interesting to see OSes
⠈⠳⣄ go back and forth wrt their intended target.

Re: [PATCH v2 0/4] have the vt console preserve unicode characters

[Adam Borowski]

On Sun, Jun 17, 2018 at 03:07:02PM -0400, Nicolas Pitre wrote:
> The vt code translates UTF-8 strings into glyph index values and stores
> those glyph values directly in the screen buffer. Because there can only
> be at most 512 glyphs, it is impossible to represent most unicode
> characters, in which case a default glyph (often '?') is displayed
> instead. The original unicode value is then lost.
> 
> The 512-glyph limitation is inherent to VGA displays, but users of
> /dev/vcs* shouldn't have to be restricted to a narrow unicode space from
> lossy screen content because of that. This is especially true for
> accessibility applications such as BRLTTY that rely on /dev/vcs to rander
> screen content onto braille terminals.

You're thinking small.  That 256 possible values for Braille are easily
encodable within the 512-glyph space (256 char + stolen fg brightness bit,
another CGA peculiarity).  Your patchset, though, can be used for proper
Unicode support for the rest of us.

The 256/512 value limitation applies only to CGA-compatible hardware; these
days this means vgacon.  But most people use other drivers.  Nouveau forces
graphical console, on arm* there's no such thing as VGA[1], etc.

Thus, it'd be nice to use the structure you add to implement full Unicode
range for the vast majority of people.  This includes even U+2800..FF.  :)

> This patch series introduces unicode support to /dev/vcs* devices,
> allowing full unicode access from userspace to the vt console which
> can, amongst other purposes, appropriately translate actual unicode
> screen content into braille. Memory is allocated, and possible CPU
> overhead introduced, only if /dev/vcsu is read at least once.

What about doing so if any updated console driver is loaded?  Possibly, once
the vt in question has been switched to (>99% people never see anything but
tty1 during boot-up, all others showing nothing but getty).  Or perhaps the
moment any non-ASCII character is output to the given vt.

If memory usage is a concern, it's possible to drop the old structure and
convert back only in the rare case the driver is unloaded; reads of old-
style /dev/vc{s,sa}\d* are not speed-critical thus can use conversion on the
fuly.  Unicode takes only 21 bits out of 32 you allocate, that's plenty of
space for attributes: they currently take 8 bits; naive way gives us free 3
bits that could be used for additional attributes.

Especially underline is in common use these days; efficient support for CJK
would also use one bit to mark left/right half.  And it's decades overdue to
drop blink, which is not even supported by anything but vgacon anyway!
(Graphical drivers tend to show this bit as bright background, but don't
accept SGR codes other thank blink[2].)

> I'm a prime user of this feature, as well as the BRLTTY maintainer Dave Mielke
> who implemented support for this in BRLTTY. There is therefore a vested
> interest in maintaining this feature as necessary. And this received
> extensive testing as well at this point.

So, you care only about people with faulty wetware.  Thus, it sounds like
work that benefits sighted people would need to be done by people other than
you.  So I'm only mentioning possible changes; they could possibly go after
your patchset goes in:

A) if memory is considered to be at premium, what about storing only one
   32-bit value, masked 21 bits char 11 bits attr?  On non-vgacon, there's
   no reason to keep the old structures.
B) if being this frugal wrt memory is ridiculous today, what about instead
   going for 32 bits char (wasteful) 32 bits attr?  This would be much nicer
   15 bit fg color + 15 bit bg color + underline + CJK or something.
You already triple memory use; variant A) above would reduce that to 2x,
variant B) to 4x.

Considering that modern machines can draw complex scenes of several
megapixels 60 times a second, it could be reasonable to drop the complexity
of two structures even on vgacon: converting characters on the fly during vt
switch is beyond notice on any hardware Linux can run.

> This is also available on top of v4.18-rc1 here:
> 
>   git://git.linaro.org/people/nicolas.pitre/linux vt-unicode



Meow!

[1]. config VGA_CONSOLE
  depends on !4xx && !PPC_8xx && !SPARC && !M68K && !PARISC &&  !SUPERH && \
  (!ARM || ARCH_FOOTBRIDGE || ARCH_INTEGRATOR || ARCH_NETWINDER) && \
  !ARM64 && !ARC && !MICROBLAZE && !OPENRISC && !NDS32 && !S390

[2]. Sounds like an easy improvement; not so long ago I added "\e[48;5;m",
"\e[48;2;m" and "\e[100m" which could be improved when on unblinking drivers.
Heck, even VGA can be switched to unblinking by flipping bit 3 of the
Attribute Mode Control Register -- like we already flip foreground
brightness when 512 glyphs are needed.
-- 
⢀⣴⠾⠻⢶⣦⠀ There's an easy way to tell toy operating systems from real ones.
⣾⠁⢰⠒⠀⣿⡁ Just look at how their shipped fonts display U+1F52B, this makes
⢿⡄⠘⠷⠚⠋⠀ the intended audience obvious.  It's also interesting to see OSes
⠈⠳⣄ go 

On Tue, Apr 24, 2018 at 11:08:34AM +0200, Paul Menzel wrote:

[Adam Borowski]

'ere you go.  KERNEL_ZSTD is not in mainline yet but knowing its magic
can't hurt -- especially that scripts may be out of sync with an installed
kernel.  LZ4 is in since 3.11.

--8<8<8<8<8<8<8<8<8<8<8<8<--
>From 30886e965e7aeae8d3729c4bacf614a19e103cea Mon Sep 17 00:00:00 2001
From: Adam Borowski 
Date: Wed, 25 Apr 2018 02:29:18 +0200
Subject: [PATCH] scripts: teach extract-vmlinux about LZ4 and ZSTD

Note that the LZ4 signature is different than that of modern LZ4 as we
use the "legacy" format which suffers from some downsides like inability
to disable compression.

Signed-off-by: Adam Borowski 
---
 scripts/extract-vmlinux | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/scripts/extract-vmlinux b/scripts/extract-vmlinux
index 5061abcc2540..e6239f39abad 100755
--- a/scripts/extract-vmlinux
+++ b/scripts/extract-vmlinux
@@ -57,6 +57,8 @@ try_decompress '\3757zXZ\000' abcde unxz
 try_decompress 'BZh'  xybunzip2
 try_decompress '\135\0\0\0'   xxx   unlzma
 try_decompress '\211\114\132' xy'lzop -d'
+try_decompress '\002!L\030'   xxx   'lz4 -d'
+try_decompress '(\265/\375'   xxx   unzstd
 
 # Bail out:
 echo "$me: Cannot find vmlinux." >&2
-- 
2.17.0

Re: How to disable Linux kernel self-extraction (KERNEL_GZIP, KERNEL_BZIP2, …)?

[Adam Borowski]

On Tue, Apr 24, 2018 at 11:08:34AM +0200, Paul Menzel wrote:
> On 04/24/18 04:08, Adam Borowski wrote:
> > On Mon, Apr 23, 2018 at 07:02:30PM +0200, Pavel Machek wrote:
> > > > > > > I try to decrease boot time, and my system has an SSD and enough 
> > > > > > > space, so
> > > > > > > loading 18 instead of 12 MB doesn’t make a difference, but the
> > > > > > > self-extraction is noticeable. So, I like to disable it.
> > > > > > 
> > > > > > How long does GZIP extraction take on your hardware?
> > > > > 
> > > > > It’s hard to measure – at least I didn’t find a way to do so –, but 
> > > > > counting
> > > > > from the last GRUB message to the first message of Linux (with `quiet`
> > > > > removed from the the command line), it takes roughly *two* seconds.
> > 
> > I took a somewhat different approach: I recorded the output from grub+kernel
> > to ttyrec over serial line, and rigged ttyrec2ansi to output timestamp
> > difference from the last checkpoint every time an '\e' or '\n' is seen.
> > '\e' is important, as there's no other marking for when grub stops the
> > interactive phase and starts the actual boot.
> 
> Nice work. It’d be great, if you shared these patches, so others and I can
> reproduce it.

ttyrec2ansi.c attached.  To use: save the serial output as ttyrec (via the
eponymous tool, termrec, conversion from a similar format, etc), pipe
through modified ttyrec2ansi to a terminal, "less -R".


userland lz4:
diff --git a/lib/lz4.c b/lib/lz4.c
index 213b085..39d2cff 100644
--- a/lib/lz4.c
+++ b/lib/lz4.c
@@ -499,6 +499,7 @@ LZ4_FORCE_INLINE U32 LZ4_hashPosition(const void* const p, 
tableType_t const tab
 
 static void LZ4_putPositionOnHash(const BYTE* p, U32 h, void* tableBase, 
tableType_t const tableType, const BYTE* srcBase)
 {
+return;
 switch (tableType)
 {
 case byPtr: { const BYTE** hashTable = (const BYTE**)tableBase; 
hashTable[h] = p; return; }

Somehow this affects only /usr/bin/lz4 not /usr/bin/lz4c, which I did not
bother to fix but hacked via:

diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index f2014876405f..91534a801090 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -332,7 +332,7 @@ cmd_lzo = (cat $(filter-out FORCE,$^) | \
 
 quiet_cmd_lz4 = LZ4 $@
 cmd_lz4 = (cat $(filter-out FORCE,$^) | \
-   lz4c -l -c1 stdin stdout && $(call size_append, $(filter-out 
FORCE,$^))) > $@ || \
+   lz4 -l -c1 stdin stdout && $(call size_append, $(filter-out FORCE,$^))) 
> $@ || \
(rm -f $@ ; false)
 
 # U-Boot mkimage


I have serious doubts this approach is sound, but worked well enough at
least to spot the GRUB read slowness issue.

> > Turns out that, reading from SSD, grub is way way slower than it should take
> > normally.  The machine is old (AMD Phenom II X6 1055T), SSD is Crucial
> > CT240M500SSD1.
> 
> What firmware does the device (mainboard) have? Legacy BIOS or UEFI (or even
> coreboot ;-)). It’s my understanding, that GRUB does not have a native
> driver with the legacy BIOS and UEFI, and just uses the BIOS calls or the
> UEFI equivalent.

An old machine -- real BIOS.

> > I also have the zstd patch applied, which adds another data point.
> > 
> > The two "Loading XXX ..." lines come from grub, those timestamped within []
> > brackets from the kernel, 〈〉are ttyrec timestamps, ⤸ is wrapped lines.
> > 
> > 
> > zstd:
> > 
> > Loading Linux 4.17.0-rc2-debug-00025-gd426b0ba363d ...〈0.739823〉
> > ^MLoading initial ramdisk ...〈0.402010〉
> > ^M[0.00] Linux version 4.17.0-rc2-debug-00025-gd426b0ba363d ⤸
> > (kilobyte@umbar) (gcc version 7.3.0 (Debian 7.3.0-16)) #1 SMP Mon Apr 23⤸
> > 10:25:58 CEST 2018^M〈0.785922〉
> > 
> > gzip:
> > 
> > Loading Linux 4.17.0-rc2-debug-gz-00025-gd426b0ba363d ...〈0.724988〉
> > ^MLoading initial ramdisk ...〈0.357951〉
> > ^M[0.00] Linux version 4.17.0-rc2-debug-gz-00025-gd426b0ba363d ⤸
> > (kilobyte@umbar) (gcc version 7.3.0 (Debian 7.3.0-16)) #1 SMP Mon Apr 23 ⤸
> > 23:15:07 CEST 2018^M〈0.777977〉
> > 
> > lz4:
> > 
> > Loading Linux 4.17.0-rc2-debug-none-00025-gd426b0ba363d ...〈0.799969〉
> > ^MLoading initial ramdisk ...〈0.424959〉
> > ^M[0.00] Linux version 4.17.0-rc2-debug-lz4-00025-gd426b0ba363d ⤸
> > (kilobyte@umbar) (gcc version 7.3.0 (Debian 7.3.0-16)) #1 SMP Tue Apr 24 ⤸
> > 00:34:59 CEST 2018^M〈0.732925〉
> > 
> > zstd again:
> > 
> > Loading Linux 4.17.0-rc2-debug-00025-gd426b0ba363d ...〈0.728852〉
> > ^MLoading ini

Re: How to disable Linux kernel self-extraction (KERNEL_GZIP, KERNEL_BZIP2, …)?

[Adam Borowski]

On Mon, Apr 23, 2018 at 07:02:30PM +0200, Pavel Machek wrote:
> > > >>I try to decrease boot time, and my system has an SSD and enough space, 
> > > >>so
> > > >>loading 18 instead of 12 MB doesn’t make a difference, but the
> > > >>self-extraction is noticeable. So, I like to disable it.
> > > >
> > > >How long does GZIP extraction take on your hardware?
> > > 
> > > It’s hard to measure – at least I didn’t find a way to do so –, but 
> > > counting
> > > from the last GRUB message to the first message of Linux (with `quiet`
> > > removed from the the command line), it takes roughly *two* seconds.

I took a somewhat different approach: I recorded the output from grub+kernel
to ttyrec over serial line, and rigged ttyrec2ansi to output timestamp
difference from the last checkpoint every time an '\e' or '\n' is seen.
'\e' is important, as there's no other marking for when grub stops the
interactive phase and starts the actual boot.

Turns out that, reading from SSD, grub is way way slower than it should take
normally.  The machine is old (AMD Phenom II X6 1055T), SSD is Crucial
CT240M500SSD1.

I also have the zstd patch applied, which adds another data point.

The two "Loading XXX ..." lines come from grub, those timestamped within []
brackets from the kernel, 〈〉are ttyrec timestamps, ⤸ is wrapped lines.


zstd:

Loading Linux 4.17.0-rc2-debug-00025-gd426b0ba363d ...〈0.739823〉
^MLoading initial ramdisk ...〈0.402010〉
^M[0.00] Linux version 4.17.0-rc2-debug-00025-gd426b0ba363d ⤸
(kilobyte@umbar) (gcc version 7.3.0 (Debian 7.3.0-16)) #1 SMP Mon Apr 23⤸
10:25:58 CEST 2018^M〈0.785922〉
[0.00] Command line: 
BOOT_IMAGE=/sys/boot/vmlinuz-4.17.0-rc2-debug-00025-gd426b0ba363d⤸
 root=UUID=b7c38da9-ae84-4083-a1f8-6d7b4fc33961 ro rootflags=subvol=sys 
syscall.x32=y⤸
 console=tty0 console=ttyS0,115200n8 no_console_suspend^M〈0.020199〉

gzip:

Loading Linux 4.17.0-rc2-debug-gz-00025-gd426b0ba363d ...〈0.724988〉
^MLoading initial ramdisk ...〈0.357951〉
^M[0.00] Linux version 4.17.0-rc2-debug-gz-00025-gd426b0ba363d ⤸
(kilobyte@umbar) (gcc version 7.3.0 (Debian 7.3.0-16)) #1 SMP Mon Apr 23 ⤸
23:15:07 CEST 2018^M〈0.777977〉
[0.00] Command line: 
BOOT_IMAGE=/sys/boot/vmlinuz-4.17.0-rc2-debug-gz-00025-gd426b0ba363d⤸
 root=UUID=b7c38da9-ae84-4083-a1f8-6d7b4fc33961 ro rootflags=subvol=sys 
syscall.x32=y⤸
 console=tty0 console=ttyS0,115200n8 no_console_suspend^M〈0.020117〉

lz4:

Loading Linux 4.17.0-rc2-debug-none-00025-gd426b0ba363d ...〈0.799969〉
^MLoading initial ramdisk ...〈0.424959〉
^M[0.00] Linux version 4.17.0-rc2-debug-lz4-00025-gd426b0ba363d ⤸
(kilobyte@umbar) (gcc version 7.3.0 (Debian 7.3.0-16)) #1 SMP Tue Apr 24 ⤸
00:34:59 CEST 2018^M〈0.732925〉
[0.00] Command line: 
BOOT_IMAGE=/sys/boot/vmlinuz-4.17.0-rc2-debug-lz4-00025-gd426b0ba363d⤸
 root=UUID=b7c38da9-ae84-4083-a1f8-6d7b4fc33961 ro rootflags=subvol=sys 
syscall.x32=y⤸
 console=tty0 console=ttyS0,115200n8 no_console_suspend^M〈0.021019〉

zstd again:

Loading Linux 4.17.0-rc2-debug-00025-gd426b0ba363d ...〈0.728852〉
^MLoading initial ramdisk ...〈0.399968〉
^M[0.00] Linux version 4.17.0-rc2-debug-00025-gd426b0ba363d ⤸
(kilobyte@umbar) (gcc version 7.3.0 (Debian 7.3.0-16)) #1 SMP Mon Apr 23 ⤸
10:25:58 CEST 2018^M〈0.786964〉
[0.00] Command line: 
BOOT_IMAGE=/sys/boot/vmlinuz-4.17.0-rc2-debug-00025-gd426b0ba363d⤸
 root=UUID=b7c38da9-ae84-4083-a1f8-6d7b4fc33961 ro rootflags=subvol=sys 
syscall.x32=y⤸
 console=tty0 console=ttyS0,115200n8 no_console_suspend^M〈0.020071〉

lz4 rigged for no compression:

Loading Linux 4.17.0-rc2-debug-none-00025-gd426b0ba363d-dirty ...〈0.479834〉
^MLoading initial ramdisk ...〈2.246985〉
^M[0.00] Linux version 4.17.0-rc2-debug-none-00025-gd426b0ba363d-dirty ⤸
(kilobyte@umbar) (gcc version 7.3.0 (Debian 7.3.0-16)) #5 SMP Tue Apr 24 ⤸
02:57:18 CEST 2018^M〈0.711949〉
[0.00] Command line: 
BOOT_IMAGE=/sys/boot/vmlinuz-4.17.0-rc2-debug-none-00025-gd426b0ba363d-dirty⤸
 root=UUID=b7c38da9-ae84-4083-a1f8-6d7b4fc33961 ro rootflags=subvol=sys 
syscall.x32=y⤸
 console=tty0 console=ttyS0,115200n8 no_console_suspend^M〈0.021902〉

Sizes of relevant files:

14826134 initrd.img-4.17.0-rc2-debug-00025-gd426b0ba363d(zstd)
14826352 initrd.img-4.17.0-rc2-debug-gz-00025-gd426b0ba363d
14826909 initrd.img-4.17.0-rc2-debug-lz4-00025-gd426b0ba363d
14826761 initrd.img-4.17.0-rc2-debug-none-00025-gd426b0ba363d-dirty
 6567408 vmlinuz-4.17.0-rc2-debug-00025-gd426b0ba363d   (zstd)
 7230960 vmlinuz-4.17.0-rc2-debug-gz-00025-gd426b0ba363d
 8775152 vmlinuz-4.17.0-rc2-debug-lz4-00025-gd426b0ba363d
27821552 vmlinuz-4.17.0-rc2-debug-none-00025-gd426b0ba363d-dirty
(I did not alter initrd compression, which is zstd in all cases).

> > So yes, looks like uncompressed kernel image may be good idea.

Seems like the time to actually read this far bigger file from the disk
using grub's inefficient way, takes longer than the gains from faster
decompression.  You can eliminate the decompression step altogether by
av

Re: [PATCH 1/2] lib: vsprintf: Implement %pCOW

[Adam Borowski]

On Sun, Apr 01, 2018 at 10:56:21AM +0200, Richard Weinberger wrote:
> + .cow_lines = {
> + "\\   ^__^",
> + " \\  (oo)\\___",
> + "(__)\\   )\\/\\",
> + "||w |",
> + "|| ||",

Userspace cowsay(6) knows of different cows.  What about using those as a
visual indication of message level?


Meow! err... Moo!
-- 
ᛊᚨᚾᛁᛏᚣ᛫ᛁᛊ᛫ᚠᛟᚱ᛫ᚦᛖ᛫ᚹᛖᚨᚲ

Re: [RFC] new SYSCALL_DEFINE/COMPAT_SYSCALL_DEFINE wrappers

[Adam Borowski]

On Fri, Mar 30, 2018 at 12:58:02PM +0200, Ingo Molnar wrote:
> * John Paul Adrian Glaubitz  wrote:
> 
> > On 03/27/2018 12:40 PM, Linus Torvalds wrote:
> > > On Mon, Mar 26, 2018 at 4:37 PM, John Paul Adrian Glaubitz
> > >  wrote:
> > >>
> > >> What about a tarball with a minimal Debian x32 chroot? Then you can
> > >> install interesting packages you would like to test yourself.

> Here's the direct download link:
>   $ wget 
> https://people.debian.org/~glaubitz/chroots/debian-x32-unstable.tar.gz

> Seems to work fine here (on a distro kernel) even if I extract all the files 
> as a 
> non-root user and do:
> 
>   ~/s/debian-x32-unstable> fakechroot /usr/sbin/chroot . /usr/bin/dpkg -l  | 
> tail -2
> 
>   ERROR: ld.so: object 'libfakechroot.so' from LD_PRELOAD cannot be preloaded 
> (cannot open shared object file): ignored.
>   ii  util-linux:x32 2.31.1-0.5   x32  miscellaneous 
> system utilities
>   ii  zlib1g:x32 1:1.2.8.dfsg-5   x32  compression 
> library - runtime

> So that 'dpkg' instance appears to be running inside the chroot environment 
> and is 
> listing x32 installed packages.

> Although I did get this warning:
>   ERROR: ld.so: object 'libfakechroot.so' from LD_PRELOAD cannot be preloaded 
> (cannot open shared object file): ignored.
> Even with that warning, is still still a sufficiently complex test of x32 
> syscall 
> code paths?

Instead of mucking with fakechroot which would require installing its :x32
part inside the guest, or running the test as root, what about using any
random static binary?  For example, a shell like sash or bash-static would
have a decentish syscall coverage even by itself.

I've extracted sash from
http://ftp.ports.debian.org/debian-ports//pool-x32/main/s/sash/sash_3.8-4_x32.deb
and placed at https://angband.pl/tmp/sash.x32

$ sha256sum sash.x32 
de24097c859b313fa422af742b648c9d731de6b33b98cb995658d1da16398456  sash.x32

Obviously, you can compile a static binary that uses whatever syscalls you
want.  Without a native chroot, you can "gcc -mx32" although you'd need some
kind of libc unless your program is stand-alone.


It might be worth mentioning my "arch-test:
https://github.com/kilobyte/arch-test
Because of many toolchain pieces it needs, you want a prebuilt copy:
https://github.com/kilobyte/arch-test/releases/download/v0.10/arch-test_prebuilt_0.10.tar.xz
https://github.com/kilobyte/arch-test/releases/download/v0.10/arch-test_prebuilt_0.10.tar.xz.asc
-- while it has _extremely_ small coverage of syscalls (just write() and
_exit(), enough to check endianness and pointer width), concentrating on
instruction set inadequacies (broken SWP on arm, POWER7 vs POWER8, powerpc
vs powerpcspe, etc), it provides minimal test binaries for a wide range of
architectures.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢰⠒⠀⣿⡁ I was born a dumb, ugly and work-loving kid, then I got swapped on
⢿⡄⠘⠷⠚⠋⠀ the maternity ward.
⠈⠳⣄


signature.asc
Description: PGP signature

Re: [RESEND PATCH v2 0/2] Add support for ZSTD-compressed kernel

[Adam Borowski]

On Thu, Mar 22, 2018 at 12:09:45PM +0100, René Rebe wrote:
> Should this currently just work without any arch change on e.g.
> ppc64, sparc64 et al.? I could do a test build and boot if that is
> of any value, ...

Initrd: no reason it wouldn't work, although for anything related to the
boot process testing is a very good idea.  If you test, you'd really want to
apply that printk patch I posted, it's too easy to accidentally get a silent
fallback to gzip or uncompressed.  I wonder, perhaps such a printk could be
permanently added, at a low message priority?

Kernel itself: needs per-arch porting, but it's not advertised in kconfig
outside x86 so that's not a problem.  

If you are knowledgeful about bootloaders on ppc64, sparc64, etc, such
information would be great.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢠⠒⠀⣿⡁ A dumb species has no way to open a tuna can.
⢿⡄⠘⠷⠚⠋⠀ A smart species invents a can opener.
⠈⠳⣄ A master species delegates.

Re: [RESEND PATCH v2 0/2] Add support for ZSTD-compressed kernel

[Adam Borowski]

On Wed, Mar 21, 2018 at 06:29:41PM -0700, Nick Terrell wrote:
> This patch set adds support for a ZSTD-compressed kernel and ramdisk
> images in the kernel boot process. It only integrates the support with
> x86, though the first patch is generic to all architectures.

I'm running this patch set since October on amd64 armhf arm64, no
explosions -- besides the obvious when it's not applied yet I forget to
reconfigure initramfs-tools.  Which is getting tedious, so let's merge this
already. :)

I've tested initrd on all of these archs; here's a debug patch to check if
you're actually using it.

As for compressing the kernel itself: the second patch works as submitted
(ie, x86 only).  Porting it to other architectures is straightforward, but
eg. on arm, most boards use u-boot which insists on decompressing the kernel
by itself instead of passing control.  EFI/grub should work, but I haven't
tested that yet.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢠⠒⠀⣿⡁ A dumb species has no way to open a tuna can.
⢿⡄⠘⠷⠚⠋⠀ A smart species invents a can opener.
⠈⠳⣄ A master species delegates.

[PATCH] [NOT FOR MERGING] lib: Be noisy about used decompression method.

[Adam Borowski]

It's too easy to build the initrd with wrong options during testing, after
which it may silently work anyway.

Signed-off-by: Adam Borowski 
---
 lib/decompress.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/lib/decompress.c b/lib/decompress.c
index ab3fc90ffc64..556b9d916e10 100644
--- a/lib/decompress.c
+++ b/lib/decompress.c
@@ -78,7 +78,9 @@ decompress_fn __init decompress_method(const unsigned char 
*inbuf, long len,
break;
 
}
-   if (name)
+   if (name) {
*name = cf->name;
+   printk("Decompressing using %s.\n", *name);
+   }
return cf->decompressor;
 }
-- 
2.16.2

Re: [PATCH 1/2] vsprintf: distinguish between (null), (err) and (invalid) pointer derefs

[Adam Borowski]

On Wed, Mar 07, 2018 at 03:17:19PM +0200, Andy Shevchenko wrote:
> On Tue, 2018-03-06 at 19:11 +0100, Adam Borowski wrote:
> 
> Thanks for the patch, my comments below.

(Review snipped.)

It looks pretty obvious that it'd take a lot less of your time to roll new
patch[es] from scratch than to try to educate me wrt how you'd want to see
it done; thus I'll sit out this one.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢠⠒⠀⣿⡁ A dumb species has no way to open a tuna can.
⢿⡄⠘⠷⠚⠋⠀ A smart species invents a can opener.
⠈⠳⣄ A master species delegates.

Re: [PATCH 1/2] vsprintf: distinguish between (null), (err) and (invalid) pointer derefs

[Adam Borowski]

On Tue, Mar 06, 2018 at 09:22:17PM +0300, Alexey Dobriyan wrote:
> > +#define BAD_PTR_STRING(x) (!(x) ? "(null)" : IS_ERR(x) ? "(err)" : 
> > "(invalid)")
> 
> This is getting ridiculous.
> 
> Instead of simply printing a pointer as %08lx or %016llx, not only glibc
> (null) stupidity is propagated but expanded and "improved".

This is not about printing a pointer, this is about attempting to print an
object referenced by such a bad pointer.  Which leads to a crash: in
userspace, you get a segfault; in the kernel, at least in the case I tested,
the system is dead without even a squeal on either console or serial.

> I assure you reading  is just as obvious as (null) and
> reading fffa is just as good as -ENOMEM.
> 
> In fact printing with hex is more information. Maybe it is important
> that buggy pointer is small value but it's value is lost.
>
> Sure don't dereference a pointer for very small or very erry values
> but print it without all the bell and whistles.

That's a reasonable suggestion, but it still needs to be special cased.
Note the difference between printk("%px", 42) and printk("%s", 42).

See this part:
-   if (!ptr && *fmt != 'K' && *fmt != 'x') {
+   if (IS_BAD_PTR(ptr) && *fmt != 'K' && *fmt != 'x') {
Printing the pointer is already excluded; what I'm fixing is:
1. lying that the bad pointer was (null) when it was -ENOMEM (commit 1)
2. crash when some bad code tries to printk("%s", -ENOMEM) (commit 2)

So, if what you propose is applying commit 2, and changing 1 to print the
raw value instead of (null)/(err)/(invalid), that sounds good.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢠⠒⠀⣿⡁ A dumb species has no way to open a tuna can.
⢿⡄⠘⠷⠚⠋⠀ A smart species invents a can opener.
⠈⠳⣄ A master species delegates.

[PATCH 1/2] vsprintf: distinguish between (null), (err) and (invalid) pointer derefs

[Adam Borowski]

Attempting to print an object pointed to by a bad (usually ERR_PTR) pointer
is a not so surprising error.  Our code handles them inconsistently:
 * two places print (null) if ptr
---
 lib/vsprintf.c | 14 --
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/lib/vsprintf.c b/lib/vsprintf.c
index d7a708f82559..1c2c3cc5a321 100644
--- a/lib/vsprintf.c
+++ b/lib/vsprintf.c
@@ -47,6 +47,8 @@
 #include 
 #include "kstrtox.h"
 
+#define BAD_PTR_STRING(x) (!(x) ? "(null)" : IS_ERR(x) ? "(err)" : "(invalid)")
+
 /**
  * simple_strtoull - convert a string to an unsigned long long
  * @cp: The start of the string
@@ -588,7 +590,7 @@ char *string(char *buf, char *end, const char *s, struct 
printf_spec spec)
size_t lim = spec.precision;
 
if ((unsigned long)s < PAGE_SIZE)
-   s = "(null)";
+   s = BAD_PTR_STRING(s);
 
while (lim--) {
char c = *s++;
@@ -1582,7 +1584,7 @@ char *device_node_string(char *buf, char *end, struct 
device_node *dn,
return string(buf, end, "(!OF)", spec);
 
if ((unsigned long)dn < PAGE_SIZE)
-   return string(buf, end, "(null)", spec);
+   return string(buf, end, BAD_PTR_STRING(dn), spec);
 
/* simple case without anything any more format specifiers */
fmt++;
@@ -1851,12 +1853,12 @@ char *pointer(const char *fmt, char *buf, char *end, 
void *ptr,
 
if (!ptr && *fmt != 'K' && *fmt != 'x') {
/*
-* Print (null) with the same width as a pointer so it makes
-* tabular output look nice.
+* Print (null)/etc with the same width as a pointer so it
+* makes tabular output look nice.
 */
if (spec.field_width == -1)
spec.field_width = default_width;
-   return string(buf, end, "(null)", spec);
+   return string(buf, end, BAD_PTR_STRING(ptr), spec);
}
 
switch (*fmt) {
@@ -2575,7 +2577,7 @@ int vbin_printf(u32 *bin_buf, size_t size, const char 
*fmt, va_list args)
 
if ((unsigned long)save_str > (unsigned long)-PAGE_SIZE
|| (unsigned long)save_str < PAGE_SIZE)
-   save_str = "(null)";
+   save_str = BAD_PTR_STRING(save_str);
len = strlen(save_str) + 1;
if (str + len < end)
memcpy(str, save_str, len);
-- 
2.16.2

[PATCH 2/2] vsprintf: don't dereference pointers to the first or last page

[Adam Borowski]

As old code to avoid so is inconsistent, let's unify it within a single
macro.

Signed-off-by: Adam Borowski 
---
 lib/vsprintf.c | 11 ++-
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/lib/vsprintf.c b/lib/vsprintf.c
index 1c2c3cc5a321..4914dac03f0a 100644
--- a/lib/vsprintf.c
+++ b/lib/vsprintf.c
@@ -47,6 +47,8 @@
 #include 
 #include "kstrtox.h"
 
+#define IS_BAD_PTR(x) ((unsigned long)(x) >= (unsigned long)-PAGE_SIZE \
+   || (unsigned long)(x) < PAGE_SIZE)
 #define BAD_PTR_STRING(x) (!(x) ? "(null)" : IS_ERR(x) ? "(err)" : "(invalid)")
 
 /**
@@ -589,7 +591,7 @@ char *string(char *buf, char *end, const char *s, struct 
printf_spec spec)
int len = 0;
size_t lim = spec.precision;
 
-   if ((unsigned long)s < PAGE_SIZE)
+   if (IS_BAD_PTR(s))
s = BAD_PTR_STRING(s);
 
while (lim--) {
@@ -1583,7 +1585,7 @@ char *device_node_string(char *buf, char *end, struct 
device_node *dn,
if (!IS_ENABLED(CONFIG_OF))
return string(buf, end, "(!OF)", spec);
 
-   if ((unsigned long)dn < PAGE_SIZE)
+   if (IS_BAD_PTR(dn))
return string(buf, end, BAD_PTR_STRING(dn), spec);
 
/* simple case without anything any more format specifiers */
@@ -1851,7 +1853,7 @@ char *pointer(const char *fmt, char *buf, char *end, void 
*ptr,
 {
const int default_width = 2 * sizeof(void *);
 
-   if (!ptr && *fmt != 'K' && *fmt != 'x') {
+   if (IS_BAD_PTR(ptr) && *fmt != 'K' && *fmt != 'x') {
/*
 * Print (null)/etc with the same width as a pointer so it
 * makes tabular output look nice.
@@ -2575,8 +2577,7 @@ int vbin_printf(u32 *bin_buf, size_t size, const char 
*fmt, va_list args)
const char *save_str = va_arg(args, char *);
size_t len;
 
-   if ((unsigned long)save_str > (unsigned long)-PAGE_SIZE
-   || (unsigned long)save_str < PAGE_SIZE)
+   if (IS_BAD_PTR(save_ptr))
save_str = BAD_PTR_STRING(save_str);
len = strlen(save_str) + 1;
if (str + len < end)
-- 
2.16.2

Re: Removing architectures without upstream gcc support

[Adam Borowski]

On Fri, Feb 23, 2018 at 02:32:08PM -0500, James Bottomley wrote:
> On Fri, 2018-02-23 at 18:19 +, Al Viro wrote:
> [...]
> > IIRC, parisc/qemu stuff had been announced a while ago;
> 
> I have, but it didn't work sufficiently for me to either boot a kernel
> using system emulation or start an architecture container using user
> emulation.  I'll try again now that qemu has gone through several
> revisions.

Doesn't seem to work.  Debian package (1:2.11+dfsg-1) ships hppa support but
it doesn't even install binfmt; otherwise, -user is functional enough for a
minimal executable (so arch-test reports it as working[1]), but not for
anything libc:

[/srv/chroots/hppa]# chroot . /usr/bin/qemu-hppa-static /bin/true
qemu-hppa-static: 
/build/qemu-v8TF72/qemu-2.11+dfsg/target/hppa/translate.c:422: nullify_end: 
Assertion `status != DISAS_NORETURN && status != DISAS_IAQ_N_UPDATED' failed.
Segmentation fault

This looks bad enough that I didn't even look at qemu-system.


ᛗᛖᛟᚹ!

[1]. On archs without changed baseline, the test is just
write(1,"ok\n",3);_exit(0); unless there's a known issue to look for
such as swpb emulation being nop, etc.
-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢰⠒⠀⣿⡁ Vat kind uf sufficiently advanced technology iz dis!?
⢿⡄⠘⠷⠚⠋⠀ -- Genghis Ht'rok'din
⠈⠳⣄ 

Re: [PATCH 00/31 v2] PTI support for x86_32

[Adam Borowski]

On Sat, Feb 10, 2018 at 12:22:59PM -0800, Linus Torvalds wrote:
> On Sat, Feb 10, 2018 at 1:15 AM, Adam Borowski  wrote:
> >
> > Alas, we got some data:
> > https://popcon.debian.org/ says 20% of x86 users have i386 as their main ABI
> > (current; people with popcon installed).
> 
> One of the issues I've seen is that people often simply move a disk
> (or copy an installation) when upgrading machines.

Less skilled users (ie, most of them) had until recently a different hurdle:
the "32-bit" option was shown way too prominently, without an explanation.

> Does Debian make it easy to upgrade to a 64-bit kernel if you have a
> 32-bit install?

Quite easy, yeah.  Crossgrading userspace is not for the faint of the heart,
but changing just the kernel is fine.

> Because if not, then it's entirely possible that a lot
> of people started out with a 32-bit install (maybe they even had a
> 64-bit kernel, but they started when the 32-bit install was the
> default one), and never upgraded their kernel.
> 
> It really should be easy to _just_ upgrade the kernel. But if the
> distro doesn't support it, most people won't do it.

I just realized that, for people who use distro kernels, the right way is a
message during upgrade.  Ie, it's up to the packagers rather than you.

Having a dire-sounding printk, though, would still be nice.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢰⠒⠀⣿⡁ Vat kind uf sufficiently advanced technology iz dis!?
⢿⡄⠘⠷⠚⠋⠀ -- Genghis Ht'rok'din
⠈⠳⣄

Re: [PATCH 00/31 v2] PTI support for x86_32

[Adam Borowski]

On Fri, Feb 09, 2018 at 08:11:12PM +0100, Joerg Roedel wrote:
> On Fri, Feb 09, 2018 at 05:47:43PM +, Andy Lutomirski wrote:
> > One thing worth noting is that performance of this whole series is
> > going to be abysmal due to the complete lack of 32-bit PCID.  Maybe
> > any kernel built with this option set that runs on a CPU that has the
> > PCID bit set in CPUID should print a big fat warning like "WARNING:
> > you are using 32-bit PTI on a 64-bit PCID-capable CPU.  Your
> > performance will increase dramatically if you switch to a 64-bit
> > kernel."
> 
> Thanks for your review. I can add this warning, but I just hope that not
> a lot of people will actually see it :)

Alas, we got some data:
https://popcon.debian.org/ says 20% of x86 users have i386 as their main ABI
(current; people with popcon installed).

Of those, 80% use 32-bit kernels: i686 881, x86_64 229, i586 14
(uname -m included in bug reports; data for 2016) -- and bug reporters tend
to have more clue than the average user.  There's no way so many folks still
use pre-2004 computers.

Thus, if you could include that big fat warning, distro developers would be
thankful.  Make it show fiery letters if you can.  Preferably, we'd want a
huge mallet reach out of the screen and bonk the user on the head, but with
that impossible, a scary message would help.

Let them use 32-bit userland, but if someone runs such a kernel on a modern
machine, some kind of verbal abuse is warranted.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢰⠒⠀⣿⡁ Vat kind uf sufficiently advanced technology iz dis!?
⢿⡄⠘⠷⠚⠋⠀ -- Genghis Ht'rok'din
⠈⠳⣄ 

Re: [PATCH v2 0/4] ARM/arm64: exynos: Fix missing missing reg warning for syscon restart nodes

[Adam Borowski]

On Thu, Feb 08, 2018 at 02:46:32PM +0100, Marek Szyprowski wrote:
> Hi Krzysztof,
> 
> On 2018-01-30 22:18, Krzysztof Kozlowski wrote:
> > Hi,
> > 
> > Changes since v1:
> > 1. New patch (1/4) calling devm_of_platform_populate() in PMU driver,
> > following Rob's advice.
> > 2. The DTS patches moving reboot/poweroff nodes (3/4 and 4/4) now depend
> > on this.
> 
> Tested-by: Marek Szyprowski 
> 
> Tested following boards and reboot works fine:
> Exynos3250 (artik5-eval)
> Exynos4412 (trats2)
> Exynos5410 (odroid xu)
> Exynos5422 (odroid xu3/xu4)
> Exynos5433 (tm2)

Doesn't seem to work on mine:
Exynos4412 (odroid u2)

Rebooting worked correctly with 3.8 + gazillion non-mainlined patches (as
shipped with providen images), never did with mainline.

Tested on current mid-of-the-merge-window, plus these four patches.


It's possible that I'm doing something wrong; might need an updated u-boot,
etc.  Log of a boot-till-reboot+hang session attached.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢰⠒⠀⣿⡁ Imagine there are bandits in your house, your kid is bleeding out,
⢿⡄⠘⠷⠚⠋⠀ the house is on fire, and seven big-ass trumpets are playing in the
⠈⠳⣄ sky.  Your cat demands food.  The priority should be obvious...
U-Boot 2016.11+dfsg1-3 (Dec 22 2016 - 04:44:44 +)

CPU:   Exynos4412 @ 1 GHz
Model: Odroid based on Exynos4412
Board: Odroid based on Exynos4412
Type:  u3
DRAM:  2 GiB
LDO20@VDDQ_EMMC_1.8V: set 180 uV; enabling
LDO22@VDDQ_EMMC_2.8V: set 280 uV; enabling
LDO21@TFLASH_2.8V: set 280 uV; enabling
MMC:   EXYNOS DWMMC: 0
*** Warning - bad CRC, using default environment

Net:   No ethernet found.
Hit any key to stop autoboot:  2 ^H^H^H 1 ^H^H^H 0 
MMC Device 1 not found
no mmc device at slot 1
switch to partitions #0, OK
mmc0(part 0) is current device
Scanning mmc 0:1...
Found /extlinux/extlinux.conf
Retrieving file: /extlinux/extlinux.conf
2303 bytes read in 9 ms (249 KiB/s)
U-Boot menu
1:  Debian GNU/Linux kernel 4.15.0-12182-g6de178c233be
2:  Debian GNU/Linux kernel 4.15.0-12182-g6de178c233be (rescue target)
3:  Debian GNU/Linux kernel 4.15.0-10690-gec9e48cd93be
4:  Debian GNU/Linux kernel 4.15.0-10690-gec9e48cd93be (rescue target)
5:  Debian GNU/Linux kernel 4.15.0-00027-g38a981d152b8
6:  Debian GNU/Linux kernel 4.15.0-00027-g38a981d152b8 (rescue target)
7:  Debian GNU/Linux kernel 4.14.0-00115-g3d7c587c4c1b
8:  Debian GNU/Linux kernel 4.14.0-00115-g3d7c587c4c1b (rescue target)
Enter choice: 1:Debian GNU/Linux kernel 4.15.0-12182-g6de178c233be
Retrieving file: /initrd.img-4.15.0-12182-g6de178c233be
5766183 bytes read in 237 ms (23.2 MiB/s)
Retrieving file: /vmlinuz-4.15.0-12182-g6de178c233be
5680536 bytes read in 224 ms (24.2 MiB/s)
append: root=LABEL=ereshkigal 
Retrieving file: /dtbs/4.15.0-12182-g6de178c233be/exynos4412-odroidu3.dtb
53406 bytes read in 41 ms (1.2 MiB/s)
Kernel image @ 0x4200 [ 0x00 - 0x56ad98 ]
## Flattened Device Tree blob at 4300
   Booting using the fdt blob at 0x4300
   Loading Ramdisk to 4fa8, end 4c27 ... OK
   Loading Device Tree to 4fa6f000, end 4fa7f09d ... OK

Starting kernel ...

[0.00] Booting Linux on physical CPU 0xa00
[0.00] Linux version 4.15.0-12182-g6de178c233be (kilobyte@kholdan) (gcc 
version 7.3.0 (Debian 7.3.0-3)) #1 SMP PREEMPT Fri Feb 9 04:03:33 CET 2018
[0.00] CPU: ARMv7 Processor [413fc090] revision 0 (ARMv7), cr=10c5387d
[0.00] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing 
instruction cache
[0.00] OF: fdt: Machine model: Hardkernel ODROID-U3 board based on 
Exynos4412
[0.00] Memory policy: Data cache writealloc
[0.00] Reserved memory: created DMA memory pool at 0xbf70, size 8 
MiB
[0.00] OF: reserved mem: initialized node region_mfc_right, compatible 
id shared-dma-pool
[0.00] Reserved memory: created DMA memory pool at 0xbd30, size 36 
MiB
[0.00] OF: reserved mem: initialized node region_mfc_left, compatible 
id shared-dma-pool
[0.00] cma: Reserved 64 MiB at 0xb900
[0.00] Samsung CPU ID: 0xe4412220
[0.00] Running under secure firmware.
[0.00] percpu: Embedded 16 pages/cpu @(ptrval) s35800 r8192 d21544 
u65536
[0.00] Built 1 zonelists, mobility grouping on.  Total pages: 511040
[0.00] Kernel command line: root=LABEL=ereshkigal 
[0.00] Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
[0.00] Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
[0.00] Memory: 1947660K/2051072K available (8192K kernel code, 321K 
rwdata, 2536K rodata, 1024K init, 313K bss, 37876K reserved, 65536K 
cma-reserved, 1199104K highmem)
[0.00] Virtual kernel memory layout:
[0.00] vector  : 0x - 0x1000   (   4 kB)
[0.00] fixmap  : 0xffc0 - 0xfff0   (3072 kB)
[0.00] vmalloc : 0xf080 - 0xff80   ( 240 MB)
[0.00] lowmem  : 0xc000 - 0xf000   ( 76

Re: [PATCH] vsprintf: avoid misleading "(null)" for %px

[Adam Borowski]

On Tue, Feb 06, 2018 at 07:32:32AM +1100, Kees Cook wrote:
> On Tue, Feb 6, 2018 at 7:15 AM, Tobin C. Harding  wrote:
> > On Tue, Feb 06, 2018 at 05:57:17AM +1100, Kees Cook wrote:
> >> On Mon, Feb 5, 2018 at 8:44 PM, Petr Mladek  wrote:
> >> > On Sun 2018-02-04 18:45:21, Adam Borowski wrote:
> >> >> Like %pK already does, print "" instead.
> >> >>
> >> >> This confused people -- the convention is that "(null)" means you tried 
> >> >> to
> >> >> dereference a null pointer as opposed to printing the address.
> >
> > Leaving aside what is converting to %px.  If we consider that using %px
> > is meant to convey to us that we _really_ want the address, in hex hence
> > the 'x', then it is not surprising that we will get ""'s for a
> > null pointer, right?  Yes it is different to before but since we are
> > changing the specifier does this not imply that there may be some
> > change?
> 
> I personally prefer s, but if we're going to change this, we need
> to be aware of the difference.

It's easy to paint this bikeshed any color you guys want to: there's an "if"
already.  My preference is also ; NULL would be good, too -- I just
don't want (null) as that has a special meaning in usual userspace
implementations; (null) also fits well most other modes of %p as they show
some object the argument points to.  Confusion = wasted debugging time.

This is consistent with what we had before, with %pK special-cased.

> > In what is now to be expected fashion for %p the discussion appears to
> > have split into two different things - what to do with %px and what to
> > do with %pK :)
> 
> I say leave %pK alone. :)

As in, printing some random (hashed) value?


Let's recap:

Currently:
  not-null  null
%pponies  object's description  (null)
%px   address   (null)
%pK   hash  hash

I'd propose:
  not-null  null
%pponies  object's description  (null)
%px   address   
%pK   hash  

The initial patch in this thread changes printk("%px",0) from (null) to
; what Tobin complained about is that printk("%pK",0) prints a
random value.   


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ The bill with 3 years prison for mentioning Polish concentration
⣾⠁⢰⠒⠀⣿⡁ camps is back.  What about KL Warschau (operating until 1956)?
⢿⡄⠘⠷⠚⠋⠀ Zgoda?  Łambinowice?  Most ex-German KLs?  If those were "soviet
⠈⠳⣄ puppets", Bereza Kartuska?  Sikorski's camps in UK (thanks Brits!)?

Re: [PATCH] vsprintf: avoid misleading "(null)" for %px

[Adam Borowski]

On Mon, Feb 05, 2018 at 09:03:05PM +1100, Tobin C. Harding wrote:
> On Mon, Feb 05, 2018 at 10:44:38AM +0100, Petr Mladek wrote:
> > On Sun 2018-02-04 18:45:21, Adam Borowski wrote:
> > > Like %pK already does, print "" instead.
> > >
> > > This confused people -- the convention is that "(null)" means you tried to
> > > dereference a null pointer as opposed to printing the address.
> > 
> > By other words, this avoids regressions when people convert
> > %x to %px. Do I get it right, please?

It's a regression in the sense that it confuses people.  %px never could
dereference a pointer so the information provided doesn't change, merely its
presentation.

> > > diff --git a/lib/vsprintf.c b/lib/vsprintf.c
> > > index 77ee6ced11b1..d7a708f82559 100644
> > > --- a/lib/vsprintf.c
> > > +++ b/lib/vsprintf.c
> > > @@ -1849,7 +1849,7 @@ char *pointer(const char *fmt, char *buf, char 
> > > *end, void *ptr,
> > >  {
> > >   const int default_width = 2 * sizeof(void *);
> > >  
> > > - if (!ptr && *fmt != 'K') {
> > > + if (!ptr && *fmt != 'K' && *fmt != 'x') {
> 
> I don't know if it matters but with this it won't be immediately
> apparent that a null pointer was printed (since zero could hash to
> anything).

My change touches %px only, where your concern doesn't apply.

You're right, though, when it comes to %pK:
printk("%%pK: %pK, %%px: %px\n", 0, 0);
says
%pK: ba8bdc0a, %px: 

So what should we do?  Avoid hashing 0?  Print a special value?


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ The bill with 3 years prison for mentioning Polish concentration
⣾⠁⢰⠒⠀⣿⡁ camps is back.  What about KL Warschau (operating until 1956)?
⢿⡄⠘⠷⠚⠋⠀ Zgoda?  Łambinowice?  Most ex-German KLs?  If those were "soviet
⠈⠳⣄ puppets", Bereza Kartuska?  Sikorski's camps in UK (thanks Brits!)?

[PATCH] vsprintf: avoid misleading "(null)" for %px

[Adam Borowski]

Like %pK already does, print "" instead.

This confused people -- the convention is that "(null)" means you tried to
dereference a null pointer as opposed to printing the address.

Signed-off-by: Adam Borowski 
---
 lib/vsprintf.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/vsprintf.c b/lib/vsprintf.c
index 77ee6ced11b1..d7a708f82559 100644
--- a/lib/vsprintf.c
+++ b/lib/vsprintf.c
@@ -1849,7 +1849,7 @@ char *pointer(const char *fmt, char *buf, char *end, void 
*ptr,
 {
const int default_width = 2 * sizeof(void *);
 
-   if (!ptr && *fmt != 'K') {
+   if (!ptr && *fmt != 'K' && *fmt != 'x') {
/*
 * Print (null) with the same width as a pointer so it makes
 * tabular output look nice.
-- 
2.15.1

Re: [PATCH v3] drm/nouveau: Move irq setup/teardown to pci ctor/dtor

[Adam Borowski]

On Thu, Jan 25, 2018 at 06:29:53PM -0500, Lyude Paul wrote:
> This was made apparent by what appeared to be a regression in the
> mainline kernel that started introducing suspend/resume issues for
> nouveau:
> 
> a0c9259dc4e1 (irq/matrix: Spread interrupts on allocation)

I'm just a dumb user here, but I confirm:
CPU: AMD Phenom II X6 1055T, GPU: GTX 560 Ti
100% fail to resume GPU on 4.15-rc*, 100% ok with your patch.

And that spinning rust hates being repeatedly powered off and on... :/

> ---
> Changes since v2:
>  - Remove teardown, just reuse pci->irq to indicate when we're tearing
>down the driver
> 
>  drivers/gpu/drm/nouveau/nvkm/subdev/pci/base.c | 44 
> +-
>  1 file changed, 29 insertions(+), 15 deletions(-)

Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢰⠒⠀⣿⡁ Imagine there are bandits in your house, your kid is bleeding out,
⢿⡄⠘⠷⠚⠋⠀ the house is on fire, and seven big-ass trumpets are playing in the
⠈⠳⣄ sky.  Your cat demands food.  The priority should be obvious...

Re: [RESEND PATCH] blackfin: defconfig: Cleanup from old Kconfig options

[Adam Borowski]

On Wed, Dec 27, 2017 at 12:29:33PM +0100, Linus Walleij wrote:
> On Tue, Dec 26, 2017 at 2:30 PM, Krzysztof Kozlowski  wrote:
> 
> > Remove old, dead Kconfig options (in order appearing in this commit):
> >
> > Signed-off-by: Krzysztof Kozlowski 
> 
> Acked-by: Linus Walleij 
> 
> This architecture is becoming a burden :/
> 
> Last blackfin pull request was in June 2014 by Steven Miao.

April 2015, actually.  Doesn't change the conclusion, though.

> Steven, what's up with this? I don't mind the arch/blackfin as much
> as the plethora of boardfiles that need to be maintained as soon as
> we change some platform data or so, and it's just a mystery whether
> things really get tested and any of the changes made here since 2014
> are screwing up the blackfin. No ACKs or Tested-by's ever appear.
> 
> Is there active testing of mainline with blackfin?

After multiple pings, I just (two days ago) sent the following:
https://marc.info/?l=linux-kernel&m=151421639229901

So even if there's no actual testing, at the very least it'd be good to mark
the architecture as orphaned, so people don't wait for ACKs that never come.


Meow!
-- 
// If you believe in so-called "intellectual property", please immediately
// cease using counterfeit alphabets.  Instead, contact the nearest temple
// of Amon, whose priests will provide you with scribal services for all
// your writing needs, for Reasonable And Non-Discriminatory prices.

[PATCH] MAINTAINERS: mark arch/blackfin/ and its gubbins as orphaned

[Adam Borowski]

The blackfin architecture has seen no maintainer action of any kind since
April 2015.  No new code, no pull requests, no acks to patches, no response
to mails, nothing.

The web site has an expired certificate (expiration Sep 2017, issued in
2013), the mailing list sees no answers either, with one exception:

https://sourceforge.net/p/adi-buildroot/mailman/adi-buildroot-devel/
> Hi all,
>
> Steven is no longer working on this for ADI. Acked by me if this works. 
> Thanks.
>
> Best regards,
> Aaron Wu
> Analog Devices Inc.

But, Aaron doesn't seem to respond to queries either.

Signed-off-by: Adam Borowski 
---
Obviously, plankton like me with no relation to the architecture in question
shouldn't be orphaning it, but consider this mail telling Linus that in the
state of Denmark there is an odor of decay.

I also did not pester Scott Jiang (BLACKFIN MEDIA DRIVER) before (just
Steven Miao and Aaron Wu), but with the last ack on 2015-04-09, maintenance
of that part looks like it's on the same boat.


God Jul!
 MAINTAINERS | 16 +++-
 1 file changed, 7 insertions(+), 9 deletions(-)

diff --git a/MAINTAINERS b/MAINTAINERS
index a6e86e20761e..2d0773007c89 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2621,24 +2621,22 @@ F:  fs/bfs/
 F: include/uapi/linux/bfs_fs.h
 
 BLACKFIN ARCHITECTURE
-M: Steven Miao 
 L: adi-buildroot-de...@lists.sourceforge.net (moderated for 
non-subscribers)
 T: git git://git.code.sf.net/p/adi-linux/code
 W: http://blackfin.uclinux.org
-S: Supported
+S: Orphan
 F: arch/blackfin/
 
 BLACKFIN EMAC DRIVER
 L: adi-buildroot-de...@lists.sourceforge.net (moderated for 
non-subscribers)
 W: http://blackfin.uclinux.org
-S: Supported
+S: Orphan
 F: drivers/net/ethernet/adi/
 
 BLACKFIN MEDIA DRIVER
-M: Scott Jiang 
 L: adi-buildroot-de...@lists.sourceforge.net (moderated for 
non-subscribers)
 W: http://blackfin.uclinux.org/
-S: Supported
+S: Orphan
 F: drivers/media/platform/blackfin/
 F: drivers/media/i2c/adv7183*
 F: drivers/media/i2c/vs6624*
@@ -2646,25 +2644,25 @@ F:  drivers/media/i2c/vs6624*
 BLACKFIN RTC DRIVER
 L: adi-buildroot-de...@lists.sourceforge.net (moderated for 
non-subscribers)
 W: http://blackfin.uclinux.org
-S: Supported
+S: Orphan
 F: drivers/rtc/rtc-bfin.c
 
 BLACKFIN SDH DRIVER
 L: adi-buildroot-de...@lists.sourceforge.net (moderated for 
non-subscribers)
 W: http://blackfin.uclinux.org
-S: Supported
+S: Orphan
 F: drivers/mmc/host/bfin_sdh.c
 
 BLACKFIN SERIAL DRIVER
 L: adi-buildroot-de...@lists.sourceforge.net (moderated for 
non-subscribers)
 W: http://blackfin.uclinux.org
-S: Supported
+S: Orphan
 F: drivers/tty/serial/bfin_uart.c
 
 BLACKFIN WATCHDOG DRIVER
 L: adi-buildroot-de...@lists.sourceforge.net (moderated for 
non-subscribers)
 W: http://blackfin.uclinux.org
-S: Supported
+S: Orphan
 F: drivers/watchdog/bfin_wdt.c
 
 BLINKM RGB LED DRIVER
-- 
2.15.1

Re: [PATCH] drm/nouveau/imem/nv50: fix incorrect use of refcount API

[Adam Borowski]

On Fri, Dec 08, 2017 at 06:30:34PM +, Ard Biesheuvel wrote:
> Commit be55287aa5b ("drm/nouveau/imem/nv50: embed nvkm_instobj directly
> into nv04_instobj") introduced some new calls to the refcount api to
> the nv50 mapping code. In one particular instance, it does the
> following:
> 
> if (!refcount_inc_not_zero(&iobj->maps)) {
> ...
> refcount_inc(&iobj->maps);
> }
> 
> i.e., it calls refcount_inc() to increment the refcount when it is known
> to be zero, which is explicitly forbidden by the API. Instead, use
> refcount_set() to set it to 1.
> 
> Signed-off-by: Ard Biesheuvel 
> ---

Awesome!  Works for me.

> Apologies if this was already found and fixed. I don't usually follow
> the DRM or nouveau mailing lists.

I see nothing relevant in dri-devel and nouveau archives, except my
complaint (GTX 560 Ti (GF114)):
https://lists.freedesktop.org/archives/nouveau/2017-December/029264.html
and Richard Narron seconding it (MSI GeForce 210):
https://lists.freedesktop.org/archives/nouveau/2017-December/029276.html

>  drivers/gpu/drm/nouveau/nvkm/subdev/instmem/nv50.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/instmem/nv50.c 
> b/drivers/gpu/drm/nouveau/nvkm/subdev/instmem/nv50.c
> index 1ba7289684aa..db48a1daca0c 100644
> --- a/drivers/gpu/drm/nouveau/nvkm/subdev/instmem/nv50.c
> +++ b/drivers/gpu/drm/nouveau/nvkm/subdev/instmem/nv50.c
> @@ -249,7 +249,7 @@ nv50_instobj_acquire(struct nvkm_memory *memory)
>   iobj->base.memory.ptrs = &nv50_instobj_fast;
>   else
>   iobj->base.memory.ptrs = &nv50_instobj_slow;
> - refcount_inc(&iobj->maps);
> + refcount_set(&iobj->maps, 1);
>   }
>  
>   mutex_unlock(&imem->subdev.mutex);
> -- 

I'm just a dumb user here, my tags don't carry a weight, but Tested-by:.


Meow!
-- 
// If you believe in so-called "intellectual property", please immediately
// cease using counterfeit alphabets.  Instead, contact the nearest temple
// of Amon, whose priests will provide you with scribal services for all
// your writing needs, for Reasonable And Non-Discriminatory prices.

Re: [PATCH 0/2] Add support for ZSTD-compressed kernel

[Adam Borowski]

On Wed, Oct 11, 2017 at 02:01:41AM +, Nick Terrell wrote:
> On 10/10/17, 5:08 PM, "Adam Borowski"  wrote:
> > On Tue, Oct 10, 2017 at 10:40:13PM +, Nick Terrell wrote:
> > > On 10/10/17, 2:56 PM, "h...@zytor.com"  wrote:
> > > >On October 10, 2017 2:22:42 PM PDT, Nick Terrell  wrote:
> > > >>This patch set adds support for a ZSTD-compressed kernel and ramdisk
> > > >>images in the kernel boot process. It only integrates the support with
> > > >>x86, though the first patch is generic to all architectures.
> 
> > > Comparing the command line tools on a kernel image that is 68970616 B 
> > > large:
> > >
> > > | Algorithm | Compression Ratio | Decompression MB/s |
> > > |---|---||
> > > | zstd  |  4.42 |  436.5 |
> > > | gzip  |  3.72 |  134.1 |
> > > | xz|  4.83 |   53.1 |
> > > | lz4   |  3.18 | 1682.2 |
> > > | lzo   |  3.36 |  389.6 |
> > > | bzip2 |  4.03 |   33.3 |
> 
> > Perhaps it'd be a good idea to cull some of bad algorithms?  I don't know
> > the memory used by those, but envelope of the table you just shown suggests
> > using bzip2 and lzo is pointless.  So is gzip, but it's widespread as the
> > default for initramfs producers, thus it'd be unsafe to kill it.
> 
> I'm not sure there is a great use case for bzip2. It requires more memory
> than xz, compresses worse, and decompresses slower. lzo in the kernel might
> decompress a bit faster than zstd (looking back at the BtrFS benchmarks, it
> did). More importantly, it uses less memory than zstd. When decompressing
> the kernel zstd only needs 192 KB, but for initramfs, it will need more.
> Still, unless you really need 5% more compression, lz4 is probably a better
> option than lzo for speed.

The main reason I'm raising this question is, bzip2 has no other users in
the kernel, thus removing support for it would allow us to delete its code.

As for lzo, even if there are cases when it's a bit faster (overcoming the
9% lead zstd has in your userspace benchmark), it wouldn't be faster by
much -- certainly nowhere to make anyone want the massive compression loss.

Thus, let's enumerate use cases:
* a fast modern machine: zstd boots a second faster at the cost of 1.3MB
  image size; doesn't matter much either way.  No one wants weaker options.
* a weaker machine: zstd rapidly gains as boot times rise.
* a very slow machine with fast I/O that boots often: lz4.
* disk space at extreme premium: xz.

In no case I can think of other algorithms are the rational choice.
There's little gain in removing the rest, though: the code is used for other
purposes in the kernel, and doesn't get compiled in unless manually
selected.

Zstd looks like a good default, but it's nowhere near mature enough: this
patch is x86-only, tools which people may use to analyze compressed kernels
don't know about zstd yet, etc.

Thus, I'd propose the following plan:
* add zstd
* remove bzip2
* update recommendations (Kconfig text)
* in a year or two, consider making zstd the default

> > > I know that this isn't a real benchmark of the kernel decompression. I
> > > still need to figure out how to time the kernel decompression. If you have
> > > any suggestions let me know. Otherwise, I'll get back to you when I've
> > > figured out how to run the benchmark.
> 
> I've found a way to benchmark the kernel decompression time during boot
> with QEMU. I add timestamps to every line of the output. I also had to
> print 100 lines before the decompression starts to get consistent results.
> 
> I've found that zstd is decompressing 2x slower than it should. I narrowed
> down the problem to ZSTD_wildcopy() and ZSTD_copy8() in
> lib/zstd/zstd_internal.h. ZSTD_wildcopy() calls memcpy(src, dst, 8) in
> a loop and doesn't handle the freestanding memcpy() well. Replacing it with
> __builtin_mcmpy(src, dst, 8) doubles the speed.

... and by maturing I meant issues like this.

> I'm not an expert in freestanding gcc compilation, but I believe it is okay
> to call __builtin_memcpy() in freestanding mode, and gcc will either
> inline it, or add the right function call. The difference being that gcc
> will be able to apply its memcpy() analysis. I also see that
> arch/x86/boot/string.h defines memcpy() to __builtin_memcpy. Is it safe to
> directly use __builtin_memcpy() in lib/zstd/zstd_internal.h?

Try it, and see if it builds and fails to crash. :)


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ We domesticated dogs 36000 years ago; together we chased
⣾⠁⢰⠒⠀⣿⡁ animals, hung out and licked or scratched our private parts.
⢿⡄⠘⠷⠚⠋⠀ Cats domesticated us 9500 years ago, and immediately we got
⠈⠳⣄ agriculture, towns then cities. -- whitroth on /.

Re: [PATCH 0/2] Add support for ZSTD-compressed kernel

[Adam Borowski]

On Tue, Oct 10, 2017 at 10:40:13PM +, Nick Terrell wrote:
> On 10/10/17, 2:56 PM, "h...@zytor.com"  wrote:
> >On October 10, 2017 2:22:42 PM PDT, Nick Terrell  wrote:
> >>This patch set adds support for a ZSTD-compressed kernel and ramdisk
> >>images in the kernel boot process. It only integrates the support with
> >>x86, though the first patch is generic to all architectures.
> >>
> >>Zstandard requires slightly more memory during the kernel decompression
> >>on x86 (192 KB vs 64 KB), and the memory usage is independent of the
> >>window size.
> >>
> >>Zstandard requires memory proprortional to the window size used during
> >>compression for decompressing the ramdisk image, since streaming mode
> >>is
> >>used. Newer versions of zstd (1.3.2+) list the window size of a file
> >>with `zstd -lv '. The absolute maximum amount of memory required
> >>is just over 8 MB.

> > And, pray tell, what are the actual results?  What is the trade-off of
> > kernel size versus decompression performance versus the other algorithms
> > that we already support?  Adding algorithms for their own sake is a bad
> > thing not a good thing.
> 
> Sorry I neglected to include the benchmarks I've run so far. I've included
> them below, and will add them to the next version's cover letter.
> 
> Comparing the command line tools on a kernel image that is 68970616 B large:
> 
> | Algorithm | Compression Ratio | Decompression MB/s |
> |---|---||
> | zstd  |  4.42 |  436.5 |
> | gzip  |  3.72 |  134.1 |
> | xz|  4.83 |   53.1 |
> | lz4   |  3.18 | 1682.2 |
> | lzo   |  3.36 |  389.6 |
> | bzip2 |  4.03 |   33.3 |

Perhaps it'd be a good idea to cull some of bad algorithms?  I don't know
the memory used by those, but envelope of the table you just shown suggests
using bzip2 and lzo is pointless.  So is gzip, but it's widespread as the
default for initramfs producers, thus it'd be unsafe to kill it.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ We domesticated dogs 36000 years ago; together we chased
⣾⠁⢰⠒⠀⣿⡁ animals, hung out and licked or scratched our private parts.
⢿⡄⠘⠷⠚⠋⠀ Cats domesticated us 9500 years ago, and immediately we got
⠈⠳⣄ agriculture, towns then cities. -- whitroth on /.

Re: Linux 4.14: Reported regressions as of Sunday, 2017-10-08

[Adam Borowski]

On Sun, Oct 08, 2017 at 02:37:41PM +0200, Thorsten Leemhuis wrote:
> Hi! Find below my second regression report for Linux 4.14. It lists 8
> regressions I'm currently aware of. One regression was fixed since last
> weeks report. One was in there that shouldn't have been there.
> 
> == Current regressions ==
> 
> "hangs when building e.g. perf" & "Random insta-reboots on AMD Phenom II"
> Status: "Mr. Luto better revert the new lazy TLB flushing fun'n'games"
> -> "Yeah, working on it.  It's not a straightforward revert."
> Note: TWIMC: Workaround: wrmsr -a 0xc0010015 0x118
> Reported: 2017-09-05
> https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1484723.html
> https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1501379.html
> https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1501570.html
> Cause: 94b1b03b519b81c494900cb112aa00ed205cc2d9

Uhm, that particular register value in the workaround applies only to a
particular CPU, and it sounds dangerous to use it as-is otherwise.

On affected families (apparently 15h as well) it's bit 0x0008 in
register 0xc0010015, you need to rdmsr that register then or that bit with
whatever value you had before.

(And I have no clue about how it works -- merely repeating what the guys say.
You'd want to not do so blindly.)


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ We domesticated dogs 36000 years ago; together we chased
⣾⠁⢰⠒⠀⣿⡁ animals, hung out and licked or scratched our private parts.
⢿⡄⠘⠷⠚⠋⠀ Cats domesticated us 9500 years ago, and immediately we got
⠈⠳⣄ agriculture, towns then cities. -- whitroth on /.

Re: [PATCH] vfs: hard-ban creating files with control characters in the name

[Adam Borowski]

On Tue, Oct 03, 2017 at 12:40:12PM -0400, Theodore Ts'o wrote:
> On Tue, Oct 03, 2017 at 03:07:24AM +0100, Al Viro wrote:
> > That essay is full of shit, and you've even mentioned parts of that just 
> > above...
> > NAK; you'd _still_ need proper quoting (or a shell with something 
> > resembling an
> > actual syntax, rather than the "more or less what srb had ended up 
> > implementing"),
> > so it doesn't really buy you anything.  Badly written script will still be
> > exploitable.  And since older kernels and other Unices are not going away,
> > you would've created an inconsistently vulnerable set of scripts, on top of
> > the false sense of security.
> 
> Banning certain characters is certainly not a panacea, and there are a
> lot of best practices that you have to follow when writing good
> scripts which most people don't follow, and so it's arguable that
> benefits are being overstated.

Yeah, it's "most people don't follow" which is the main issue here.  And
it appears to me that it's easy to plug the worst issues, especially \n.

> That being said the costs of suppressing certain bytes from appearing
> in pathnames seem fairly low.

There are three categories of problematic bytes/byte sequences:
* those that don't appear in the wild, require a bit of knowledge to input
  (inserting a control char is no rocket science but is beyond an average
  user's skill), and also have potential to cause damage
  - This is why I picked 1-31,127 in the initial patch.
* stuff that's unwanted but not unknown in the wild
* stuff that's used by every "normal" user, and considered problematic only
  by us traditional Unix folks

> Would this be more palatable if the ban on control characters were made
> into a compile-time or mount-time option?

For malformed Unicode or such, it'd make sense, yeah.

But Al has a good point that if most people were protected, they won't
bother escaping badness anymore -- leaving those whose systems allow control
chars vulnerable if they run a script that doesn't do quoting.

Thus, it'd be good to make at least worst cases non-configurable.

I went bold and submitted 1-31,127, as those have very low cost to block;
but if that's not conservative enough, blocking just \n has both very low
cost and a high benefit (special burdensome quoting required).

Discussing a configurable policy (perhaps here in vfs, perhaps as a LSM, a
seccomp hack or even LD_PRELOAD) would be interesting, but for the above
reason I'd want \n hard-banned.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ We domesticated dogs 36000 years ago; together we chased
⣾⠁⢰⠒⠀⣿⡁ animals, hung out and licked or scratched our private parts.
⢿⡄⠘⠷⠚⠋⠀ Cats domesticated us 9500 years ago, and immediately we got
⠈⠳⣄ agriculture, towns then cities. -- whitroth on /.