Re: unknown NMI on AMD Rome

2021-03-16 Thread Adam Borowski
On Tue, Mar 16, 2021 at 04:45:02PM +0100, Jiri Olsa wrote:
> hi,
> when running 'perf top' on AMD Rome (/proc/cpuinfo below)
> with fedora 33 kernel 5.10.22-200.fc33.x86_64
> 
> we got unknown NMI messages:
> 
> [  226.700160] Uhhuh. NMI received for unknown reason 3d on CPU 90.
> [  226.700162] Do you have a strange power saving mode enabled?
> [  226.700163] Dazed and confused, but trying to continue
> 
> also when discussing ths with Borislav, he managed to reproduce easily
> on his AMD Rome machine

Likewise, 3c on Pinnacle Ridge.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ in the beginning was the boot and root floppies and they were good.
⢿⡄⠘⠷⠚⠋⠀   --  on #linux-sunxi
⠈⠳⣄


Re: [PATCH v2 00/10] fsdax,xfs: Add reflink support for fsdax

2021-03-13 Thread Adam Borowski
On Sat, Mar 13, 2021 at 11:24:00AM -0500, Neal Gompa wrote:
> On Sat, Mar 13, 2021 at 8:09 AM Adam Borowski  wrote:
> >
> > On Wed, Mar 10, 2021 at 02:26:43PM +, Matthew Wilcox wrote:
> > > On Wed, Mar 10, 2021 at 08:21:59AM -0600, Goldwyn Rodrigues wrote:
> > > > DAX on btrfs has been attempted[1]. Of course, we could not
> > >
> > > But why?  A completeness fetish?  I don't understand why you decided
> > > to do this work.
> >
> > * xfs can shapshot only single files, btrfs entire subvolumes
> > * btrfs-send|receive
> > * enumeration of changed parts of a file
> 
> XFS cannot do snapshots since it lacks metadata COW. XFS reflinking is
> primarily for space efficiency.

A reflink is a single-file snapshot.

My work team really wants this very patchset -- reflinks on DAX allow
backups and/or checkpointing, without stopping the world (there's a single
file, "pool", here).

Besides, you can still get poor-man's whole-subvolume(/directory)
snapshots by manually walking the tree and reflinking everything.
That's not atomic -- but rsync isn't atomic either.  That's enough for
eg. dnf/dpkg purposes.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢰⠒⠀⣿⡁
⢿⡄⠘⠷⠚⠋⠀ NADIE anticipa la inquisición de españa!
⠈⠳⣄


Re: [PATCH v2 00/10] fsdax,xfs: Add reflink support for fsdax

2021-03-13 Thread Adam Borowski
On Wed, Mar 10, 2021 at 02:26:43PM +, Matthew Wilcox wrote:
> On Wed, Mar 10, 2021 at 08:21:59AM -0600, Goldwyn Rodrigues wrote:
> > DAX on btrfs has been attempted[1]. Of course, we could not
> 
> But why?  A completeness fetish?  I don't understand why you decided
> to do this work.

* xfs can shapshot only single files, btrfs entire subvolumes
* btrfs-send|receive
* enumeration of changed parts of a file


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ I've read an article about how lively happy music boosts
⣾⠁⢠⠒⠀⣿⡁ productivity.  You can read it, too, you just need the
⢿⡄⠘⠷⠚⠋⠀ right music while doing so.  I recommend Skepticism
⠈⠳⣄ (funeral doom metal).


MaxLinear, please maintain your drivers was Re: [PATCH] leds: lgm: fix gpiolib dependency

2021-03-09 Thread Adam Borowski
On Tue, Mar 09, 2021 at 08:39:10PM +0100, Pavel Machek wrote:
> > I'd like people from Intel to contact me. There's more to fix there,
> > and AFAICT original author went away.
> 
> The following message to  was
> undeliverable.

> : Recipient
> +address rejected: User unknown in virtual mailbox table'

> commit c3987cd2bca34ddfec69027acedb2fae5ffcf7a0
> Author: Amireddy Mallikarjuna reddy 

I asked around, and got told Mallikarjuna has been "sold" to MaxLinear,
together with the rest of the Connected Home Division.  So he most likely
still works on this stuff, just under a different banner.

> If someone knows how to contact the author, that would be welcome.

Alas, no idea about his MaxLinear address.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ in the beginning was the boot and root floppies and they were good.
⢿⡄⠘⠷⠚⠋⠀   --  on #linux-sunxi
⠈⠳⣄


Re: TLS for 5.10

2021-02-08 Thread Adam Borowski
On Mon, Feb 08, 2021 at 07:08:05AM +, Wadepohl, Wolfram wrote:
> I'm sad to hear that 5.10 has still an EOL of Dec, 2022. We are in
> development of our 1st GNU/Linux based System,  50k devices for IoT.
[...]
> In general a 2 year support for embedded systems in automation is not a
> reasonable thing. Nevertheless the CIP project has commited to 5.10 as the
> next SLTS kernel.
[...]
> What can we do as a very small development team to support an extended LTS
> period of 5.10.?

The two years are a minimal promise from Greg.  Debian is using 5.10 for
upcoming Bullseye, thus even if Greg won't extend (99% chance he will), Ben
Hutchings will continue the support for as long as Debian Bullseye is alive.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ A dumb species has no way to open a tuna can.
⢿⡄⠘⠷⠚⠋⠀ A smart species invents a can opener.
⠈⠳⣄ A master species delegates.


Re: 5.10 LTS Kernel: 2 or 6 years?

2021-01-26 Thread Adam Borowski
On Mon, Jan 25, 2021 at 11:55:11AM -0800, Scott Branden wrote:
> The 5.10 LTS kernel being officially LTS supported for 2 years presents a 
> problem:
> why would anyone select a 5.10 kernel with 2 year LTS when 5.4 kernel has a 6 
> year LTS.
> 
> Yet, various unofficial reports indicate it will be supported for 6
> years.  And AOSP has already declared the use of 5.10 kernel in their
> Android S and T releases.

5.10 will also be used for Debian Bullseye.

-- 
⢀⣴⠾⠻⢶⣦⠀ .--[ Makefile ]
⣾⠁⢠⠒⠀⣿⡁ # beware of races
⢿⡄⠘⠷⠚⠋⠀ all: pillage burn
⠈⠳⣄ `


Re: Linux 5.11-rc1

2021-01-03 Thread Adam Borowski
On Sun, Jan 03, 2021 at 05:45:16PM +, David Laight wrote:
> From: Ilkka Prusi
> > Note that /sbin is now just a symlink to /usr/sbin on Debian since 10 
> > (Buster) as per FHS[1][2].
> > 
> > [1] https://wiki.linuxfoundation.org/lsb/fhs
> > [2] 
> > https://arstechnica.com/information-technology/2019/09/debian-10-playing-catch-up-with-the-rest-
> > of-the-linux-world-thats-a-good-thing/
> 
> Which is exactly 100% backwards :-)

I agree with you that, if all, it's /usr/sbin which should be a symlink,
to reduce typing and to get rid of a vestige of _The_ Unix machine having
/bin spill to a disk that used to have user homes.

But, /sbin is a symlink on only _some_ Debian installations.  There's
currently a discussion whether to go deeper into this scheme, abandon it
or do nothing.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ .--[ Makefile ]
⣾⠁⢠⠒⠀⣿⡁ # beware of races
⢿⡄⠘⠷⠚⠋⠀ all: pillage burn
⠈⠳⣄ `


Re: [PATCH] tty: Remove dead termiox code

2020-12-07 Thread Adam Borowski
On Fri, Dec 04, 2020 at 10:10:16AM +0100, Greg Kroah-Hartman wrote:
> On Fri, Dec 04, 2020 at 09:51:07AM +0100, Jiri Slaby wrote:
> > > > > On Fri, Dec 04, 2020 at 08:22:41AM +0100, Jiri Slaby wrote:
> > > > > > On 03. 12. 20, 3:03, Jann Horn wrote:
> > > > > > > Delete this dead code; but leave the definition of struct termiox 
> > > > > > > in the
> > > > > > > UAPI headers intact.
[was snipped]
> > > > > > I am thinking -- can/should we mark the structure as deprecated so 
> > > > > > that 
> > > > > > userspace stops using it eventually?   

> > Note this ^. He is talking about _not_ touching the definition in the
> > UAPI header. Does the rest below makes more sense now?
> 
> No, I'm still confused :)
> 
> We can't touch the UAPI definitions, but the fact that this api never
> did anything still is ok as after this patch it continues to not do
> anything.
> 
> I'm confused as to what you are proposing...

The UAPI definition can't be removed, but it would be nice to issue a
compiler _warning_ if it's ever used.

Like eg. __attribute__ ((deprecated))


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀.--[ Makefile ]
⣾⠁⢠⠒⠀⣿⡁# beware of races
⢿⡄⠘⠷⠚⠋⠀all: pillage burn
⠈⠳⣄`


Re: [PATCH 2/2] printk: Make the console flush configurable in hotplug path

2020-09-25 Thread Adam Borowski
On Fri, Sep 25, 2020 at 11:27:54AM +0200, Greg KH wrote:
> On Thu, Sep 24, 2020 at 08:21:07PM +0200, Thomas Gleixner wrote:
> > On Thu, Sep 24 2020 at 08:33, Greg KH wrote:
> > > On Wed, Sep 23, 2020 at 05:08:32PM -0700, Prasad Sodagudi wrote:
> > >> +config CONSOLE_FLUSH_ON_HOTPLUG
> > >> +bool "Enable console flush configurable in hot plug code path"

> > CPU hotplug is not meant to be a high speed operation and if people
> > think they need it to be fast then its pretty much guaranteed that they
> > want it for the completely wrong reasons.
> 
> Odds are, it's the big/little systems that are trying to use cpu hotplug
> for this type of thing :(

Just a bit of info:
My MT6797X (10 core: 4×A53 + 4×A53 + 2×A72), flickers its cores this way:
the right-hand piece is CPUs, one character per core: bars show utilization,
"o" stands for offline; every line is 0.1 second interval.

topline -i 0.1
mmcblk(⠀) (▄▆oo▅o)
mmcblk(⠀) (▅▄)
mmcblk(⠀) (▆▆)
mmcblk(⠀) (▅ooo▆o)
mmcblk(⠀) (▆▆oo▄o)
mmcblk(⠀) (▆▇)
mmcblk(⠀) (▇ooo▅o)
mmcblk(⠀) (▆ooo█o)
mmcblk(⠀) (▆ooo▄o)
mmcblk(⠀) (▅ooo▆o)
mmcblk(⠀) (▆ooo▅o)
mmcblk(⠀) (▄ooo▇o)
mmcblk(⠀) (▇▆oo▆o)
mmcblk(⠀) (▆ooo▅o)
mmcblk(⠀) (▅▆)
mmcblk(⠀) (▆█)
mmcblk(⠀) (▆▇)
mmcblk(⠀) (▆▆)
mmcblk(⠀) (▅▆)
mmcblk(⠀) (▆▅)
mmcblk(⠀) (▆ooo▆o)
mmcblk(⠀) (▆ooo▇o)
mmcblk(⢀) (▆▇oo▆o)
mmcblk(⠀) (▄ooo▆o)
mmcblk(⠀) (▆ooo█o)
mmcblk(⠀) (▄ooo▇o)
mmcblk(⠀) (▄▆)
mmcblk(⠀) (▆▆)

So it's on the order of a few ons/offs per second.

The offline CPUs are "present" and "offline"; not sure if this means hotplug
or not (I'd expect dropping from "present" to "possible", but I don't know
these parts).


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ in the beginning was the boot and root floppies and they were good.
⢿⡄⠘⠷⠚⠋⠀   --  on #linux-sunxi
⠈⠳⣄


Re: fbcon: remove soft scrollback code (missing Doc. patch)

2020-09-18 Thread Adam Borowski
On Wed, Sep 16, 2020 at 10:54:34PM +0200, Pavel Machek wrote:
> On Mon 2020-09-14 18:28:34, Linus Torvalds wrote:
> > Note that scrollback hasn't actually gone away entirely - the original
> > scrollback supported by _hardware_ still exists.
> > 
> > Of course, that's really just the old-fashioned text VGA console, but
> > that one actually scrolls not by moving any bytes around, but by
> > moving the screen start address. And the scrollback similarly isn't
> > about any software buffering, but about the ability of moving back
> > that screen start address.

> Could we pause this madness? Scrollback is still useful. I needed it
> today... it was too small, so command results I was looking for
> already scrolled away, but... life will be really painful with 0 scrollback.
> 
> You'll need it, too... as soon as you get oops and will want to see
> errors just prior to that oops.

I concur -- this a serious usability regression for regular users.  Linus:
you have a serial cable on your main dev machine, so do I, but hardly any
regular people do -- that's restricted to mostly IPMI and such.

And without some kind of scrollback, there's no way of knowing why eg.
your rootfs failed to mount (there was some oops, but its reason was at
the beginning...).  Or, any other problem the user would be able to solve,
or pass the error messages to someone more knowledgeable.

I also wonder why did you choose to remove softscrollback which is actually
useful, yet leave hardscrollback which doesn't come to use on any
non-ancient hardware:
* on !x86 there's no vgacon at all
* on x86, in-tree drivers for GPUs by Intel, nVidia and AMD (others are
  dead) default to switching away from vgacon
* EFI wants its own earlycon
... thus, the only niche left is nVidia proprietary drivers which, the last
time I looked, still used CGA text mode.

> If it means I get to maintain it... I'm not happy about it but that's
> better than no scrollback.

That'd be greatly appreciated.  There are also some simplifications/rewrites
that could be done, like getting rid of redundant 1-byte/4-byte storage (or
even the code for 1-byte...).  Hard scrollback could be axed altogether (it
provides only a small amount of scroll).  Etc...

>  Kernel is now very verbose, so important messages
>  during bootup scroll away. It is way bigger deal when you can no
>  longer get to them using shift-pageup.

Thus hard scrollback is inadequate in the rare cases it's even present.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ in the beginning was the boot and root floppies and they were good.
⢿⡄⠘⠷⠚⠋⠀   --  on #linux-sunxi
⠈⠳⣄


build failure: aicasm: renamed yaccage

2020-08-22 Thread Adam Borowski
Hi!
My randconfig builds notoriously fail on this:

[~/linux/drivers/scsi/aic7xxx/aicasm](vanilla)$ make -j1
bison -d -b aicasm_gram aicasm_gram.y
mv aicasm_gram.tab.c .//aicasm_gram.c
mv aicasm_gram.tab.h .//aicasm_gram.h
bison -d -b aicasm_macro_gram -p mm aicasm_macro_gram.y
mv aicasm_macro_gram.tab.c .//aicasm_macro_gram.c
mv aicasm_macro_gram.tab.h .//aicasm_macro_gram.h
flex  -o aicasm_scan.c aicasm_scan.l
flex  -Pmm -o aicasm_macro_scan.c aicasm_macro_scan.l
cc -I/usr/include -I. -I./ aicasm.c aicasm_symbol.c .//aicasm_gram.c 
.//aicasm_macro_gram.c .//aicasm_scan.c .//aicasm_macro_scan.c -o .//aicasm -ldb
aicasm_symbol.c: In function ‘aic_print_reg_dump_end’:
aicasm_symbol.c:393:13: warning: implicit declaration of function ‘tolower’ 
[-Wimplicit-function-declaration]
  393 |   *letter = tolower(*letter);
  | ^~~
aicasm_gram.tab.c:204:10: fatal error: aicasm_gram.tab.h: No such file or 
directory
compilation terminated.
aicasm_macro_gram.tab.c:167:10: fatal error: aicasm_macro_gram.tab.h: No such 
file or directory
compilation terminated.

And the generated yaccage has:
#include "aicasm_gram.tab.h"
which tries to refer to the just renamed file.

As the files in question are generated, with the filename coming from $YACC
rather than source, it'd take some after-processing with sed or a similar
hack.  Thus, instead of sending a patch, I thought it'd better to ask:
what the renames are for?


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁
⢿⡄⠘⠷⠚⠋⠀ It's time to migrate your Imaginary Protocol from version 4i to 6i.
⠈⠳⣄


Re: [PATCH] x86/pci: don't set acpi stuff if !CONFIG_ACPI

2020-08-21 Thread Adam Borowski
On Fri, Aug 21, 2020 at 10:13:25PM +0200, Thomas Gleixner wrote:
> On Thu, Aug 20 2020 at 14:53, Adam Borowski wrote:
> > Found by randconfig builds.
> >
> >  arch/x86/pci/intel_mid_pci.c | 2 ++
> >  arch/x86/pci/xen.c   | 2 ++

> > --- a/arch/x86/pci/intel_mid_pci.c
> > +++ b/arch/x86/pci/intel_mid_pci.c
> > @@ -299,8 +299,10 @@ int __init intel_mid_pci_init(void)
> > +#ifdef CONFIG_ACPI
> > /* Continue with standard init */
> > acpi_noirq_set();
> > +#endif

> If CONFIG_ACPI=n then acpi_noirq_set() is an empty stub inline. So I'm
> not sure what you are trying to solve here.
> 
> Ah, I see with CONFIG_ACPI=n linux/acpi.h does not include asm/acpi.h so
> the stubs are unreachable. So that needs to be fixed and not papered
> over with #ifdeffery

If I understand Randy Dunlap correctly, he already sent a pair of patches
that do what you want.


Meow.
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁
⢿⡄⠘⠷⠚⠋⠀ It's time to migrate your Imaginary Protocol from version 4i to 6i.
⠈⠳⣄


Re: [PATCH v10 4/8] usr: add support for zstd compressed initramfs

2020-08-04 Thread Adam Borowski
On Tue, Aug 04, 2020 at 09:25:23AM +0200, Sedat Dilek wrote:
> On Tue, Aug 4, 2020 at 8:52 AM Geert Uytterhoeven  
> wrote:
> > On Thu, Jul 30, 2020 at 9:13 PM Nick Terrell  wrote:
> > > From: Nick Terrell 
> > > * Add support for a zstd compressed initramfs.
> > > * Add compression for compressing built-in initramfs with zstd.

> > > --- a/usr/Kconfig
> > > +++ b/usr/Kconfig
> > > @@ -100,6 +100,15 @@ config RD_LZ4
> > >   Support loading of a LZ4 encoded initial ramdisk or cpio buffer
> > >   If unsure, say N.
> > >
> > > +config RD_ZSTD
> > > +   bool "Support initial ramdisk/ramfs compressed using ZSTD"
> > > +   default y
> > > +   depends on BLK_DEV_INITRD
> > > +   select DECOMPRESS_ZSTD
> > > +   help
> > > + Support loading of a ZSTD encoded initial ramdisk or cpio 
> > > buffer.
> > > + If unsure, say N.
> >
> > I'm aware you copied this from the other entries, but IMHO "default y",
> > and "If unsure, say N" are not a good combination.

> you are right - for new stuff it should be "default n".

It got already applied to Linus' tree with "y", and I think it'd be nice
to have it as a default.  Let's disable other compressors instead.

On the other hand, having an unsupported rd compressor results in a boot
failure that's not immediately obvious, so that's a reason for keeping
the setting as "y".

On the third hand, distributions default to either gz or xz, thus I'd say:
* let's have gz xz zstd default to y, all others to n
* drop bzip2 lzma1 completely
* distros can't switch the mkinitramfs default yet, but if RD_ZSTD=y now,
  they'll be able to once they drop support for old kernels in a few years

> What I am missing - still - is a note - that your user-space should
> have the correct bits to support zstd-initramfs.
> Unsure where to place such an information.

Looks like INITRAMFS_COMPRESSION_* have lengthy prose but are not shown in
menuconfig, while RD_*, with no such prose, are shown.

The prose itself is grossly obsolete, too.  I have some updates in:
https://github.com/kilobyte/linux/commits/nobz2-v3
but that patchset needs rebasing and refreshing.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁
⢿⡄⠘⠷⠚⠋⠀ It's time to migrate your Imaginary Protocol from version 4i to 6i.
⠈⠳⣄


Re: [PATCH v8 6/7] x86: Add support for ZSTD compressed kernel

2020-07-24 Thread Adam Borowski
On Fri, Jul 24, 2020 at 02:26:40PM +0200, Ingo Molnar wrote:
> > -#ifdef CONFIG_KERNEL_BZIP2
> > +#if defined(CONFIG_KERNEL_BZIP2)
> >  # define BOOT_HEAP_SIZE0x40
> > -#else /* !CONFIG_KERNEL_BZIP2 */
> > +#elif defined(CONFIG_KERNEL_ZSTD)
> > +# define BOOT_HEAP_SIZE 0x3
> > +#else
> >  # define BOOT_HEAP_SIZE 0x1
> >  #endif
> 
> So the other patches explain why the decompression buffer extra space 
> was increased from 64k to 128k, but is there a similar 
> calculation/estimate for bumping BOOT_HEAD_SIZE from 64k to 192k?
> 
> Admittedly the BZ2 exception doesn't set a good example, but maybe we 
> can do this for ZSTD?

By the way, I have a patchset on top of this, to drop BZ2 and LZMA(1)
support, that should clean up this code somewhat.  And bring a lot of
lines of Linus happiness, as both bzip2 and lzma code are not used by
anything else in the kernel, unlike lzma2 (xz).

If you draw a speed-vs-size graph, at no point bzip2 or lzma are a good
choice, while zstd wins by a large margin for most of the range.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁
⢿⡄⠘⠷⠚⠋⠀ It's time to migrate your Imaginary Protocol from version 4i to 6i.
⠈⠳⣄


Re: [PATCH] kbuild: fix broken builds because of GZIP,BZIP2,LZOP variables

2020-06-08 Thread Adam Borowski
On Mon, Jun 08, 2020 at 12:59:44PM +0300, Denis Efremov wrote:
> Redefine GZIP, BZIP2, LZOP variables as KGZIP, KBZIP2, KLZOP resp.
> GZIP, BZIP2, LZOP env variables are reserved by the tools. The original
> attempt to redefine them internally doesn't work in makefiles/scripts
> intercall scenarios, e.g., "make GZIP=gzip bindeb-pkg" and results in
> broken builds. There can be other broken build commands because of this,
> so the universal solution is to use non-reserved env variables for the
> compression tools.
> 
> Fixes: 8dfb61dcbace ("kbuild: add variables for compression tools")

Same said my bisect before I noticed your fix. :)

> Signed-off-by: Denis Efremov 

However, I run just basic "make bindeb-pkg" without forcing any variables,
thus the commit message is wrong.

Yet, your patch fixes the functionality.  Thanks!

> ---
>  Makefile  | 24 +---
>  arch/arm/boot/deflate_xip_data.sh |  2 +-
>  arch/ia64/Makefile|  2 +-
>  arch/m68k/Makefile|  8 
>  arch/parisc/Makefile  |  2 +-
>  scripts/Makefile.lib  |  6 +++---
>  scripts/Makefile.package  |  6 +++---
>  scripts/package/buildtar  |  4 ++--
>  8 files changed, 20 insertions(+), 34 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index 839f9fee22cb..e43d193bb3b2 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -458,27 +458,13 @@ PYTHON  = python
>  PYTHON3  = python3
>  CHECK= sparse
>  BASH = bash
> -GZIP = gzip
> -BZIP2= bzip2
> -LZOP = lzop
> +KGZIP= gzip
> +KBZIP2   = bzip2
> +KLZOP= lzop
>  LZMA = lzma
>  LZ4  = lz4c
>  XZ   = xz
>  
> -# GZIP, BZIP2, LZOP env vars are used by the tools. Support them as the 
> command
> -# line interface, but use _GZIP, _BZIP2, _LZOP internally.
> -_GZIP  := $(GZIP)
> -_BZIP2 := $(BZIP2)
> -_LZOP  := $(LZOP)
> -
> -# Reset GZIP, BZIP2, LZOP in this Makefile
> -override GZIP=
> -override BZIP2=
> -override LZOP=
> -
> -# Reset GZIP, BZIP2, LZOP in recursive invocations
> -MAKEOVERRIDES += GZIP= BZIP2= LZOP=
> -
>  CHECKFLAGS := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ \
> -Wbitwise -Wno-return-void -Wno-unknown-attribute $(CF)
>  NOSTDINC_FLAGS :=
> @@ -526,7 +512,7 @@ CLANG_FLAGS :=
>  export ARCH SRCARCH CONFIG_SHELL BASH HOSTCC KBUILD_HOSTCFLAGS CROSS_COMPILE 
> LD CC
>  export CPP AR NM STRIP OBJCOPY OBJDUMP OBJSIZE READELF PAHOLE LEX YACC AWK 
> INSTALLKERNEL
>  export PERL PYTHON PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX
> -export _GZIP _BZIP2 _LZOP LZMA LZ4 XZ
> +export KGZIP KBZIP2 KLZOP LZMA LZ4 XZ
>  export KBUILD_HOSTCXXFLAGS KBUILD_HOSTLDFLAGS KBUILD_HOSTLDLIBS 
> LDFLAGS_MODULE
>  
>  export KBUILD_CPPFLAGS NOSTDINC_FLAGS LINUXINCLUDE OBJCOPYFLAGS 
> KBUILD_LDFLAGS
> @@ -1047,7 +1033,7 @@ export mod_strip_cmd
>  mod_compress_cmd = true
>  ifdef CONFIG_MODULE_COMPRESS
>ifdef CONFIG_MODULE_COMPRESS_GZIP
> -mod_compress_cmd = $(_GZIP) -n -f
> +mod_compress_cmd = $(KGZIP) -n -f
>endif # CONFIG_MODULE_COMPRESS_GZIP
>ifdef CONFIG_MODULE_COMPRESS_XZ
>  mod_compress_cmd = $(XZ) -f
> diff --git a/arch/arm/boot/deflate_xip_data.sh 
> b/arch/arm/boot/deflate_xip_data.sh
> index 739f0464321e..304495c3c2c5 100755
> --- a/arch/arm/boot/deflate_xip_data.sh
> +++ b/arch/arm/boot/deflate_xip_data.sh
> @@ -56,7 +56,7 @@ trap 'rm -f "$XIPIMAGE.tmp"; exit 1' 1 2 3
>  # substitute the data section by a compressed version
>  $DD if="$XIPIMAGE" count=$data_start iflag=count_bytes of="$XIPIMAGE.tmp"
>  $DD if="$XIPIMAGE"  skip=$data_start iflag=skip_bytes |
> -$_GZIP -9 >> "$XIPIMAGE.tmp"
> +$KGZIP -9 >> "$XIPIMAGE.tmp"
>  
>  # replace kernel binary
>  mv -f "$XIPIMAGE.tmp" "$XIPIMAGE"
> diff --git a/arch/ia64/Makefile b/arch/ia64/Makefile
> index f817f3d5e758..2876a7df1b0a 100644
> --- a/arch/ia64/Makefile
> +++ b/arch/ia64/Makefile
> @@ -40,7 +40,7 @@ $(error Sorry, you need a newer version of the assember, 
> one that is built from
>  endif
>  
>  quiet_cmd_gzip = GZIP$@
> -cmd_gzip = cat $(real-prereqs) | $(_GZIP) -n -f -9 > $@
> +cmd_gzip = cat $(real-prereqs) | $(KGZIP) -n -f -9 > $@
>  
>  quiet_cmd_objcopy = OBJCOPY $@
>  cmd_objcopy = $(OBJCOPY) $(OBJCOPYFLAGS) $(OBJCOPYFLAGS_$(@F)) $< $@
> diff --git a/arch/m68k/Makefile b/arch/m68k/Makefile
> index ce6db5e5a5a3..0415d28dbe4f 100644
> --- a/arch/m68k/Makefile
> +++ b/arch/m68k/Makefile
> @@ -135,10 +135,10 @@ vmlinux.gz: vmlinux
>  ifndef CONFIG_KGDB
>   cp vmlinux vmlinux.tmp
>   $(STRIP) vmlinux.tmp
> - $(_GZIP) -9c vmlinux.tmp >vmlinux.gz
> + $(KGZIP) -9c vmlinux.tmp >vmlinux.gz
>   rm vmlinux.tmp
>  else
> - $(_GZIP) -9c vmlinux >vmlinux.gz
> + $(KGZIP) -9c vmlinux >vmlinux.gz
>  endif
>  
>  bzImage: vmlinux.bz2
> @@ -148,10 +148,10 @@ vmlinux.bz2: vmlinux
>  ifndef 

Re: boot failure: stack-protector: Kernel stack is corrupted in: start_secondary

2020-04-28 Thread Adam Borowski
On Tue, Apr 28, 2020 at 09:45:27AM +0200, Borislav Petkov wrote:
> On Tue, Apr 21, 2020 at 03:32:34AM +0200, Adam Borowski wrote:
> > Hi!
> > With kernels compiled with gcc-10, on two different machines (AMD Phenom2,
> > AMD 2990WX) I get the following panic during boot:
> 
> Welcome to the party:
> 
> https://git.kernel.org/tip/f670269a42bfdd2c83a1118cc3d1b475547eac22
> 
> Try this branch to check whether it works for ya:
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git/log/?h=x86/build

✓


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ in the beginning was the boot and root floppies and they were good.
⢿⡄⠘⠷⠚⠋⠀   --  on #linux-sunxi
⠈⠳⣄


Re: Stop breaking the CSRNG

2019-10-03 Thread Adam Borowski
On Thu, Oct 03, 2019 at 10:13:39AM +, David Laight wrote:
> From: Kurt Roeckx
> > Sent: 02 October 2019 17:56
> > As OpenSSL, we want cryptograhic secure random numbers. Before
> > getrandom(), Linux never provided a good API for that, both
> > /dev/random and /dev/urandom have problems. getrandom() fixed
> > that, so we switched to it were available.
> 
> The fundamental problem is that you can't always get ' cryptograhic secure
> random numbers'. No API changes are ever going to change that.
> 
> The system can either return an error or sleep (possibly indefinitely)
> until some 'reasonably random' numbers are available.
> 
> A RISC-V system running on an FGPA (I've only used Altera NIOS cpu)
> may have absolutely no sources of randomness at boot time.

I'd say this is a hardware security vulnerability; no different from eg.
having no or faulty MMU, speculation that allows exfiltrating data, etc.
We did not understand the seriousness of lacking hardware sources of
randomness, but that's a common thing to many other security
vulnerabilities.

Machines that lack any sources of entropy have their uses, but they're akin
to processors with no MMU.  You should never run a world-accessible ssh
daemon on either of them.

> Saying the architecture must include a random number instruction
> doesn't help!

It won't fix existing systems, and is irrelevant to deeply embedded, but
communicating this requirement to SoC designers sounds like a good idea to
me.  IoTrash appliance makers won't care but their security is already so
atrocious that lack of entropy is nowhere near the easiest way to get in,
while anyone else will at least notice the warning.

Any real-silicon hardware can include an entropy source, and if it doesn't,
shaming the maker is the way to go.  Calling the problem a security
vulnerability (which I say it is) sends a stronger message.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ A MAP07 (Dead Simple) raspberry tincture recipe: 0.5l 95% alcohol,
⣾⠁⢠⠒⠀⣿⡁ 1kg raspberries, 0.4kg sugar; put into a big jar for 1 month.
⢿⡄⠘⠷⠚⠋⠀ Filter out and throw away the fruits (can dump them into a cake,
⠈⠳⣄ etc), let the drink age at least 3-6 months.


[PATCH] Documentation: sysrq: don't recommend 'S' 'U' before 'B'

2019-09-03 Thread Adam Borowski
This advice is obsolete and slightly harmful for filesystems from this
millenium: any modern filesystem can handle unexpected crashes without
requiring fsck -- and on the other hand, trying to write to the disk when
the kernel is in a bad state risks introducing corruption.

For ext2, any unsafe shutdown meant widespread breakage, but it's no longer
a reasonable filesystem for any non-special use.

Signed-off-by: Adam Borowski 
---
 Documentation/admin-guide/sysrq.rst | 20 +---
 1 file changed, 9 insertions(+), 11 deletions(-)

diff --git a/Documentation/admin-guide/sysrq.rst 
b/Documentation/admin-guide/sysrq.rst
index 7b9035c01a2e..72b2cfb066f4 100644
--- a/Documentation/admin-guide/sysrq.rst
+++ b/Documentation/admin-guide/sysrq.rst
@@ -171,22 +171,20 @@ It seems others find it useful as (System Attention Key) 
which is
 useful when you want to exit a program that will not let you switch consoles.
 (For example, X or a svgalib program.)
 
-``reboot(b)`` is good when you're unable to shut down. But you should also
-``sync(s)`` and ``umount(u)`` first.
+``reboot(b)`` is good when you're unable to shut down, it is an equivalent
+of pressing the "reset" button.
 
 ``crash(c)`` can be used to manually trigger a crashdump when the system is 
hung.
 Note that this just triggers a crash if there is no dump mechanism available.
 
-``sync(s)`` is great when your system is locked up, it allows you to sync your
-disks and will certainly lessen the chance of data loss and fscking. Note
-that the sync hasn't taken place until you see the "OK" and "Done" appear
-on the screen. (If the kernel is really in strife, you may not ever get the
-OK or Done message...)
+``sync(s)`` is handy before yanking removable medium or after using a rescue
+shell that provides no graceful shutdown -- it will ensure your data is
+safely written to the disk. Note that the sync hasn't taken place until you see
+the "OK" and "Done" appear on the screen.
 
-``umount(u)`` is basically useful in the same ways as ``sync(s)``. I generally
-``sync(s)``, ``umount(u)``, then ``reboot(b)`` when my system locks. It's saved
-me many a fsck. Again, the unmount (remount read-only) hasn't taken place until
-you see the "OK" and "Done" message appear on the screen.
+``umount(u)`` can be used to mark filesystems as properly unmounted. From the
+running system's point of view, they will be remounted read-only. The remount
+isn't complete until you see the "OK" and "Done" message appear on the screen.
 
 The loglevels ``0``-``9`` are useful when your console is being flooded with
 kernel messages you do not want to see. Selecting ``0`` will prevent all but
-- 
2.23.0



Re: [PATCH v3 0/8] AMD64 EDAC fixes

2019-08-22 Thread Adam Borowski
On Thu, Aug 22, 2019 at 10:35:48AM +0200, Borislav Petkov wrote:
> On Thu, Aug 22, 2019 at 02:50:20AM +0200, Adam Borowski wrote:
> > While you're editing that code, could you please also cut the spam if ECC is
> > actually disabled?  For example, a 2990WX with non-ECC RAM gets 1024 lines;
> 
> Patch is in there. I'll give you extra points if you spot it.

Yeah, some of messages are no longer emitted for memory-less nodes (NUMA 1
and 3).  Your patch set also overhauls the messages.

But, the amount of redundant messages I'm complaining about has actually
increased:

dmesg|grep EDAC|cut -c 16-|sort|uniq -c
256 EDAC MC: UMC0 chip selects:
256 EDAC MC: UMC1 chip selects:
  1 EDAC MC: Ver: 3.0.0
128 EDAC amd64: ECC disabled in the BIOS or no ECC capability, module will 
not load.
^ three lines each
 64 EDAC amd64: F17h detected (node 0).
 64 EDAC amd64: F17h detected (node 1).
 64 EDAC amd64: F17h detected (node 2).
 64 EDAC amd64: F17h detected (node 3).
512 EDAC amd64: MC: 0: 0MB 1: 0MB
256 EDAC amd64: MC: 2: 0MB 3: 0MB
256 EDAC amd64: MC: 2:  8192MB 3: 0MB
 64 EDAC amd64: Node 0: DRAM ECC disabled.
 64 EDAC amd64: Node 2: DRAM ECC disabled.
256 EDAC amd64: using x4 syndromes.

(Full dmesg at http://ix.io/1T1o)

While on 5.3-rc5 without the patchset I get:

  1 EDAC MC: Ver: 3.0.0
256 EDAC amd64: ECC disabled in the BIOS or no ECC capability, module will 
not load.
^ three lines each
 64 EDAC amd64: Node 0: DRAM ECC disabled.
 64 EDAC amd64: Node 1: DRAM ECC disabled.
 64 EDAC amd64: Node 2: DRAM ECC disabled.
 64 EDAC amd64: Node 3: DRAM ECC disabled.

So I wonder if we could deduplicate those.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢠⠒⠀⣿⡁ A dumb species has no way to open a tuna can.
⢿⡄⠘⠷⠚⠋⠀ A smart species invents a can opener.
⠈⠳⣄ A master species delegates.


Re: [PATCH v3 0/8] AMD64 EDAC fixes

2019-08-21 Thread Adam Borowski
On Wed, Aug 21, 2019 at 11:59:53PM +, Ghannam, Yazen wrote:
> I've also added RFC patches to avoid the "ECC disabled" message for
> nodes without memory. I haven't fully tested these, but I wanted to get
> your thoughts. Here's an earlier discussion:
> https://lkml.kernel.org/r/20180321191335.7832-1-yazen.ghan...@amd.com

While you're editing that code, could you please also cut the spam if ECC is
actually disabled?  For example, a 2990WX with non-ECC RAM gets 1024 lines;
64 copies of:

[8.186164] EDAC amd64: Node 0: DRAM ECC disabled.
[8.188364] EDAC amd64: ECC disabled in the BIOS or no ECC capability, 
module will not load.
Either enable ECC checking or force module loading by setting 
'ecc_enable_override'.
(Note that use of the override may cause unknown side effects.)
[8.194762] EDAC amd64: Node 1: DRAM ECC disabled.
[8.196307] EDAC amd64: ECC disabled in the BIOS or no ECC capability, 
module will not load.
Either enable ECC checking or force module loading by setting 
'ecc_enable_override'.
(Note that use of the override may cause unknown side effects.)
[8.199840] EDAC amd64: Node 2: DRAM ECC disabled.
[8.200963] EDAC amd64: ECC disabled in the BIOS or no ECC capability, 
module will not load.
Either enable ECC checking or force module loading by setting 
'ecc_enable_override'.
(Note that use of the override may cause unknown side effects.)
[8.204326] EDAC amd64: Node 3: DRAM ECC disabled.
[8.205436] EDAC amd64: ECC disabled in the BIOS or no ECC capability, 
module will not load.
Either enable ECC checking or force module loading by setting 
'ecc_enable_override'.
(Note that use of the override may cause unknown side effects.)


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁
⢿⡄⠘⠷⠚⠋  The root of a real enemy is an imaginary friend.
⠈⠳⣄


Re: Device to write to all (serial) consoles

2019-08-03 Thread Adam Borowski
On Sat, Aug 03, 2019 at 03:55:37PM +0200, Greg Kroah-Hartman wrote:
> On Sat, Aug 03, 2019 at 03:23:23PM +0200, Adam Borowski wrote:
> > On Fri, Aug 02, 2019 at 09:59:06PM +0200, Paul Menzel wrote:
> > > Because the cable is always connected to the port on the back side, and
> > > sometimes the port in the front has ID 0, and the one in the back 1, and
> > > other times vice versa. We do not want to track that, and it would be
> > > convenient to just write to both ports.
> > 
> > Sounds like an XY problem then: what you want is not writing to all ports,
> > but to have the port assignments stable (see also: disk device reordering).
> 
> You can get that information from the symlinks in /dev/serial/ which
> udev creates.

Doesn't seem to work for me, for any ttyS0 ttyS1 ttySAC1 device:

Box one, PCIe card + one USB dongle:
07:00.0 Serial controller: MosChip Semiconductor Technology Ltd. MCS9922 PCIe 
Multi-I/O Controller
07:00.1 Serial controller: MosChip Semiconductor Technology Ltd. MCS9922 PCIe 
Multi-I/O Controller
Bus 003 Device 004: ID 1a86:7523 QinHeng Electronics HL-340 USB-Serial adapter

/dev/serial/by-id/usb-1a86_USB2.0-Serial-if00-port0
/dev/serial/by-path/pci-:0b:00.3-usb-0:4:1.0-port0

Only ttyUSB0 is there.


Box two, on-board + one USB dongle:
[3.404340] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[3.431287] 00:01: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 
16550A
Bus 001 Device 002: ID 10c4:ea60 Cygnal Integrated Products, Inc. CP2102/CP2109 
UART Bridge Controller [CP210x family]

/dev/serial/by-id/usb-Silicon_Labs_CP2104_USB_to_UART_Bridge_Controller_00DB1604-if00-port0
/dev/serial/by-path/pci-:00:14.0-usb-0:1:1.0-port0


Box three: RockPro64, euler + USB dongle, kernel 4.4.
Box four: Pine64, euler.
Box five: Odroid-U2, something GPIOish (ttySAC1), kernel 5.0.


Most are running kernel 5.2, Debian unstable.

And indeed, in /lib/udev/rules.d/60-serial.rules :
# /dev/serial/by-path/, /dev/serial/by-id/ for USB devices
KERNEL!="ttyUSB[0-9]*|ttyACM[0-9]*", GOTO="serial_end"


Like me, Paul is using ttyS0 for server-side.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢰⠒⠀⣿⡁ 10 people enter a bar: 1 who understands binary,
⢿⡄⠘⠷⠚⠋⠀ 1 who doesn't, D who prefer to write it as hex,
⠈⠳⣄ and 1 who narrowly avoided an off-by-one error.


Re: Device to write to all (serial) consoles

2019-08-03 Thread Adam Borowski
On Fri, Aug 02, 2019 at 09:59:06PM +0200, Paul Menzel wrote:
> On 02.08.19 18:02, Greg Kroah-Hartman wrote:
> > On Fri, Aug 02, 2019 at 03:23:08PM +0200, Paul Menzel wrote:
> > > On a lot of devices, like servers, you have more than one serial console,
> > > and you do not always know, how they are numbered. Therefore, we start a
> > > console on ttyS0 and ttyS1.
> 
> Because the cable is always connected to the port on the back side, and
> sometimes the port in the front has ID 0, and the one in the back 1, and
> other times vice versa. We do not want to track that, and it would be
> convenient to just write to both ports.

Sounds like an XY problem then: what you want is not writing to all ports,
but to have the port assignments stable (see also: disk device reordering).


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ Latin:   meow 4 characters, 4 columns,  4 bytes
⣾⠁⢠⠒⠀⣿⡁ Greek:   μεου 4 characters, 4 columns,  8 bytes
⢿⡄⠘⠷⠚⠋  Runes:   ᛗᛖᛟᚹ 4 characters, 4 columns, 12 bytes
⠈⠳⣄ Chinese: 喵   1 character,  2 columns,  3 bytes <-- best!


Re: [RFC] remove arch/sh?

2019-06-25 Thread Adam Borowski
On Tue, Jun 25, 2019 at 11:02:36AM +0200, John Paul Adrian Glaubitz wrote:
> On 6/25/19 10:56 AM, Christoph Hellwig wrote:
> > arch/sh seems pretty much unmaintained these days.  The last time I got
> > any reply to sh patches from the list maintainers, and the last maintainer
> > pull request was over a year ago, and even that has been rather sporadic.
> > 
> > In the meantime we've not really seen any updates for new kernel features
> > and code seems to be bitrotting.
> 
> We're still using sh4 in Debian

I wouldn't call it "used": it has popcon of 1, and despite watching many
Debian channels, I don't recall hearing a word about sh4 in quite a while.

Hardware development is dead: we were promised modern silicon by j-core
after original patents expired, but after J2 nothing happened, there was
silence from their side, and now https://j-core.org is down.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Packager's rule #1: upstream _always_ screws something up.  This
⢿⡄⠘⠷⠚⠋⠀ is true especially if you're packaging your own project.
⠈⠳⣄ 


Re: [RESEND PATCH v2 0/2] Add support for ZSTD-compressed kernel

2019-06-07 Thread Adam Borowski
On Fri, Jun 07, 2019 at 07:20:46PM +, Nick Terrell wrote:
> We'd love to get this mainlined as well!
> 
> We're using these patches internally as well. We're seeing an improvement on 
> an
> Intel Atom N3710, where boot time is reduced by one second over using an xz
> compressed kernel. It looks like Ubuntu just switched to a lz4 compressed 
> kernel,
> but zstd is likely a better trade off, because it compresses much better and 
> still has
> excellent decompression speed.
> 
> Since its been nearly a year since I sent these out, I will take some time to 
> rebase
> and retest the patches in case anything changed, and then then resend the 
> patches
> in the next weeks.

Hi!
After the ping, I intended to resend the patch-set (with removals included)
after I return from miniDebconf Hamburg, but you 1. are the author of the
non-trivial part, 2. you have a better test machinery, and 3. I have a
deeply seated preference for effort to be done by people who are not me.

A rebased and working version is at 
https://github.com/kilobyte/linux/tree/nobz2-v3
but there are no real improvements beyond rebases, a typo fix, and Paul Burton's
ACK for mips.

There's an unaddressed comment by Ingo Molnar
https://lore.kernel.org/lkml/20181112042200.ga96...@gmail.com/
for your part of the code.

So what do you suggest?


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ Latin:   meow 4 characters, 4 columns,  4 bytes
⣾⠁⢠⠒⠀⣿⡁ Greek:   μεου 4 characters, 4 columns,  8 bytes
⢿⡄⠘⠷⠚⠋  Runes:   ᛗᛖᛟᚹ 4 characters, 4 columns, 12 bytes
⠈⠳⣄ Chinese: 喵   1 character,  2 columns,  3 bytes <-- best!


Re: [PATCH RFC 0/5] mm: process_vm_mmap() -- syscall for duplication a process mapping

2019-05-16 Thread Adam Borowski
On Thu, May 16, 2019 at 04:10:07PM +0300, Kirill Tkhai wrote:
> On 15.05.2019 22:38, Adam Borowski wrote:
> > On Wed, May 15, 2019 at 06:11:15PM +0300, Kirill Tkhai wrote:
> >> This patchset adds a new syscall, which makes possible
> >> to clone a mapping from a process to another process.
> >> The syscall supplements the functionality provided
> >> by process_vm_writev() and process_vm_readv() syscalls,
> >> and it may be useful in many situation.
> >>
> >> For example, it allows to make a zero copy of data,
> >> when process_vm_writev() was previously used:
> > 
> > I wonder, why not optimize the existing interfaces to do zero copy if
> > properly aligned?  No need for a new syscall, and old code would immediately
> > benefit.
> 
> Because, this is just not possible. You can't zero copy anonymous pages
> of a process to pages of a remote process, when they are different pages.

fork() manages that, and so does KSM.  Like KSM, you want to make a page
shared -- you just skip the comparison step as you want to overwrite the old
contents.

And there's no need to touch the page, as fork() manages that fine no matter
if the page is resident, anonymous in swap, or file-backed, all without
reading from swap.

> >> There are several problems with process_vm_writev() in this example:
> >>
> >> 1)it causes pagefault on remote process memory, and it forces
> >>   allocation of a new page (if was not preallocated);
> >>
> >> 2)amount of memory for this example is doubled in a moment --
> >>   n pages in current and n pages in remote tasks are occupied
> >>   at the same time;
> >>
> >> 3)received data has no a chance to be properly swapped for
> >>   a long time.
> > 
> > That'll handle all of your above problems, except for making pages
> > subject to CoW if written to.  But if making pages writeably shared is
> > desired, the old functions have a "flags" argument that doesn't yet have a
> > single bit defined.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ Latin:   meow 4 characters, 4 columns,  4 bytes
⣾⠁⢠⠒⠀⣿⡁ Greek:   μεου 4 characters, 4 columns,  8 bytes
⢿⡄⠘⠷⠚⠋  Runes:   ᛗᛖᛟᚹ 4 characters, 4 columns, 12 bytes
⠈⠳⣄ Chinese: 喵   1 character,  2 columns,  3 bytes <-- best!


Re: [PATCH RFC 0/5] mm: process_vm_mmap() -- syscall for duplication a process mapping

2019-05-15 Thread Adam Borowski
On Wed, May 15, 2019 at 06:11:15PM +0300, Kirill Tkhai wrote:
> This patchset adds a new syscall, which makes possible
> to clone a mapping from a process to another process.
> The syscall supplements the functionality provided
> by process_vm_writev() and process_vm_readv() syscalls,
> and it may be useful in many situation.
> 
> For example, it allows to make a zero copy of data,
> when process_vm_writev() was previously used:

I wonder, why not optimize the existing interfaces to do zero copy if
properly aligned?  No need for a new syscall, and old code would immediately
benefit.

> There are several problems with process_vm_writev() in this example:
> 
> 1)it causes pagefault on remote process memory, and it forces
>   allocation of a new page (if was not preallocated);
> 
> 2)amount of memory for this example is doubled in a moment --
>   n pages in current and n pages in remote tasks are occupied
>   at the same time;
> 
> 3)received data has no a chance to be properly swapped for
>   a long time.

That'll handle all of your above problems, except for making pages
subject to CoW if written to.  But if making pages writeably shared is
desired, the old functions have a "flags" argument that doesn't yet have a
single bit defined.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ Latin:   meow 4 characters, 4 columns,  4 bytes
⣾⠁⢠⠒⠀⣿⡁ Greek:   μεου 4 characters, 4 columns,  8 bytes
⢿⡄⠘⠷⠚⠋  Runes:   ᛗᛖᛟᚹ 4 characters, 4 columns, 12 bytes
⠈⠳⣄ Chinese: 喵   1 character,  2 columns,  3 bytes <-- best!


Re: [PATCH RESEND v5 0/5] namei: vfs flags to restrict path resolution

2019-04-25 Thread Adam Borowski
On Thu, Apr 25, 2019 at 01:38:06AM +1000, Aleksa Sarai wrote:
>  * openat(2) ignores unknown flags, meaning that old kernels will ignore
>new programs trying to use O_THISROOT and might end up causing
>security issues. Yes, it'd be trivial to check whether the new O_*
>flags are supported at start-up, but I think a security feature
>shouldn't have a foot-gun associated with it. In fact, I didn't know
>openat(2) ignored unknown flags until I wrote this patchset -- I
>doubt many other userspace developers do either.

For this reason, I propose every new syscall that has flags to follow a
bitmask scheme, where any flag assigned a bit in the upper half returns
EOPNOTSUPP when called on an old kernel.  That would allow defining which
flags can be safely ignored and which can't.

It otherwise takes major hacks to implement a fail-if-not-supported flag
while keeping compat with old kernels.  For example, for mmap(), MAP_SHARED
has been duplicated as MAP_SHARED_VALIDATE just to allow an unrelated flag
(MAP_SYNC) to fail on old kernels.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢰⠒⠀⣿⡁ Imagine there are bandits in your house, your kid is bleeding out,
⢿⡄⠘⠷⠚⠋⠀ the house is on fire, and seven giant trumpets are playing in the
⠈⠳⣄ sky.  Your cat demands food.  The priority should be obvious...


Re: How to turn off IPv4 without disabling IPv6

2019-04-25 Thread Adam Borowski
On Thu, Apr 25, 2019 at 11:32:38AM +0200, Nico Schottelius wrote:
> running some IPv6 only
> networks. The systems in the IPv6 only networks do not need any IPv4
> support anymore and thus for switches/routers we turned the support off.

> Today we tried to turn off IPv4 in the Linux kernel at compile time.
> But it seems that as soon as we turn off CONFIG_INET, CONFIG_IPV6 is
> automatically turned off as well.

Even if you don't want global nor even link-scope IPv4, way too many
programs assume that at least 127.0.0.1 (ie, lo) is working.  They can't be
reconfigured to use ::1 without patching and rebuilding.

> Coming back to my original question: is there a way or how would we turn
> off IPv4 support in the Linux kernel?

I believe this is not worth your time for today.

Just do what IPv6-haters do on stock modern distros: have no routes for the
other IP version configured; non-buggy programs will do the right thing.

This seems to work well.  Heck, I had a busy dev server with broken IPv4, I
didn't notice that for 1.5 years until I tried to pull something directly
from Github (which is still v4 only).

You're a network admin so you know far more than me wrt anything that goes
over the wire -- but as as a distro developer/user, I'd say there's a
considerable cost to have every of tens of thousands programs shipped by a
distro, and many more that are private to a company/university/etc, updated
to autodetect how to access "localhost" on a particular box.

That's an extra moving part where there was none before.  Complexity is bad.
Having the IPv4 stack built just for the lo interface simplifies things.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Did ya know that typing "test -j8" instead of "ctest -j8"
⢿⡄⠘⠷⠚⠋⠀ will make your testsuite pass much faster, and fix bugs?
⠈⠳⣄


Re: Staging status of speakup

2019-03-19 Thread Adam Borowski
On Tue, Mar 19, 2019 at 04:31:21PM +, Alan Cox wrote:
> On Sat, 16 Mar 2019 10:35:43 +0100
> Samuel Thibault  wrote:
> > Chris Brannon, le ven. 15 mars 2019 18:19:39 -0700, a ecrit:
> > > What kind of reproducer do you need here?  It's straightforward to
> > > reproduce in casual use, at least with a software synthesizer.  
> > 
> > we need a walk-through of the kind of operation that
> > produces the issue. It does not have to be reproducible each time it is
> > done. Perhaps (I really don't know what that bug is about actually) it
> > is a matter of putting text in the selection buffer, and try to paste it
> > 100 times, and once every 10 times it will be garbled, for instance.
> 
> paste_selection still says
> 
> /* Insert the contents of the selection buffer into the
>  * queue of the tty associated with the current console.
>  * Invoked by ioctl().
>  *
>  * Locking: called without locks. Calls the ldisc wrongly with
>  * unsafe methods,
>  */
> 
> from which I deduce that with everyone using X nobody ever bothered to
> fix it. So before you look too hard at the speakup code you might want to
> review the interaction with selection.c too.

This looks like https://bugs.debian.org/849474 which causes a lockup, and
for which Bill Allombert wrote a nice reproducer.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Did ya know that typing "test -j8" instead of "ctest -j8"
⢿⡄⠘⠷⠚⠋⠀ will make your testsuite pass much faster, and fix bugs?
⠈⠳⣄


Re: [PATCH] selftests/ftrace: Make the coloring POSIX compliant

2019-02-20 Thread Adam Borowski
On Wed, Feb 20, 2019 at 09:20:20PM +0100, Juerg Haefliger wrote:
> On Wed, 20 Feb 2019 14:49:34 -0500
> Steven Rostedt  wrote:
> 
> > On Wed, 20 Feb 2019 17:13:33 +0100
> > Juerg Haefliger  wrote:
> > 
> > > echo -e and \e are not POSIX. Depending on what /bin/sh is, we can get
> > > incorrect output like:  
> > 
> > I'm curious to which shell this is.
> 
> Quite frankly I don't know but that's the output we get when we run it in
> Jenkins. I'll try to find out.

The only shell that did not support \e was dash -- I fixed it myself on
2017-01-24; thus I expect any Ubuntus prior to Zesty to require \033.  This
means your Jenkins likely runs Xenial.

\e has been supported since ages in at least: bash zsh mksh sash posh ksh
busybox:sh; also in perl python ruby lua php, gcc clang tcc -- MSVC being
the only other exception I know about.

Indeed POSIX doesn't specify \e, but as it pretends there are charsets other
than ASCII and EBCDIC, it can't.  There's no escape in its "portable
character set".


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁
⢿⡄⠘⠷⠚⠋⠀ Have you accepted Khorne as your lord and saviour?
⠈⠳⣄


Re: GRSec is vital to Linux security

2019-01-24 Thread Adam Borowski
On Thu, Jan 24, 2019 at 04:31:10PM +0100, Enrico Weigelt, metux IT consult 
wrote:
> On 23.01.19 21:46, Ivan Ivanov wrote:
> 
> > Linux really needs to stop adding new features and
> > refactor itself to a smaller and more secure codebase before going
> > forward. Maybe 1 year break would be nice.
> 
> Do you have some actual proposals / patches ?

Enrico, you're responding to a notorious troll.  If you haven't noticed,
this "Ivan Ivanov" sock puppet is a persona of some bastard who talks to
him/herself while tarnishing the name of our dear friend MikeeUSA (a true
pillar of the community!).  His/her methods evolve, but the gist is the
same.  Expect bringing up a bogus but semi-plausible controversy in order
to start as big a thread as possible, then once people who this bastard
wants to attack have joined, try to equate their position in the public view
with statements such as:

(Excuse the quotation, please wipe your monitor afterwards.)

# But from a man?
#
# Well, goes to show you. White men ain't men. Best they are is 40 year
# old bois. Faggots to say for short in American parlance.
#
# Same reason they won't hold it down when a bunch of fucking cunts CoC
# them. You build the whole edifice, then you let a bunch of do-nothing
# white women rule over the thing you built and you.

And this has been going for quite a while.

Connecting to systemd threads doesn't seem to work any longer, as people on
debian-user vs dng have wisened up.  Same with license rescinsion threads. 
What you read is just a yet another attempt to stir up some excrement.
Don't let any of it spray on you.  Because that's the fake-Mikee's way.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Remember, the S in "IoT" stands for Security, while P stands
⢿⡄⠘⠷⠚⠋⠀ for Privacy.
⠈⠳⣄


[PATCH] doc: process: GPL -> GPL-compatible

2019-01-22 Thread Adam Borowski
Drivers under MIT, BSD-17-clause, or uncle-Bob's-newest-take-on-PD are
all fine, not just GPL.

Signed-off-by: Adam Borowski 
---
Not reformatting to fill lines, it'll semi-conflict with another patch
that's been acked but not yet pushed.

 Documentation/process/stable-api-nonsense.rst | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Documentation/process/stable-api-nonsense.rst 
b/Documentation/process/stable-api-nonsense.rst
index 24f5aeecee91..9b69fb08b65e 100644
--- a/Documentation/process/stable-api-nonsense.rst
+++ b/Documentation/process/stable-api-nonsense.rst
@@ -170,7 +170,8 @@ nightmare, and trying to keep up with an ever changing 
kernel interface
 is also a rough job.
 
 Simple, get your kernel driver into the main kernel tree (remember we
-are talking about GPL released drivers here, if your code doesn't fall
+are talking about drivers released under a GPL-compatible license here,
+if your code doesn't fall
 under this category, good luck, you are on your own here, you leech
 .)  If your
 driver is in the tree, and a kernel interface changes, it will be fixed
-- 
2.20.1



Re: [PATCH] mm/mmu_notifier: mm/rmap.c: Fix a mmu_notifier range bug in try_to_unmap_one

2019-01-10 Thread Adam Borowski
On Wed, Jan 09, 2019 at 04:51:17PM -0800, Sean Christopherson wrote:
> Manifests as KVM use-after-free WARNINGs and subsequent "BUG: Bad page
> state in process X" errors when reclaiming from a KVM guest due to KVM
> removing the wrong pages from its own mappings.

With your patch, no badness happened so far.  Thanks!

> Reported-by: Adam Borowski 
> Fixes: ac46d4f3c432 ("mm/mmu_notifier: use structure for 
> invalidate_range_start/end calls v2")

> --- a/mm/rmap.c
> +++ b/mm/rmap.c
> - mmu_notifier_range_init(, vma->vm_mm, vma->vm_start,
> - min(vma->vm_end, vma->vm_start +
> + mmu_notifier_range_init(, vma->vm_mm, address,
> + min(vma->vm_end, address +


Meow.
-- 
⢀⣴⠾⠻⢶⣦⠀ Hans 1 was born and raised in Johannesburg, then moved to Boston,
⣾⠁⢠⠒⠀⣿⡁ and has just became a naturalized citizen.  Hans 2's grandparents
⢿⡄⠘⠷⠚⠋⠀ came from Melanesia to Düsseldorf, and he hasn't ever been outside
⠈⠳⣄ Germany until yesterday.  Which one is an African-American?


Re: 5.0-rc1 KVM inspired "BUG: Bad page state in process" spew

2019-01-09 Thread Adam Borowski
On Wed, Jan 09, 2019 at 06:38:58AM +0100, Mike Galbraith wrote:
> KVM seems to be busted in master ATM.  All I have to do to have host
> start screaming and maybe exploding (if the guest doesn't do so first)
> is to try to install a (obese in this case) kernel over nfs mount of
> the host in a guest.
> 
> Kernel producing the spew below is 3bd6e94, config attached.

I get same, except that the BUGs were preceded by a bunch of warnings,

> homer: # grep BUG: /netconsole.log
> [ 1531.909703] BUG: Bad page state in process X  pfn:100491
> [ 1531.958141] BUG: Bad page state in process systemd-journal  pfn:100412
> [ 1532.662359] BUG: Bad page state in process X  pfn:10043f
> [ 1532.664033] BUG: Bad page state in process X  pfn:10044d
> [ 1532.686433] BUG: Bad page state in process systemd-journal  pfn:1027b0

the first one being:

Jan  9 00:41:22 umbar kernel: [74122.790461] WARNING: CPU: 2 PID: 26769 at 
arch/x86/kvm/mmu.c:830 mmu_spte_clear_track_bits+0x7e/0x100
Jan  9 00:41:22 umbar kernel: [74122.799676] Modules linked in: tun dm_mod 
cdc_acm rndis_wlan rndis_host cdc_ether usbnet sha256_generic cfg80211 rfkill 
mii nfnetlink snd_usb_audio snd_usbmidi_lib cp210x usbserial radeon ttm 
pcc_cpufreq
Jan  9 00:41:22 umbar kernel: [74122.817764] CPU: 2 PID: 26769 Comm: 
qemu-system-x86 Not tainted 5.0.0-rc1-debug-00035-gdce22716f1b4 #1
Jan  9 00:41:22 umbar kernel: [74122.827069] Hardware name: System manufacturer 
System Product Name/M4A77T, BIOS 240105/18/2011
Jan  9 00:41:22 umbar kernel: [74122.836025] RIP: 
0010:mmu_spte_clear_track_bits+0x7e/0x100
Jan  9 00:41:22 umbar kernel: [74122.841528] Code: 48 89 e8 48 ba 00 00 00 00 
00 ea ff ff 48 c1 e0 06 48 01 d0 48 8b 50 08 48 8d 4a ff 83 e2 01 48 0f 45 c1 
8b 40 34 85 c0 75 02 <0f> 0b 48 0f ba e3 3e 73 34 48 85 1d d2 32 26 01 75 5a 48 
d1 eb 83
Jan  9 00:41:22 umbar kernel: [74122.860290] RSP: 0018:c900028634d0 EFLAGS: 
00010246
Jan  9 00:41:22 umbar kernel: [74122.865516] RAX:  RBX: 
00010198bc67 RCX: dead00ff
Jan  9 00:41:22 umbar kernel: [74122.872649] RDX:  RSI: 
 RDI: ea00040662c0
Jan  9 00:41:22 umbar kernel: [74122.879790] RBP: 0010198b R08: 
28f5c28f5c28f5c3 R09: 8882264dd000
Jan  9 00:41:22 umbar kernel: [74122.886912] R10: 88822fffa000 R11: 
88822fffa000 R12: 
Jan  9 00:41:22 umbar kernel: [74122.894046] R13: c90002863580 R14: 
 R15: c90002863580
Jan  9 00:41:22 umbar kernel: [74122.901170] FS:  7f19b9953700() 
GS:888227a8() knlGS:
Jan  9 00:41:22 umbar kernel: [74122.909249] CS:  0010 DS:  ES:  CR0: 
80050033
Jan  9 00:41:22 umbar kernel: [74122.914994] CR2: 102630e4 CR3: 
0001c80ee000 CR4: 06e0
Jan  9 00:41:22 umbar kernel: [74122.922135] Call Trace:
Jan  9 00:41:22 umbar kernel: [74122.924590]  drop_spte+0x1b/0xc0
Jan  9 00:41:22 umbar kernel: [74122.927822]  mmu_page_zap_pte+0xb2/0xe0
Jan  9 00:41:22 umbar kernel: [74122.931661]  
kvm_mmu_prepare_zap_page+0x4f/0x2b0
Jan  9 00:41:22 umbar kernel: [74122.936297]  mmu_shrink_scan+0x199/0x240
Jan  9 00:41:22 umbar kernel: [74122.940223]  do_shrink_slab+0x11e/0x1a0
Jan  9 00:41:22 umbar kernel: [74122.944054]  shrink_slab+0x220/0x2b0
Jan  9 00:41:22 umbar kernel: [74122.947632]  shrink_node+0x168/0x460
Jan  9 00:41:22 umbar kernel: [74122.951203]  do_try_to_free_pages+0xc1/0x370
Jan  9 00:41:22 umbar kernel: [74122.955467]  try_to_free_pages+0xb0/0xe0
Jan  9 00:41:22 umbar kernel: [74122.959385]  __alloc_pages_slowpath+0x37d/0xb60
Jan  9 00:41:22 umbar kernel: [74122.963917]  __alloc_pages_nodemask+0x255/0x270
Jan  9 00:41:22 umbar kernel: [74122.968441]  
do_huge_pmd_anonymous_page+0x13c/0x5d0
Jan  9 00:41:22 umbar kernel: [74122.973322]  __handle_mm_fault+0x984/0xfb0
Jan  9 00:41:22 umbar kernel: [74122.977438]  handle_mm_fault+0xc2/0x200
Jan  9 00:41:22 umbar kernel: [74122.981278]  __get_user_pages+0x258/0x6c0
Jan  9 00:41:22 umbar kernel: [74122.985290]  
get_user_pages_unlocked+0x153/0x1d0
Jan  9 00:41:22 umbar kernel: [74122.989954]  __gfn_to_pfn_memslot+0x149/0x410
Jan  9 00:41:22 umbar kernel: [74122.994320]  try_async_pf+0x96/0x1b0
Jan  9 00:41:22 umbar kernel: [74122.997900]  ? kvm_host_page_size+0x81/0xa0
Jan  9 00:41:22 umbar kernel: [74123.002093]  tdp_page_fault+0x168/0x2b0
Jan  9 00:41:22 umbar kernel: [74123.005927]  ? svm_vcpu_run+0x294/0x680
Jan  9 00:41:22 umbar kernel: [74123.009765]  kvm_mmu_page_fault+0x89/0x3f0
Jan  9 00:41:22 umbar kernel: [74123.013865]  ? kvm_set_cr8.part.87+0xa/0x30
Jan  9 00:41:22 umbar kernel: [74123.018049]  ? svm_vcpu_run+0x4fd/0x680
Jan  9 00:41:22 umbar kernel: [74123.021888]  ? kvm_fast_pio+0x140/0x190
Jan  9 00:41:22 umbar kernel: [74123.025729]  
kvm_arch_vcpu_ioctl_run+0x59e/0x19c0
Jan  9 00:41:22 umbar kernel: [74123.030435]  ? _copy_to_user+0x26/0x30
Jan  9 00:41:22 umbar kernel: [74123.034187]  ? kvm_vm_ioctl+0x69a/0x950
Jan  9 00:41:22 

Re: Can we drop upstream Linux x32 support?

2018-12-13 Thread Adam Borowski
On Thu, Dec 13, 2018 at 10:37:31AM +0100, Sven Hartrumpf wrote:
> Will the proposed patch ("only") remove the possibility to build x32 kernels
> or will it make impossible to compile and run any x32 binaries?

There's no such thing as x32 kernels.  It's an ABI atop amd64 kernels; the
kernel is always 64-bit -- short pointers apply only to userspace and
syscalls.

It could be interesting to block regular amd64 syscalls and leave x32 only
-- for "security through rarity", but the kernel itself would still look the
same.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢠⠒⠀⣿⡁ Ivan was a worldly man: born in St. Petersburg, raised in
⢿⡄⠘⠷⠚⠋⠀ Petrograd, lived most of his life in Leningrad, then returned
⠈⠳⣄ to the city of his birth to die.


Re: Official Linux system wrapper library?

2018-11-14 Thread Adam Borowski
On Sun, Nov 11, 2018 at 12:46:35PM +0100, Florian Weimer wrote:
> A lot of multi-threaded applications assume that most high-level
> functionality remains usable even after fork in a multi-threaded
> process.

How would this be even possible?  Currently fork kills all threads
(save for the caller).

Glibc's manpage also warns:

# After a fork() in a multithreaded program, the child can safely call only
# async-signal-safe functions (see signal-safety(7)) until such time as it
# calls execve(2).

Which makes sense as its malloc uses a mutex, and you can't take a breath
without a library call using malloc somewhere (or in C++, the language
itself).

So any functionality remaining usable after fork is pretty strictly
limited...


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ I've read an article about how lively happy music boosts
⣾⠁⢰⠒⠀⣿⡁ productivity. You can read it, too, you just need the
⢿⡄⠘⠷⠚⠋⠀ right music while doing so. I recommend Skepticism
⠈⠳⣄ (funeral doom metal).


Re: Official Linux system wrapper library?

2018-11-14 Thread Adam Borowski
On Sun, Nov 11, 2018 at 12:46:35PM +0100, Florian Weimer wrote:
> A lot of multi-threaded applications assume that most high-level
> functionality remains usable even after fork in a multi-threaded
> process.

How would this be even possible?  Currently fork kills all threads
(save for the caller).

Glibc's manpage also warns:

# After a fork() in a multithreaded program, the child can safely call only
# async-signal-safe functions (see signal-safety(7)) until such time as it
# calls execve(2).

Which makes sense as its malloc uses a mutex, and you can't take a breath
without a library call using malloc somewhere (or in C++, the language
itself).

So any functionality remaining usable after fork is pretty strictly
limited...


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ I've read an article about how lively happy music boosts
⣾⠁⢰⠒⠀⣿⡁ productivity. You can read it, too, you just need the
⢿⡄⠘⠷⠚⠋⠀ right music while doing so. I recommend Skepticism
⠈⠳⣄ (funeral doom metal).


[PATCH 01/17] lib: Add support for ZSTD-compressed kernel

2018-11-09 Thread Adam Borowski
From: Nick Terrell 

Add support for extracting ZSTD-compressed kernel images, as well as
ZSTD-compressed ramdisk images in the kernel boot process.

When neither `fill' nor `flush' are used, the decompression function
requires a constant amount of memory (192 KB is sufficient). When either
is used the decompression function requires memory proportional to the
window size used during compression, which is limited to 8 MB. The maximum
memory usage is just over 8 MB.

Fix up lib/zstd and lib/xxhash.c for the preboot environment. They avoid
declaring themselves as modules. A memcpy() call needs to be a
__builtin_memcpy() for performance. The gcc-7.1 bug in ZSTD_wildcopy() was
fixed in gcc-7.2, so it can be gated, since it hurts performance.

Signed-off-by: Nick Terrell 
---
 include/linux/decompress/unzstd.h |  26 +++
 init/Kconfig  |  14 +-
 lib/Kconfig   |   4 +
 lib/Makefile  |   1 +
 lib/decompress.c  |   5 +
 lib/decompress_unzstd.c   | 341 ++
 lib/xxhash.c  |  21 +-
 lib/zstd/decompress.c |   2 +
 lib/zstd/fse_decompress.c |   9 +-
 lib/zstd/zstd_internal.h  |  10 +-
 scripts/Makefile.lib  |  15 ++
 usr/Kconfig   |  22 ++
 12 files changed, 451 insertions(+), 19 deletions(-)
 create mode 100644 include/linux/decompress/unzstd.h
 create mode 100644 lib/decompress_unzstd.c

diff --git a/include/linux/decompress/unzstd.h 
b/include/linux/decompress/unzstd.h
new file mode 100644
index ..6f3022cd0955
--- /dev/null
+++ b/include/linux/decompress/unzstd.h
@@ -0,0 +1,26 @@
+/*
+ * Copyright (C) 2017 Facebook
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see .
+ */
+
+#ifndef LINUX_DECOMPRESS_UNZSTD_H
+#define LINUX_DECOMPRESS_UNZSTD_H
+
+int unzstd(unsigned char *inbuf, long len,
+  long (*fill)(void*, unsigned long),
+  long (*flush)(void*, unsigned long),
+  unsigned char *output,
+  long *pos,
+  void (*error_fn)(char *x));
+#endif
diff --git a/init/Kconfig b/init/Kconfig
index a4112e95724a..ffa5ae4abc88 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -134,13 +134,16 @@ config HAVE_KERNEL_LZO
 config HAVE_KERNEL_LZ4
bool
 
+config HAVE_KERNEL_ZSTD
+   bool
+
 config HAVE_KERNEL_UNCOMPRESSED
bool
 
 choice
prompt "Kernel compression mode"
default KERNEL_GZIP
-   depends on HAVE_KERNEL_GZIP || HAVE_KERNEL_BZIP2 || HAVE_KERNEL_LZMA || 
HAVE_KERNEL_XZ || HAVE_KERNEL_LZO || HAVE_KERNEL_LZ4 || HAVE_KERNEL_UNCOMPRESSED
+   depends on HAVE_KERNEL_GZIP || HAVE_KERNEL_BZIP2 || HAVE_KERNEL_LZMA || 
HAVE_KERNEL_XZ || HAVE_KERNEL_LZO || HAVE_KERNEL_LZ4 || HAVE_KERNEL_ZSTD || 
HAVE_KERNEL_UNCOMPRESSED
help
  The linux kernel is a kind of self-extracting executable.
  Several compression algorithms are available, which differ
@@ -219,6 +222,15 @@ config KERNEL_LZ4
  is about 8% bigger than LZO. But the decompression speed is
  faster than LZO.
 
+config KERNEL_ZSTD
+   bool "ZSTD"
+   depends on HAVE_KERNEL_ZSTD
+   help
+ ZSTD is a compression algorithm targeting intermediate compression
+ with fast decompression speed. It will compress better than GZIP and
+ decompress around the same speed as LZO, but slower than LZ4. You
+ will need at least 192 KB RAM or more for booting.
+
 config KERNEL_UNCOMPRESSED
bool "None"
depends on HAVE_KERNEL_UNCOMPRESSED
diff --git a/lib/Kconfig b/lib/Kconfig
index a9965f4af4dd..e7ab43fd5461 100644
--- a/lib/Kconfig
+++ b/lib/Kconfig
@@ -304,6 +304,10 @@ config DECOMPRESS_LZ4
select LZ4_DECOMPRESS
tristate
 
+config DECOMPRESS_ZSTD
+   select ZSTD_DECOMPRESS
+   tristate
+
 #
 # Generic allocator support is selected if needed
 #
diff --git a/lib/Makefile b/lib/Makefile
index db06d1237898..58b48993f48a 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -139,6 +139,7 @@ lib-$(CONFIG_DECOMPRESS_LZMA) += decompress_unlzma.o
 lib-$(CONFIG_DECOMPRESS_XZ) += decompress_unxz.o
 lib-$(CONFIG_DECOMPRESS_LZO) += decompress_unlzo.o
 lib-$(CONFIG_DECOMPRESS_LZ4) += decompress_unlz4.o
+lib-$(CONFIG_DECOMPRESS_ZSTD) += decompress_unzstd.o
 
 obj-$(CONFIG_TEXTSEARCH) += textsearch.o
 obj-$(CONFIG_TEXTSEARCH_KMP) += ts_kmp.o
diff --git a/lib/decompress.c b/lib/decompress.c

[PATCH 16/17] Kconfig: Update the prose for selection of compression algorithm

2018-11-09 Thread Adam Borowski
It was really obsolete, and some entries contradicted each other.

Let's not recommend ZSTD for kernel compression yet as it's available
only on x86, and some distros might not have the tool installed.
Proposing ZSTD for initrd is safer but let's test it first.

Signed-off-by: Adam Borowski 
---
 init/Kconfig | 25 +
 usr/Kconfig  | 24 +++-
 2 files changed, 24 insertions(+), 25 deletions(-)

diff --git a/init/Kconfig b/init/Kconfig
index 412ba93673fa..7c0180c41a3c 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -153,26 +153,27 @@ choice
  version of this functionality (bzip2 only), for 2.4, was
  supplied by Christian Ludwig)
 
- High compression options are mostly useful for users, who
- are low on disk space (embedded systems), but for whom ram
- size matters less.
+ High compression options tend to be more useful in most cases,
+ as bootloaders are often egregiously slow to read the kernel
+ from the disk/SD card/network/etc, overcoming any boot time
+ savings you would get from faster decompression.
 
- If in doubt, select 'gzip'
+ If in doubt, select 'xz'
 
 config KERNEL_GZIP
bool "Gzip"
depends on HAVE_KERNEL_GZIP
help
- The old and tried gzip compression. It provides a good balance
- between compression ratio and decompression speed.
+ The old and tried gzip compression. You generally want it if
+ some tool you use doesn't support more modern compressors.
 
 config KERNEL_LZMA
bool "LZMA"
depends on HAVE_KERNEL_LZMA
help
- This compression algorithm's ratio is best.  Decompression speed
- is between gzip and bzip2.  Compression is slowest.
- The kernel size is about 33% smaller with LZMA in comparison to gzip.
+ An old version of xz, like it providing strong compression at slow
+ speed. It lacks a header and support for filters or uncompressed
+ blocks, thus it's usually better to pick xz.
 
 config KERNEL_XZ
bool "XZ"
@@ -193,9 +194,9 @@ config KERNEL_LZO
bool "LZO"
depends on HAVE_KERNEL_LZO
help
- Its compression ratio is the poorest among the choices. The kernel
- size is about 10% bigger than gzip; however its speed
- (both compression and decompression) is the fastest.
+ Its compression ratio is pretty poor (but still better than
+ LZ4). You want to pick ZSTD (similar speed but much better
+ compression) or LZ4 (much better speed) instead.
 
 config KERNEL_LZ4
bool "LZ4"
diff --git a/usr/Kconfig b/usr/Kconfig
index 8d99edacabc9..f6e871585f05 100644
--- a/usr/Kconfig
+++ b/usr/Kconfig
@@ -131,17 +131,15 @@ config INITRAMFS_COMPRESSION_NONE
  on those architectures that support this. However, not compressing the
  initramfs may lead to slightly higher memory consumption during a
  short time at boot, while both the cpio image and the unpacked
- filesystem image will be present in memory simultaneously
+ filesystem image will be present in memory simultaneously.
 
 config INITRAMFS_COMPRESSION_GZIP
bool "Gzip"
depends on RD_GZIP
help
- Use the old and well tested gzip compression algorithm. Gzip provides
- a good balance between compression ratio and decompression speed and
- has a reasonable compression speed. It is also more likely to be
- supported by your build system as the gzip tool is present by default
- on most distros.
+ Use the old and well tested gzip compression algorithm. Gzip doesn't
+ provide very good compression nor speed, but it's the safest choice,
+ with wide support.
 
 config INITRAMFS_COMPRESSION_XZ
bool "XZ"
@@ -150,20 +148,20 @@ config INITRAMFS_COMPRESSION_XZ
  XZ uses the LZMA2 algorithm and has a large dictionary which may cause
  problems on memory constrained systems. The initramfs size is about
  30% smaller with XZ in comparison to gzip. Decompression speed is
- better than that of bzip2 but worse than gzip and LZO. Compression is
- slow.
+ okayish but still slowest of all currently available algorithms.
+ Compression is slow.
 
- If you choose this, keep in mind that you may need to install the xz
- tool to be able to compress the initram.
+ Any modern distro provides xz in its default install, but on
+ minimal build systems you might need to install xz-utils to be
+ able to compress the initram.
 
 config INITRAMFS_COMPRESSION_LZO
bool "LZO"
depends on RD_LZO
help
  It's compression ratio is the second poorest amongst the choices. The
- kernel size is about 10% bigger t

[PATCH 01/17] lib: Add support for ZSTD-compressed kernel

2018-11-09 Thread Adam Borowski
From: Nick Terrell 

Add support for extracting ZSTD-compressed kernel images, as well as
ZSTD-compressed ramdisk images in the kernel boot process.

When neither `fill' nor `flush' are used, the decompression function
requires a constant amount of memory (192 KB is sufficient). When either
is used the decompression function requires memory proportional to the
window size used during compression, which is limited to 8 MB. The maximum
memory usage is just over 8 MB.

Fix up lib/zstd and lib/xxhash.c for the preboot environment. They avoid
declaring themselves as modules. A memcpy() call needs to be a
__builtin_memcpy() for performance. The gcc-7.1 bug in ZSTD_wildcopy() was
fixed in gcc-7.2, so it can be gated, since it hurts performance.

Signed-off-by: Nick Terrell 
---
 include/linux/decompress/unzstd.h |  26 +++
 init/Kconfig  |  14 +-
 lib/Kconfig   |   4 +
 lib/Makefile  |   1 +
 lib/decompress.c  |   5 +
 lib/decompress_unzstd.c   | 341 ++
 lib/xxhash.c  |  21 +-
 lib/zstd/decompress.c |   2 +
 lib/zstd/fse_decompress.c |   9 +-
 lib/zstd/zstd_internal.h  |  10 +-
 scripts/Makefile.lib  |  15 ++
 usr/Kconfig   |  22 ++
 12 files changed, 451 insertions(+), 19 deletions(-)
 create mode 100644 include/linux/decompress/unzstd.h
 create mode 100644 lib/decompress_unzstd.c

diff --git a/include/linux/decompress/unzstd.h 
b/include/linux/decompress/unzstd.h
new file mode 100644
index ..6f3022cd0955
--- /dev/null
+++ b/include/linux/decompress/unzstd.h
@@ -0,0 +1,26 @@
+/*
+ * Copyright (C) 2017 Facebook
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see .
+ */
+
+#ifndef LINUX_DECOMPRESS_UNZSTD_H
+#define LINUX_DECOMPRESS_UNZSTD_H
+
+int unzstd(unsigned char *inbuf, long len,
+  long (*fill)(void*, unsigned long),
+  long (*flush)(void*, unsigned long),
+  unsigned char *output,
+  long *pos,
+  void (*error_fn)(char *x));
+#endif
diff --git a/init/Kconfig b/init/Kconfig
index a4112e95724a..ffa5ae4abc88 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -134,13 +134,16 @@ config HAVE_KERNEL_LZO
 config HAVE_KERNEL_LZ4
bool
 
+config HAVE_KERNEL_ZSTD
+   bool
+
 config HAVE_KERNEL_UNCOMPRESSED
bool
 
 choice
prompt "Kernel compression mode"
default KERNEL_GZIP
-   depends on HAVE_KERNEL_GZIP || HAVE_KERNEL_BZIP2 || HAVE_KERNEL_LZMA || 
HAVE_KERNEL_XZ || HAVE_KERNEL_LZO || HAVE_KERNEL_LZ4 || HAVE_KERNEL_UNCOMPRESSED
+   depends on HAVE_KERNEL_GZIP || HAVE_KERNEL_BZIP2 || HAVE_KERNEL_LZMA || 
HAVE_KERNEL_XZ || HAVE_KERNEL_LZO || HAVE_KERNEL_LZ4 || HAVE_KERNEL_ZSTD || 
HAVE_KERNEL_UNCOMPRESSED
help
  The linux kernel is a kind of self-extracting executable.
  Several compression algorithms are available, which differ
@@ -219,6 +222,15 @@ config KERNEL_LZ4
  is about 8% bigger than LZO. But the decompression speed is
  faster than LZO.
 
+config KERNEL_ZSTD
+   bool "ZSTD"
+   depends on HAVE_KERNEL_ZSTD
+   help
+ ZSTD is a compression algorithm targeting intermediate compression
+ with fast decompression speed. It will compress better than GZIP and
+ decompress around the same speed as LZO, but slower than LZ4. You
+ will need at least 192 KB RAM or more for booting.
+
 config KERNEL_UNCOMPRESSED
bool "None"
depends on HAVE_KERNEL_UNCOMPRESSED
diff --git a/lib/Kconfig b/lib/Kconfig
index a9965f4af4dd..e7ab43fd5461 100644
--- a/lib/Kconfig
+++ b/lib/Kconfig
@@ -304,6 +304,10 @@ config DECOMPRESS_LZ4
select LZ4_DECOMPRESS
tristate
 
+config DECOMPRESS_ZSTD
+   select ZSTD_DECOMPRESS
+   tristate
+
 #
 # Generic allocator support is selected if needed
 #
diff --git a/lib/Makefile b/lib/Makefile
index db06d1237898..58b48993f48a 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -139,6 +139,7 @@ lib-$(CONFIG_DECOMPRESS_LZMA) += decompress_unlzma.o
 lib-$(CONFIG_DECOMPRESS_XZ) += decompress_unxz.o
 lib-$(CONFIG_DECOMPRESS_LZO) += decompress_unlzo.o
 lib-$(CONFIG_DECOMPRESS_LZ4) += decompress_unlz4.o
+lib-$(CONFIG_DECOMPRESS_ZSTD) += decompress_unzstd.o
 
 obj-$(CONFIG_TEXTSEARCH) += textsearch.o
 obj-$(CONFIG_TEXTSEARCH_KMP) += ts_kmp.o
diff --git a/lib/decompress.c b/lib/decompress.c

[PATCH 16/17] Kconfig: Update the prose for selection of compression algorithm

2018-11-09 Thread Adam Borowski
It was really obsolete, and some entries contradicted each other.

Let's not recommend ZSTD for kernel compression yet as it's available
only on x86, and some distros might not have the tool installed.
Proposing ZSTD for initrd is safer but let's test it first.

Signed-off-by: Adam Borowski 
---
 init/Kconfig | 25 +
 usr/Kconfig  | 24 +++-
 2 files changed, 24 insertions(+), 25 deletions(-)

diff --git a/init/Kconfig b/init/Kconfig
index 412ba93673fa..7c0180c41a3c 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -153,26 +153,27 @@ choice
  version of this functionality (bzip2 only), for 2.4, was
  supplied by Christian Ludwig)
 
- High compression options are mostly useful for users, who
- are low on disk space (embedded systems), but for whom ram
- size matters less.
+ High compression options tend to be more useful in most cases,
+ as bootloaders are often egregiously slow to read the kernel
+ from the disk/SD card/network/etc, overcoming any boot time
+ savings you would get from faster decompression.
 
- If in doubt, select 'gzip'
+ If in doubt, select 'xz'
 
 config KERNEL_GZIP
bool "Gzip"
depends on HAVE_KERNEL_GZIP
help
- The old and tried gzip compression. It provides a good balance
- between compression ratio and decompression speed.
+ The old and tried gzip compression. You generally want it if
+ some tool you use doesn't support more modern compressors.
 
 config KERNEL_LZMA
bool "LZMA"
depends on HAVE_KERNEL_LZMA
help
- This compression algorithm's ratio is best.  Decompression speed
- is between gzip and bzip2.  Compression is slowest.
- The kernel size is about 33% smaller with LZMA in comparison to gzip.
+ An old version of xz, like it providing strong compression at slow
+ speed. It lacks a header and support for filters or uncompressed
+ blocks, thus it's usually better to pick xz.
 
 config KERNEL_XZ
bool "XZ"
@@ -193,9 +194,9 @@ config KERNEL_LZO
bool "LZO"
depends on HAVE_KERNEL_LZO
help
- Its compression ratio is the poorest among the choices. The kernel
- size is about 10% bigger than gzip; however its speed
- (both compression and decompression) is the fastest.
+ Its compression ratio is pretty poor (but still better than
+ LZ4). You want to pick ZSTD (similar speed but much better
+ compression) or LZ4 (much better speed) instead.
 
 config KERNEL_LZ4
bool "LZ4"
diff --git a/usr/Kconfig b/usr/Kconfig
index 8d99edacabc9..f6e871585f05 100644
--- a/usr/Kconfig
+++ b/usr/Kconfig
@@ -131,17 +131,15 @@ config INITRAMFS_COMPRESSION_NONE
  on those architectures that support this. However, not compressing the
  initramfs may lead to slightly higher memory consumption during a
  short time at boot, while both the cpio image and the unpacked
- filesystem image will be present in memory simultaneously
+ filesystem image will be present in memory simultaneously.
 
 config INITRAMFS_COMPRESSION_GZIP
bool "Gzip"
depends on RD_GZIP
help
- Use the old and well tested gzip compression algorithm. Gzip provides
- a good balance between compression ratio and decompression speed and
- has a reasonable compression speed. It is also more likely to be
- supported by your build system as the gzip tool is present by default
- on most distros.
+ Use the old and well tested gzip compression algorithm. Gzip doesn't
+ provide very good compression nor speed, but it's the safest choice,
+ with wide support.
 
 config INITRAMFS_COMPRESSION_XZ
bool "XZ"
@@ -150,20 +148,20 @@ config INITRAMFS_COMPRESSION_XZ
  XZ uses the LZMA2 algorithm and has a large dictionary which may cause
  problems on memory constrained systems. The initramfs size is about
  30% smaller with XZ in comparison to gzip. Decompression speed is
- better than that of bzip2 but worse than gzip and LZO. Compression is
- slow.
+ okayish but still slowest of all currently available algorithms.
+ Compression is slow.
 
- If you choose this, keep in mind that you may need to install the xz
- tool to be able to compress the initram.
+ Any modern distro provides xz in its default install, but on
+ minimal build systems you might need to install xz-utils to be
+ able to compress the initram.
 
 config INITRAMFS_COMPRESSION_LZO
bool "LZO"
depends on RD_LZO
help
  It's compression ratio is the second poorest amongst the choices. The
- kernel size is about 10% bigger t

drivers by default (was Re: Another HID problem this merge window..)

2018-10-28 Thread Adam Borowski
On Sat, Oct 27, 2018 at 11:13:17AM -0700, Linus Torvalds wrote:
> Ok, so this is a much smaller issue than the i2c one that cause boot
> problems, but it's annoying.
> 
> We do *not* enable new random drivers by default. And we most
> *definitely* don't do it when they are odd-ball ones that most people
> have never heard of.
> 
> Yet the new "BigBen Interactive" driver that was added this merge
> window did exactly that.
> 
> Just don't do it.

Amen to that.  But, perhaps you could encourage people to do enable drivers
once they become very popular?  For example, I just (72a9c673636) got hit by
USB 3.0 being off in defconfigs, and not having keyboard is not that cool.

So what about a policy that says "almost all drivers are to be =n initially,
but patches enabling popular stuff by default are welcome"?  Preferably if
obsolete crap gets mass-disabled at least once per decade (think of the
floppy driver's importance on day one vs now).

> Yes, yes, every developer always thinks that _their_ driver is so
> special and so magically important that it should be enabled by
> default. But no. When we have thousands of drivers, we don't randomly
> pick one new driver to be enabled by default just because some
> developer thinks it is special. It's not.

We don't know which drivers will become important and which won't, only time
can tell.  But not having such drivers on by default is also a problem.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢰⠒⠀⣿⡁ 10 people enter a bar: 1 who understands binary,
⢿⡄⠘⠷⠚⠋⠀ 1 who doesn't, D who prefer to write it as hex,
⠈⠳⣄ and 1 who narrowly avoided an off-by-one error.


drivers by default (was Re: Another HID problem this merge window..)

2018-10-28 Thread Adam Borowski
On Sat, Oct 27, 2018 at 11:13:17AM -0700, Linus Torvalds wrote:
> Ok, so this is a much smaller issue than the i2c one that cause boot
> problems, but it's annoying.
> 
> We do *not* enable new random drivers by default. And we most
> *definitely* don't do it when they are odd-ball ones that most people
> have never heard of.
> 
> Yet the new "BigBen Interactive" driver that was added this merge
> window did exactly that.
> 
> Just don't do it.

Amen to that.  But, perhaps you could encourage people to do enable drivers
once they become very popular?  For example, I just (72a9c673636) got hit by
USB 3.0 being off in defconfigs, and not having keyboard is not that cool.

So what about a policy that says "almost all drivers are to be =n initially,
but patches enabling popular stuff by default are welcome"?  Preferably if
obsolete crap gets mass-disabled at least once per decade (think of the
floppy driver's importance on day one vs now).

> Yes, yes, every developer always thinks that _their_ driver is so
> special and so magically important that it should be enabled by
> default. But no. When we have thousands of drivers, we don't randomly
> pick one new driver to be enabled by default just because some
> developer thinks it is special. It's not.

We don't know which drivers will become important and which won't, only time
can tell.  But not having such drivers on by default is also a problem.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢰⠒⠀⣿⡁ 10 people enter a bar: 1 who understands binary,
⢿⡄⠘⠷⠚⠋⠀ 1 who doesn't, D who prefer to write it as hex,
⠈⠳⣄ and 1 who narrowly avoided an off-by-one error.


[tip:x86/build] x86/defconfig: Enable CONFIG_USB_XHCI_HCD=y

2018-10-10 Thread tip-bot for Adam Borowski
Commit-ID:  72a9c673636b779e370983fea08e40f97039b981
Gitweb: https://git.kernel.org/tip/72a9c673636b779e370983fea08e40f97039b981
Author: Adam Borowski 
AuthorDate: Tue, 9 Oct 2018 08:28:03 +0200
Committer:  Ingo Molnar 
CommitDate: Wed, 10 Oct 2018 08:29:51 +0200

x86/defconfig: Enable CONFIG_USB_XHCI_HCD=y

A spanking new machine I just got has all but one USB ports wired as 3.0.
Booting defconfig resulted in no keyboard or mouse, which was pretty
uncool.  Let's enable that -- USB3 is ubiquitous rather than an oddity.
As 'y' not 'm' -- recovering from initrd problems needs a keyboard.

Also add it to the 32-bit defconfig.

Signed-off-by: Adam Borowski 
Cc: Greg Kroah-Hartman 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: linux-...@vger.kernel.org
Link: http://lkml.kernel.org/r/20181009062803.4332-1-kilob...@angband.pl
Signed-off-by: Ingo Molnar 
---
 arch/x86/configs/i386_defconfig   | 1 +
 arch/x86/configs/x86_64_defconfig | 1 +
 2 files changed, 2 insertions(+)

diff --git a/arch/x86/configs/i386_defconfig b/arch/x86/configs/i386_defconfig
index 0eb9f92f3717..6c3ab05c231d 100644
--- a/arch/x86/configs/i386_defconfig
+++ b/arch/x86/configs/i386_defconfig
@@ -247,6 +247,7 @@ CONFIG_USB_HIDDEV=y
 CONFIG_USB=y
 CONFIG_USB_ANNOUNCE_NEW_DEVICES=y
 CONFIG_USB_MON=y
+CONFIG_USB_XHCI_HCD=y
 CONFIG_USB_EHCI_HCD=y
 CONFIG_USB_EHCI_TT_NEWSCHED=y
 CONFIG_USB_OHCI_HCD=y
diff --git a/arch/x86/configs/x86_64_defconfig 
b/arch/x86/configs/x86_64_defconfig
index e32fc1f274d8..ac9ae487cfeb 100644
--- a/arch/x86/configs/x86_64_defconfig
+++ b/arch/x86/configs/x86_64_defconfig
@@ -243,6 +243,7 @@ CONFIG_USB_HIDDEV=y
 CONFIG_USB=y
 CONFIG_USB_ANNOUNCE_NEW_DEVICES=y
 CONFIG_USB_MON=y
+CONFIG_USB_XHCI_HCD=y
 CONFIG_USB_EHCI_HCD=y
 CONFIG_USB_EHCI_TT_NEWSCHED=y
 CONFIG_USB_OHCI_HCD=y


[tip:x86/build] x86/defconfig: Enable CONFIG_USB_XHCI_HCD=y

2018-10-10 Thread tip-bot for Adam Borowski
Commit-ID:  72a9c673636b779e370983fea08e40f97039b981
Gitweb: https://git.kernel.org/tip/72a9c673636b779e370983fea08e40f97039b981
Author: Adam Borowski 
AuthorDate: Tue, 9 Oct 2018 08:28:03 +0200
Committer:  Ingo Molnar 
CommitDate: Wed, 10 Oct 2018 08:29:51 +0200

x86/defconfig: Enable CONFIG_USB_XHCI_HCD=y

A spanking new machine I just got has all but one USB ports wired as 3.0.
Booting defconfig resulted in no keyboard or mouse, which was pretty
uncool.  Let's enable that -- USB3 is ubiquitous rather than an oddity.
As 'y' not 'm' -- recovering from initrd problems needs a keyboard.

Also add it to the 32-bit defconfig.

Signed-off-by: Adam Borowski 
Cc: Greg Kroah-Hartman 
Cc: Linus Torvalds 
Cc: Peter Zijlstra 
Cc: Thomas Gleixner 
Cc: linux-...@vger.kernel.org
Link: http://lkml.kernel.org/r/20181009062803.4332-1-kilob...@angband.pl
Signed-off-by: Ingo Molnar 
---
 arch/x86/configs/i386_defconfig   | 1 +
 arch/x86/configs/x86_64_defconfig | 1 +
 2 files changed, 2 insertions(+)

diff --git a/arch/x86/configs/i386_defconfig b/arch/x86/configs/i386_defconfig
index 0eb9f92f3717..6c3ab05c231d 100644
--- a/arch/x86/configs/i386_defconfig
+++ b/arch/x86/configs/i386_defconfig
@@ -247,6 +247,7 @@ CONFIG_USB_HIDDEV=y
 CONFIG_USB=y
 CONFIG_USB_ANNOUNCE_NEW_DEVICES=y
 CONFIG_USB_MON=y
+CONFIG_USB_XHCI_HCD=y
 CONFIG_USB_EHCI_HCD=y
 CONFIG_USB_EHCI_TT_NEWSCHED=y
 CONFIG_USB_OHCI_HCD=y
diff --git a/arch/x86/configs/x86_64_defconfig 
b/arch/x86/configs/x86_64_defconfig
index e32fc1f274d8..ac9ae487cfeb 100644
--- a/arch/x86/configs/x86_64_defconfig
+++ b/arch/x86/configs/x86_64_defconfig
@@ -243,6 +243,7 @@ CONFIG_USB_HIDDEV=y
 CONFIG_USB=y
 CONFIG_USB_ANNOUNCE_NEW_DEVICES=y
 CONFIG_USB_MON=y
+CONFIG_USB_XHCI_HCD=y
 CONFIG_USB_EHCI_HCD=y
 CONFIG_USB_EHCI_TT_NEWSCHED=y
 CONFIG_USB_OHCI_HCD=y


Re: Contributors can not just rescind the license (Legal Opinion)

2018-09-28 Thread Adam Borowski
On Fri, Sep 28, 2018 at 11:31:32AM +0200, www.Advocati.org wrote:
> CONTRIBUTORS CAN NOT JUST RESCIND THE LICENSE
> THEY GRANTED UNDER GPLv2. IT IS *COPYLEFTED*

> A letter from 17.09.2018 by @observerofaffairs
> titled 'GPL version 2 is a bare license. Recind. (Regarding (future)
> Code of Conduct Bannings)'
> https://lkml.org/lkml/2018/9/17/1174,

> repeated by another letter from 20.09.2018 by @unconditionedwitness
> titled 'Re: A Plea to Unfuck our Codes of Conduct'
> https://lulz.com/linux-devs-threaten-killswitch-coc-controversy-1252/

You're responding to trolling by our dear MikeeUSA, whom most of us long
since learned to recognize and ignore.  There are better uses for your time
than responding to a notorious cartooney.

He's as nasty a troll as CoralineAda, and about as harmful to the community.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 10 people enter a bar:
⣾⠁⢰⠒⠀⣿⡁ • 1 who understands binary,
⢿⡄⠘⠷⠚⠋⠀ • 1 who doesn't,
⠈⠳⣄ • and E who prefer to write it as hex.


Re: Contributors can not just rescind the license (Legal Opinion)

2018-09-28 Thread Adam Borowski
On Fri, Sep 28, 2018 at 11:31:32AM +0200, www.Advocati.org wrote:
> CONTRIBUTORS CAN NOT JUST RESCIND THE LICENSE
> THEY GRANTED UNDER GPLv2. IT IS *COPYLEFTED*

> A letter from 17.09.2018 by @observerofaffairs
> titled 'GPL version 2 is a bare license. Recind. (Regarding (future)
> Code of Conduct Bannings)'
> https://lkml.org/lkml/2018/9/17/1174,

> repeated by another letter from 20.09.2018 by @unconditionedwitness
> titled 'Re: A Plea to Unfuck our Codes of Conduct'
> https://lulz.com/linux-devs-threaten-killswitch-coc-controversy-1252/

You're responding to trolling by our dear MikeeUSA, whom most of us long
since learned to recognize and ignore.  There are better uses for your time
than responding to a notorious cartooney.

He's as nasty a troll as CoralineAda, and about as harmful to the community.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 10 people enter a bar:
⣾⠁⢰⠒⠀⣿⡁ • 1 who understands binary,
⢿⡄⠘⠷⠚⠋⠀ • 1 who doesn't,
⠈⠳⣄ • and E who prefer to write it as hex.


Re: POSIX violation by writeback error

2018-09-25 Thread Adam Borowski
On Tue, Sep 25, 2018 at 11:46:27AM -0400, Theodore Y. Ts'o wrote:
> P.S.  One thought: it might be cool if there was some way for
> userspace applications to mark files with "nuke if not closed" flag,
> such that if the system crashes, the file systems would automatically
> unlink the file after a reboot or if the process was killed or exits
> without an explicit close(2).  For networked/remote file systems that
> supported this flag, after the client comes back up after a reboot, it
> could notify the server that all files created previously from that
> client should be unlinked.
> 
> Unlike O_TMPFILE, this would require file system changes to support,
> so maybe it's not worth having something which automatically cleans up
> files that were in the middle of being written at the time of a system
> crash.

Isn't this what the snippet for O_TMPFILE in "man 2 open" does?:

  char path[PATH_MAX];
  fd = open("/path/to/dir", O_TMPFILE | O_RDWR,
  S_IRUSR | S_IWUSR);

  /* File I/O on 'fd'... */

  snprintf(path, PATH_MAX,  "/proc/self/fd/%d", fd);
  linkat(AT_FDCWD, path, AT_FDCWD, "/path/for/file",
  AT_SYMLINK_FOLLOW);


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 10 people enter a bar:
⣾⠁⢰⠒⠀⣿⡁ • 1 who understands binary,
⢿⡄⠘⠷⠚⠋⠀ • 1 who doesn't,
⠈⠳⣄ • and E who prefer to write it as hex.


Re: POSIX violation by writeback error

2018-09-25 Thread Adam Borowski
On Tue, Sep 25, 2018 at 11:46:27AM -0400, Theodore Y. Ts'o wrote:
> P.S.  One thought: it might be cool if there was some way for
> userspace applications to mark files with "nuke if not closed" flag,
> such that if the system crashes, the file systems would automatically
> unlink the file after a reboot or if the process was killed or exits
> without an explicit close(2).  For networked/remote file systems that
> supported this flag, after the client comes back up after a reboot, it
> could notify the server that all files created previously from that
> client should be unlinked.
> 
> Unlike O_TMPFILE, this would require file system changes to support,
> so maybe it's not worth having something which automatically cleans up
> files that were in the middle of being written at the time of a system
> crash.

Isn't this what the snippet for O_TMPFILE in "man 2 open" does?:

  char path[PATH_MAX];
  fd = open("/path/to/dir", O_TMPFILE | O_RDWR,
  S_IRUSR | S_IWUSR);

  /* File I/O on 'fd'... */

  snprintf(path, PATH_MAX,  "/proc/self/fd/%d", fd);
  linkat(AT_FDCWD, path, AT_FDCWD, "/path/for/file",
  AT_SYMLINK_FOLLOW);


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 10 people enter a bar:
⣾⠁⢰⠒⠀⣿⡁ • 1 who understands binary,
⢿⡄⠘⠷⠚⠋⠀ • 1 who doesn't,
⠈⠳⣄ • and E who prefer to write it as hex.


Re: [...] an apology, and a maintainership note

2018-09-16 Thread Adam Borowski
On Sun, Sep 16, 2018 at 12:22:43PM -0700, Linus Torvalds wrote:
> This is my reality.  I am not an emotionally empathetic kind of person
> and that probably doesn't come as a big surprise to anybody.  Least of
> all me.  The fact that I then misread people and don't realize (for
> years) how badly I've judged a situation and contributed to an
> unprofessional environment is not good.
> 
> This week people in our community confronted me about my lifetime of
> not understanding emotions.  My flippant attacks in emails have been
> both unprofessional and uncalled for.  Especially at times when I made
> it personal.  In my quest for a better patch, this made sense to me.
> I know now this was not OK and I am truly sorry.
> 
> The above is basically a long-winded way to get to the somewhat
> painful personal admission that hey, I need to change some of my
> behavior, and I want to apologize to the people that my personal
> behavior hurt and possibly drove away from kernel development
> entirely.

Despite me being just among bottom-rung popcorn of kernel contributors, let
me says this:

No.  Just no.  You're so successful because you're one of few people who
don't waste time beating around the bush.  You call a spade a spade instead
of polite "professional" bullshit.

You often use rude words, but you don't do so without a reason.  IMO your
most striking quality is not technical ability (pretty high...) but the
ratio of times you open your mouth to the times you're right.  And even
if you're not right, you don't take offense at getting corrected and
immediately admit someone else was right.  

Sure, there are cases when both choices are right, but your approach avoids
wasting time making a decision.  For example: recently, you forced disabling
string truncation warnings despite many people feeling otherwise.  I for one
believe GCC's warnings even though sounding bogus are good for eliminating
strncpy -- what I would have done would be giving it an aliased version
named "fixedfieldncpy" or such that disables the warning, and fixing the
whole rest.  But what you did instead deprioritizes the issue: the kernel
doesn't work any worse than it did with gcc-7, thus there are indeed more
urgent matters elsewhere.  So even if I don't fully agree with you, you are
the boss and as long as your version is acceptable, let's stick to it.

And, it's _you_ who has proven merit, not me.

> I am going to take time off and get some assistance on how to
> understand people’s emotions and respond appropriately.
> 
> Put another way: When asked at conferences, I occasionally talk about
> how the pain-points in kernel development have generally not been
> about the _technical_ issues, but about the inflection points where
> development flow and behavior changed.

Too many projects get detracted by prolonged crap about social things, don't
let this pull you down.  There's a problem when people _without merit_ are
rude -- those indeed need to get a spanking.  A spanking not ADHD meds.
Short and to the point, letting them learn.

But you, you _earned_ the right to be rude to get your point across.
I watched a video about you getting shamed on a DebConf because of breaching
some "code of conduct" by using a naughty word.  I didn't like that and
believe it was you who was right (I don't recall the details though).

> I've talked to Greg to ask him if he'd mind finishing up 4.19 for me, so
> that I can take a break, and try to at least fix my own behavior.
> 
> This is not some kind of "I'm burnt out, I need to just go away"
> break.  I'm not feeling like I don't want to continue maintaining
> Linux. Quite the reverse.  I very much *do* want to continue to do
> this project that I've been working on for almost three decades.
> 
> This is more like the time I got out of kernel development for a while
> because I needed to write a little tool called "git".  I need to take
> a break to get help on how to behave differently and fix some issues
> in my tooling and workflow.

You do deserve a vacation.  By all means, do take a break and let the
community rehearse for "Linus got mauled by a bear".  But we want you back.

> And yes, some of it might be "just" tooling.  Maybe I can get an email
> filter in place so at when I send email with curse-words, they just
> won't go out.  Because hey, I'm a big believer in tools, and at least
> _some_ problems going forward might be improved with simple
> automation.

Please don't.  When you use curse words, they're _warranted_.  They're a
tool which, in my opinion, you don't overuse.

And it's fun to listen to a true master of words.  An example: how many
pages would https://lkml.org/lkml/2016/8/2/2062 take to say politely yet get
the same effect?

> I know when I really look “myself in the mirror” it will be clear it's
> not the only change that has to happen, but hey...  You can send me
> suggestions in email.

When you look yourself in the mirror, I want you to see that guy who codes
in a bathrobe 

Re: [...] an apology, and a maintainership note

2018-09-16 Thread Adam Borowski
On Sun, Sep 16, 2018 at 12:22:43PM -0700, Linus Torvalds wrote:
> This is my reality.  I am not an emotionally empathetic kind of person
> and that probably doesn't come as a big surprise to anybody.  Least of
> all me.  The fact that I then misread people and don't realize (for
> years) how badly I've judged a situation and contributed to an
> unprofessional environment is not good.
> 
> This week people in our community confronted me about my lifetime of
> not understanding emotions.  My flippant attacks in emails have been
> both unprofessional and uncalled for.  Especially at times when I made
> it personal.  In my quest for a better patch, this made sense to me.
> I know now this was not OK and I am truly sorry.
> 
> The above is basically a long-winded way to get to the somewhat
> painful personal admission that hey, I need to change some of my
> behavior, and I want to apologize to the people that my personal
> behavior hurt and possibly drove away from kernel development
> entirely.

Despite me being just among bottom-rung popcorn of kernel contributors, let
me says this:

No.  Just no.  You're so successful because you're one of few people who
don't waste time beating around the bush.  You call a spade a spade instead
of polite "professional" bullshit.

You often use rude words, but you don't do so without a reason.  IMO your
most striking quality is not technical ability (pretty high...) but the
ratio of times you open your mouth to the times you're right.  And even
if you're not right, you don't take offense at getting corrected and
immediately admit someone else was right.  

Sure, there are cases when both choices are right, but your approach avoids
wasting time making a decision.  For example: recently, you forced disabling
string truncation warnings despite many people feeling otherwise.  I for one
believe GCC's warnings even though sounding bogus are good for eliminating
strncpy -- what I would have done would be giving it an aliased version
named "fixedfieldncpy" or such that disables the warning, and fixing the
whole rest.  But what you did instead deprioritizes the issue: the kernel
doesn't work any worse than it did with gcc-7, thus there are indeed more
urgent matters elsewhere.  So even if I don't fully agree with you, you are
the boss and as long as your version is acceptable, let's stick to it.

And, it's _you_ who has proven merit, not me.

> I am going to take time off and get some assistance on how to
> understand people’s emotions and respond appropriately.
> 
> Put another way: When asked at conferences, I occasionally talk about
> how the pain-points in kernel development have generally not been
> about the _technical_ issues, but about the inflection points where
> development flow and behavior changed.

Too many projects get detracted by prolonged crap about social things, don't
let this pull you down.  There's a problem when people _without merit_ are
rude -- those indeed need to get a spanking.  A spanking not ADHD meds.
Short and to the point, letting them learn.

But you, you _earned_ the right to be rude to get your point across.
I watched a video about you getting shamed on a DebConf because of breaching
some "code of conduct" by using a naughty word.  I didn't like that and
believe it was you who was right (I don't recall the details though).

> I've talked to Greg to ask him if he'd mind finishing up 4.19 for me, so
> that I can take a break, and try to at least fix my own behavior.
> 
> This is not some kind of "I'm burnt out, I need to just go away"
> break.  I'm not feeling like I don't want to continue maintaining
> Linux. Quite the reverse.  I very much *do* want to continue to do
> this project that I've been working on for almost three decades.
> 
> This is more like the time I got out of kernel development for a while
> because I needed to write a little tool called "git".  I need to take
> a break to get help on how to behave differently and fix some issues
> in my tooling and workflow.

You do deserve a vacation.  By all means, do take a break and let the
community rehearse for "Linus got mauled by a bear".  But we want you back.

> And yes, some of it might be "just" tooling.  Maybe I can get an email
> filter in place so at when I send email with curse-words, they just
> won't go out.  Because hey, I'm a big believer in tools, and at least
> _some_ problems going forward might be improved with simple
> automation.

Please don't.  When you use curse words, they're _warranted_.  They're a
tool which, in my opinion, you don't overuse.

And it's fun to listen to a true master of words.  An example: how many
pages would https://lkml.org/lkml/2016/8/2/2062 take to say politely yet get
the same effect?

> I know when I really look “myself in the mirror” it will be clear it's
> not the only change that has to happen, but hey...  You can send me
> suggestions in email.

When you look yourself in the mirror, I want you to see that guy who codes
in a bathrobe 

Re: Kernel-only deployments?

2018-08-23 Thread Adam Borowski
On Thu, Aug 23, 2018 at 10:43:59AM -0700, Paul E. McKenney wrote:
> The mkinitramfs approach results in about 40MB of initrd, and dracut
> about 10MB.  Most of this is completely useless for rcutorture, which
> isn't interested in mounting filesystems, opening devices, and almost
> all of the other interesting things that mkinitramfs and dracut enable.
> 
> Those who know me will not be at all surprised to learn that I went
> overboard making the resulting initrd as small as possible.  I started
> by throwing out everything not absolutely needed by the dash and sleep
> binaries, which got me down to about 2.5MB, 1.8MB of which was libc.
> This situation of course prompted me to create an initrd containing
> a statically linked binary named "init" and absolutely nothing else
> (not even /dev or /tmp directories), which weighs in at not quite 800KB.
> This is a great improvement over 10MB, to say nothing of 40MB, but 800KB
> for a C-language "for" loop containing nothing more than a single call to
> sleep()?

.globl _start
.data
req:.8byte 9, 9
.text
_start:
mov $35, %rax   # syscall: nanosleep
mov $req, %rdi
xor %rsi, %rsi
syscall
jmp _start


as sl.s -o sl.o
ld sl.o -o init

'Ere you go, no libc needed.  If your arch is not amd64, just say so.

If you want to do anything more complex, though -- you really want musl
or another lightweight libc instead.  Glibc is utterly unfit for static
linking.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ .globl _start↵.data↵rc: .ascii "/etc/init.d/rcS\0"↵.text↵_start
⣾⠁⢰⠒⠀⣿⡁ mov $57,%rax↵syscall↵cmp $0,%rax↵jne child↵parent:↵mov $61,%rax
⢿⡄⠘⠷⠚⠋⠀ mov $-1,%rdi↵xor %rsi,%rsi↵xor %rdx,%rdx↵syscall↵jmp parent↵child:
⠈⠳⣄ mov $59,%rax↵mov $rc,%rdi↵xor %rsi,%rsi↵xor %rdx,%rdx↵syscall


Re: Kernel-only deployments?

2018-08-23 Thread Adam Borowski
On Thu, Aug 23, 2018 at 10:43:59AM -0700, Paul E. McKenney wrote:
> The mkinitramfs approach results in about 40MB of initrd, and dracut
> about 10MB.  Most of this is completely useless for rcutorture, which
> isn't interested in mounting filesystems, opening devices, and almost
> all of the other interesting things that mkinitramfs and dracut enable.
> 
> Those who know me will not be at all surprised to learn that I went
> overboard making the resulting initrd as small as possible.  I started
> by throwing out everything not absolutely needed by the dash and sleep
> binaries, which got me down to about 2.5MB, 1.8MB of which was libc.
> This situation of course prompted me to create an initrd containing
> a statically linked binary named "init" and absolutely nothing else
> (not even /dev or /tmp directories), which weighs in at not quite 800KB.
> This is a great improvement over 10MB, to say nothing of 40MB, but 800KB
> for a C-language "for" loop containing nothing more than a single call to
> sleep()?

.globl _start
.data
req:.8byte 9, 9
.text
_start:
mov $35, %rax   # syscall: nanosleep
mov $req, %rdi
xor %rsi, %rsi
syscall
jmp _start


as sl.s -o sl.o
ld sl.o -o init

'Ere you go, no libc needed.  If your arch is not amd64, just say so.

If you want to do anything more complex, though -- you really want musl
or another lightweight libc instead.  Glibc is utterly unfit for static
linking.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ .globl _start↵.data↵rc: .ascii "/etc/init.d/rcS\0"↵.text↵_start
⣾⠁⢰⠒⠀⣿⡁ mov $57,%rax↵syscall↵cmp $0,%rax↵jne child↵parent:↵mov $61,%rax
⢿⡄⠘⠷⠚⠋⠀ mov $-1,%rdi↵xor %rsi,%rsi↵xor %rdx,%rdx↵syscall↵jmp parent↵child:
⠈⠳⣄ mov $59,%rax↵mov $rc,%rdi↵xor %rsi,%rsi↵xor %rdx,%rdx↵syscall


Re: linux-next: build warnings from Linus' tree

2018-08-19 Thread Adam Borowski
On Sun, Aug 19, 2018 at 04:21:57PM -0700, Linus Torvalds wrote:
> On Sun, Aug 19, 2018 at 3:13 PM Stephen Rothwell  
> wrote:
> >
> > Today's linux-next build (powerpc ppc64_defconfig) produced these
> > warnings:
> >
> > fs/cifs/cifssmb.c:605:3: warning: 'strncpy' writing 16 bytes into a region 
> > of size 1 overflows the destination [-Wstringop-overflow=]
> >strncpy(pSMB->DialectsArray+count, protocols[i].name, 16);
> >
> > Presumably caused by my update to gcc 8.2.0.
> 
> Yeah. There are some patches to mark some arrays as non-strings to get
> rid of these, but we'll see. Maybe we'll just disable the new gcc
> warning if it causes more pain than it is worth.

Every single use of strncpy() for a C string is either a bug, inefficiency,
or both.  In this particular case the code:

count = 0;
for (i = 0; i < CIFS_NUM_PROT; i++) {
strncpy(pSMB->DialectsArray+count, protocols[i].name, 16);
count += strlen(protocols[i].name) + 1;
/* null at end of source and target buffers anyway */
}

* pointlessly clears 16 bytes in every iteration
* calculates the string's length twice
* there's no protection against buffer overflow anyway

So what is the strncpy() there for, when an unbounded copy would be just as
good?  For other cases, there's a bunch of better functions: strlcpy(),
snprintf(), even strlen()+memcpy(), etc.

Valid uses of strncpy() do exist (such as SCSI structs), but those deal with
fixed-width fields.  Thus, gcc is right for warning for at least some of
misuse of strncpy() for C strings.  The function wasn't designed for them.

(Skipped analysis why strncpy is always a bad choice for C strings.)


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ What Would Jesus Do, MUD/MMORPG edition:
⣾⠁⢰⠒⠀⣿⡁ • multiplay with an admin char to benefit your mortal [Mt3:16-17]
⢿⡄⠘⠷⠚⠋⠀ • abuse item cloning bugs [Mt14:17-20, Mt15:34-37]
⠈⠳⣄ • use glitches to walk on water [Mt14:25-26]


Re: linux-next: build warnings from Linus' tree

2018-08-19 Thread Adam Borowski
On Sun, Aug 19, 2018 at 04:21:57PM -0700, Linus Torvalds wrote:
> On Sun, Aug 19, 2018 at 3:13 PM Stephen Rothwell  
> wrote:
> >
> > Today's linux-next build (powerpc ppc64_defconfig) produced these
> > warnings:
> >
> > fs/cifs/cifssmb.c:605:3: warning: 'strncpy' writing 16 bytes into a region 
> > of size 1 overflows the destination [-Wstringop-overflow=]
> >strncpy(pSMB->DialectsArray+count, protocols[i].name, 16);
> >
> > Presumably caused by my update to gcc 8.2.0.
> 
> Yeah. There are some patches to mark some arrays as non-strings to get
> rid of these, but we'll see. Maybe we'll just disable the new gcc
> warning if it causes more pain than it is worth.

Every single use of strncpy() for a C string is either a bug, inefficiency,
or both.  In this particular case the code:

count = 0;
for (i = 0; i < CIFS_NUM_PROT; i++) {
strncpy(pSMB->DialectsArray+count, protocols[i].name, 16);
count += strlen(protocols[i].name) + 1;
/* null at end of source and target buffers anyway */
}

* pointlessly clears 16 bytes in every iteration
* calculates the string's length twice
* there's no protection against buffer overflow anyway

So what is the strncpy() there for, when an unbounded copy would be just as
good?  For other cases, there's a bunch of better functions: strlcpy(),
snprintf(), even strlen()+memcpy(), etc.

Valid uses of strncpy() do exist (such as SCSI structs), but those deal with
fixed-width fields.  Thus, gcc is right for warning for at least some of
misuse of strncpy() for C strings.  The function wasn't designed for them.

(Skipped analysis why strncpy is always a bad choice for C strings.)


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ What Would Jesus Do, MUD/MMORPG edition:
⣾⠁⢰⠒⠀⣿⡁ • multiplay with an admin char to benefit your mortal [Mt3:16-17]
⢿⡄⠘⠷⠚⠋⠀ • abuse item cloning bugs [Mt14:17-20, Mt15:34-37]
⠈⠳⣄ • use glitches to walk on water [Mt14:25-26]


Re: [RESEND PATCH v2 0/2] Add support for ZSTD-compressed kernel

2018-08-17 Thread Adam Borowski
On Fri, Aug 17, 2018 at 12:22:44PM -0700, Andi Kleen wrote:
> On Fri, Aug 17, 2018 at 07:57:46PM +0200, Adam Borowski wrote:
> > > The "favourite compressor" seems to roughly change every year, so if
> > > we keep adding new ones things will get more and more convoluted.
> > 
> > The above patchset drops just bzip2.  It is the only one that's strictly
> > beaten in every way (ratio, time, memory usage), there are also no other
> 
> Does time include build time? I've been reverting back to gzip recently
> because I care very much about that.

Too lazy to benchmark a kernel image (IIRC Nick Terrell posted that a while
ago), here's copypasta of a random 16824672 byte executable, in userspace,
with default level setting:

compdecomp  size
xz  8.038s  0.356s  4320292
bz2 2.265s  0.730s  5234516
zst 0.274s  0.102s  5657626
gz  0.880s  0.152s  6515505
Z   0.499s  0.133s  8932459
lzo 0.100s  0.095s  9198874

As you can see, zstd's compression time is drastically better than gzip,
while ratio is better.  The default level is very low (-3 on -1..-22 scale)
but you can crank it up for stronger compression.

The defaults fit your use case.

> > uses of bzip2 anywhere in the kernel so we'd get to drop its code
> > completely: 900 lines of Linus' happiness.
> 
> Great!
> 
> > Other candidates are lzo and bare lzma (you want lz4, zstd or xz instead),
> > but those are used elsewhere thus there's hardly any gain.  If you want them
> > gone, please say so -- I'll include their droppage.
> 
> Yes would be good to remove the kernel image support for those too
> just to simplify the config process, even if it doesn't save much code.

There's one caveat: fast choices are quite new:
* lz4 userspace tools are not even in current Debian stable (just unstable)
* uncompressed kernel got in only this merge window
* zstd has userspace tools in Debian stable but is not merged into the
  kernel yet
(other dists are probably similar)

Thus, it might be a good idea to keep lzo for a while longer.

Bare lzma can probably go -- xz filters are nice for binaries of archs it
knows (disabled otherwise), and lack of header requires hacks to find out
the payload's size.

So it's up to you guys: do you want me to drop lzo and/or lzma?
We can also drop them just for vmlinuz but not initrd.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ What Would Jesus Do, MUD/MMORPG edition:
⣾⠁⢰⠒⠀⣿⡁ • multiplay with an admin char to benefit your mortal [Mt3:16-17]
⢿⡄⠘⠷⠚⠋⠀ • abuse item cloning bugs [Mt14:17-20, Mt15:34-37]
⠈⠳⣄ • use glitches to walk on water [Mt14:25-26]


Re: [RESEND PATCH v2 0/2] Add support for ZSTD-compressed kernel

2018-08-17 Thread Adam Borowski
On Fri, Aug 17, 2018 at 12:22:44PM -0700, Andi Kleen wrote:
> On Fri, Aug 17, 2018 at 07:57:46PM +0200, Adam Borowski wrote:
> > > The "favourite compressor" seems to roughly change every year, so if
> > > we keep adding new ones things will get more and more convoluted.
> > 
> > The above patchset drops just bzip2.  It is the only one that's strictly
> > beaten in every way (ratio, time, memory usage), there are also no other
> 
> Does time include build time? I've been reverting back to gzip recently
> because I care very much about that.

Too lazy to benchmark a kernel image (IIRC Nick Terrell posted that a while
ago), here's copypasta of a random 16824672 byte executable, in userspace,
with default level setting:

compdecomp  size
xz  8.038s  0.356s  4320292
bz2 2.265s  0.730s  5234516
zst 0.274s  0.102s  5657626
gz  0.880s  0.152s  6515505
Z   0.499s  0.133s  8932459
lzo 0.100s  0.095s  9198874

As you can see, zstd's compression time is drastically better than gzip,
while ratio is better.  The default level is very low (-3 on -1..-22 scale)
but you can crank it up for stronger compression.

The defaults fit your use case.

> > uses of bzip2 anywhere in the kernel so we'd get to drop its code
> > completely: 900 lines of Linus' happiness.
> 
> Great!
> 
> > Other candidates are lzo and bare lzma (you want lz4, zstd or xz instead),
> > but those are used elsewhere thus there's hardly any gain.  If you want them
> > gone, please say so -- I'll include their droppage.
> 
> Yes would be good to remove the kernel image support for those too
> just to simplify the config process, even if it doesn't save much code.

There's one caveat: fast choices are quite new:
* lz4 userspace tools are not even in current Debian stable (just unstable)
* uncompressed kernel got in only this merge window
* zstd has userspace tools in Debian stable but is not merged into the
  kernel yet
(other dists are probably similar)

Thus, it might be a good idea to keep lzo for a while longer.

Bare lzma can probably go -- xz filters are nice for binaries of archs it
knows (disabled otherwise), and lack of header requires hacks to find out
the payload's size.

So it's up to you guys: do you want me to drop lzo and/or lzma?
We can also drop them just for vmlinuz but not initrd.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ What Would Jesus Do, MUD/MMORPG edition:
⣾⠁⢰⠒⠀⣿⡁ • multiplay with an admin char to benefit your mortal [Mt3:16-17]
⢿⡄⠘⠷⠚⠋⠀ • abuse item cloning bugs [Mt14:17-20, Mt15:34-37]
⠈⠳⣄ • use glitches to walk on water [Mt14:25-26]


Re: [RESEND PATCH v2 0/2] Add support for ZSTD-compressed kernel

2018-08-17 Thread Adam Borowski
On Fri, Aug 17, 2018 at 09:54:03AM -0700, Andi Kleen wrote:
> On Fri, Aug 17, 2018 at 06:15:45PM +0200, René Rebe wrote:
> > Hey,
> > 
> > is there any mainline future for this zstd support?
> > Currently my most favourite compressor for this, and for what it’s worth
> > zstd/initrd now even tested on #t2sde / hp/pa-risc
> 
> Can you remove some other compressor for the kernel image if you merge this?

Funny you say this.  I want to post this after the merge window is over:
https://github.com/kilobyte/linux/commits/nobz2-next-20180731

Sorry for a tree atop next, but there are conflicts that needed to be
adressed.  The patchset is also undertested (works for me, but not even
compile-tested on archs I don't have a toolchain for at hand).

> The "favourite compressor" seems to roughly change every year, so if
> we keep adding new ones things will get more and more convoluted.

The above patchset drops just bzip2.  It is the only one that's strictly
beaten in every way (ratio, time, memory usage), there are also no other
uses of bzip2 anywhere in the kernel so we'd get to drop its code
completely: 900 lines of Linus' happiness.

Other candidates are lzo and bare lzma (you want lz4, zstd or xz instead),
but those are used elsewhere thus there's hardly any gain.  If you want them
gone, please say so -- I'll include their droppage.

> Surely zstd is universally better than some existing compressor
> the kernel already uses for itself for a particular sweet spot?
> If it isn't it's likely not needed.

zstd is better in the balanced niche: unless you want extreme speed (lz4) or
best compression (xz), zstd handily beats algorithms in the middle.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ What Would Jesus Do, MUD/MMORPG edition:
⣾⠁⢰⠒⠀⣿⡁ • multiplay with an admin char to benefit your mortal [Mt3:16-17]
⢿⡄⠘⠷⠚⠋⠀ • abuse item cloning bugs [Mt14:17-20, Mt15:34-37]
⠈⠳⣄ • use glitches to walk on water [Mt14:25-26]


Re: [RESEND PATCH v2 0/2] Add support for ZSTD-compressed kernel

2018-08-17 Thread Adam Borowski
On Fri, Aug 17, 2018 at 09:54:03AM -0700, Andi Kleen wrote:
> On Fri, Aug 17, 2018 at 06:15:45PM +0200, René Rebe wrote:
> > Hey,
> > 
> > is there any mainline future for this zstd support?
> > Currently my most favourite compressor for this, and for what it’s worth
> > zstd/initrd now even tested on #t2sde / hp/pa-risc
> 
> Can you remove some other compressor for the kernel image if you merge this?

Funny you say this.  I want to post this after the merge window is over:
https://github.com/kilobyte/linux/commits/nobz2-next-20180731

Sorry for a tree atop next, but there are conflicts that needed to be
adressed.  The patchset is also undertested (works for me, but not even
compile-tested on archs I don't have a toolchain for at hand).

> The "favourite compressor" seems to roughly change every year, so if
> we keep adding new ones things will get more and more convoluted.

The above patchset drops just bzip2.  It is the only one that's strictly
beaten in every way (ratio, time, memory usage), there are also no other
uses of bzip2 anywhere in the kernel so we'd get to drop its code
completely: 900 lines of Linus' happiness.

Other candidates are lzo and bare lzma (you want lz4, zstd or xz instead),
but those are used elsewhere thus there's hardly any gain.  If you want them
gone, please say so -- I'll include their droppage.

> Surely zstd is universally better than some existing compressor
> the kernel already uses for itself for a particular sweet spot?
> If it isn't it's likely not needed.

zstd is better in the balanced niche: unless you want extreme speed (lz4) or
best compression (xz), zstd handily beats algorithms in the middle.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ What Would Jesus Do, MUD/MMORPG edition:
⣾⠁⢰⠒⠀⣿⡁ • multiplay with an admin char to benefit your mortal [Mt3:16-17]
⢿⡄⠘⠷⠚⠋⠀ • abuse item cloning bugs [Mt14:17-20, Mt15:34-37]
⠈⠳⣄ • use glitches to walk on water [Mt14:25-26]


Re: Linux 4.18.1

2018-08-16 Thread Adam Borowski
On Thu, Aug 16, 2018 at 05:11:21PM +0200, Greg KH wrote:
> On Thu, Aug 16, 2018 at 03:59:58PM +0200, Sven Joachim wrote:
> > On 2018-08-16 15:05 +0200, Adam Borowski wrote:
> > > I'm afraid that I get a build failure; v4.18 is ok, v4.18.1 fails with:
> > >
> > > ld: arch/x86/kvm/x86.o: in function `kvm_get_arch_capabilities':
> > > (.text+0x43b2): undefined reference to `l1tf_vmx_mitigation'
> > 
> > Same here and also in 4.17.15, but not in Linus' tree.
> > 
> > > CONFIG_KVM=y
> > > # CONFIG_KVM_INTEL is not set
> > > CONFIG_KVM_AMD=y
> > 
> > I have CONFIG_KVM{,AMD}=m and CONFIG_KVM_INTEL is not set either.
> 
> This is fixed in my queue, if it really bothers you, apply commit
> 1eb46908b35d ("x86/l1tf: Fix build error seen if CONFIG_KVM_INTEL is
> disabled") to your tree, it will be in the next round of stable kernel
> releases.

Probably obvious, but: cherry-picking that commit fixes the problem, it
builds, boots and works ok.

Thanks!


Meow.
-- 
⢀⣴⠾⠻⢶⣦⠀ What Would Jesus Do, MUD/MMORPG edition:
⣾⠁⢰⠒⠀⣿⡁ • multiplay with an admin char to benefit your mortal [Mt3:16-17]
⢿⡄⠘⠷⠚⠋⠀ • abuse item cloning bugs [Mt14:17-20, Mt15:34-37]
⠈⠳⣄ • use glitches to walk on water [Mt14:25-26]


Re: Linux 4.18.1

2018-08-16 Thread Adam Borowski
On Thu, Aug 16, 2018 at 05:11:21PM +0200, Greg KH wrote:
> On Thu, Aug 16, 2018 at 03:59:58PM +0200, Sven Joachim wrote:
> > On 2018-08-16 15:05 +0200, Adam Borowski wrote:
> > > I'm afraid that I get a build failure; v4.18 is ok, v4.18.1 fails with:
> > >
> > > ld: arch/x86/kvm/x86.o: in function `kvm_get_arch_capabilities':
> > > (.text+0x43b2): undefined reference to `l1tf_vmx_mitigation'
> > 
> > Same here and also in 4.17.15, but not in Linus' tree.
> > 
> > > CONFIG_KVM=y
> > > # CONFIG_KVM_INTEL is not set
> > > CONFIG_KVM_AMD=y
> > 
> > I have CONFIG_KVM{,AMD}=m and CONFIG_KVM_INTEL is not set either.
> 
> This is fixed in my queue, if it really bothers you, apply commit
> 1eb46908b35d ("x86/l1tf: Fix build error seen if CONFIG_KVM_INTEL is
> disabled") to your tree, it will be in the next round of stable kernel
> releases.

Probably obvious, but: cherry-picking that commit fixes the problem, it
builds, boots and works ok.

Thanks!


Meow.
-- 
⢀⣴⠾⠻⢶⣦⠀ What Would Jesus Do, MUD/MMORPG edition:
⣾⠁⢰⠒⠀⣿⡁ • multiplay with an admin char to benefit your mortal [Mt3:16-17]
⢿⡄⠘⠷⠚⠋⠀ • abuse item cloning bugs [Mt14:17-20, Mt15:34-37]
⠈⠳⣄ • use glitches to walk on water [Mt14:25-26]


Re: Linux 4.18.1

2018-08-16 Thread Adam Borowski
Hi!

On Thu, Aug 16, 2018 at 12:14:29PM +0200, Greg KH wrote:
> I'm announcing the release of the 4.18.1 kernel.

I'm afraid that I get a build failure; v4.18 is ok, v4.18.1 fails with:

ld: arch/x86/kvm/x86.o: in function `kvm_get_arch_capabilities':
(.text+0x43b2): undefined reference to `l1tf_vmx_mitigation'

gcc 8.2.0-4, as shipped in Debian unstable.
.config attached.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ What Would Jesus Do, MUD/MMORPG edition:
⣾⠁⢰⠒⠀⣿⡁ • multiplay with an admin char to benefit your mortal [Mt3:16-17]
⢿⡄⠘⠷⠚⠋⠀ • abuse item cloning bugs [Mt14:17-20, Mt15:34-37]
⠈⠳⣄ • use glitches to walk on water [Mt14:25-26]


config.xz
Description: application/xz


Re: Linux 4.18.1

2018-08-16 Thread Adam Borowski
Hi!

On Thu, Aug 16, 2018 at 12:14:29PM +0200, Greg KH wrote:
> I'm announcing the release of the 4.18.1 kernel.

I'm afraid that I get a build failure; v4.18 is ok, v4.18.1 fails with:

ld: arch/x86/kvm/x86.o: in function `kvm_get_arch_capabilities':
(.text+0x43b2): undefined reference to `l1tf_vmx_mitigation'

gcc 8.2.0-4, as shipped in Debian unstable.
.config attached.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ What Would Jesus Do, MUD/MMORPG edition:
⣾⠁⢰⠒⠀⣿⡁ • multiplay with an admin char to benefit your mortal [Mt3:16-17]
⢿⡄⠘⠷⠚⠋⠀ • abuse item cloning bugs [Mt14:17-20, Mt15:34-37]
⠈⠳⣄ • use glitches to walk on water [Mt14:25-26]


config.xz
Description: application/xz


Re: get_maintainer.pl and change of email

2018-08-01 Thread Adam Borowski
On Wed, Aug 01, 2018 at 11:36:23AM -0700, Nick Desaulniers wrote:
> It seems that get_maintainer.pl will make recommendations based on
> commit history to a file, but over time, people change emails that
> they commit from, then get_maintainer.pl recommends the possibly now
> invalid email address.
> 
> I feel like we should have a look up table / file like MAINTAINERS
> that tracks previous and new email addresses, which get_maintainer.pl
> would parse, that way we have a record of movement between committer
> email addresses allowing get_maintainer.pl to make more accurate
> recommendations.  Thoughts?

.mailmap

-- 
// If you believe in so-called "intellectual property", please immediately
// cease using counterfeit alphabets.  Instead, contact the nearest temple
// of Amon, whose priests will provide you with scribal services for all
// your writing needs, for Reasonable And Non-Discriminatory prices.


Re: get_maintainer.pl and change of email

2018-08-01 Thread Adam Borowski
On Wed, Aug 01, 2018 at 11:36:23AM -0700, Nick Desaulniers wrote:
> It seems that get_maintainer.pl will make recommendations based on
> commit history to a file, but over time, people change emails that
> they commit from, then get_maintainer.pl recommends the possibly now
> invalid email address.
> 
> I feel like we should have a look up table / file like MAINTAINERS
> that tracks previous and new email addresses, which get_maintainer.pl
> would parse, that way we have a record of movement between committer
> email addresses allowing get_maintainer.pl to make more accurate
> recommendations.  Thoughts?

.mailmap

-- 
// If you believe in so-called "intellectual property", please immediately
// cease using counterfeit alphabets.  Instead, contact the nearest temple
// of Amon, whose priests will provide you with scribal services for all
// your writing needs, for Reasonable And Non-Discriminatory prices.


Re: [PATCH 0/6] vt: no blinking on console, 256/24-bit color improvements

2018-07-23 Thread Adam Borowski
On Mon, Jul 23, 2018 at 10:53:29AM +0200, Geert Uytterhoeven wrote:
> On Sat, Jul 21, 2018 at 11:39 PM Adam Borowski  wrote:
> > Technically, every console can be made to blink by drawing/clearing affected
> > characters a few times per second, but that'd be quite a waste of coding
> > time and kernel size.  There's a reason browsers dropped support for 
> > and text-decoration:blink.
> 
> It's very simple and fast to implement in fbcon for FB_TYPE_PLANES or
> FB_TYPE_INTERLEAVED_PLANES and FB_VISUAL_PSEUDOCOLOR ;-)

Interesting...  I'm still not going to do the effort to implement that
(which would require learning fbdev internals first), though.  But while I
dislike this feature, someone else might want it.

The main problem here is that there are only 8 or 7 bits available for
attributes, thus it's better to use them for something more useful.  And
here, fbcon already interprets this bit as bright background, thus this
patchset makes vt use it instead of non-existant blink.

There'll be more bits available once attributes get migrated into uniscr --
either 11 or 32 bits depending on chosen implementation.  But I still
wouldn't go too wild with them: the console is meant for recovery tasks as
on any properly working system you can have an X terminal configured for
pixel-to-pixel identical behaviour as anything console can do.  Thus, only
cheap improvements to attributes make sense.  This patchset is currently at
+3 net lines, this certainly counts as cheap.

On the other hand, displaying the _glyph_ better would be worth some effort.
Nicolas' uniscr that's already in Greg's tree did most of the work, what
remains is a way to load and store a font with more glyphs (needs a sparse
representation).


Meow!
-- 
// If you believe in so-called "intellectual property", please immediately
// cease using counterfeit alphabets.  Instead, contact the nearest temple
// of Amon, whose priests will provide you with scribal services for all
// your writing needs, for Reasonable And Non-Discriminatory prices.


Re: [PATCH 0/6] vt: no blinking on console, 256/24-bit color improvements

2018-07-23 Thread Adam Borowski
On Mon, Jul 23, 2018 at 10:53:29AM +0200, Geert Uytterhoeven wrote:
> On Sat, Jul 21, 2018 at 11:39 PM Adam Borowski  wrote:
> > Technically, every console can be made to blink by drawing/clearing affected
> > characters a few times per second, but that'd be quite a waste of coding
> > time and kernel size.  There's a reason browsers dropped support for 
> > and text-decoration:blink.
> 
> It's very simple and fast to implement in fbcon for FB_TYPE_PLANES or
> FB_TYPE_INTERLEAVED_PLANES and FB_VISUAL_PSEUDOCOLOR ;-)

Interesting...  I'm still not going to do the effort to implement that
(which would require learning fbdev internals first), though.  But while I
dislike this feature, someone else might want it.

The main problem here is that there are only 8 or 7 bits available for
attributes, thus it's better to use them for something more useful.  And
here, fbcon already interprets this bit as bright background, thus this
patchset makes vt use it instead of non-existant blink.

There'll be more bits available once attributes get migrated into uniscr --
either 11 or 32 bits depending on chosen implementation.  But I still
wouldn't go too wild with them: the console is meant for recovery tasks as
on any properly working system you can have an X terminal configured for
pixel-to-pixel identical behaviour as anything console can do.  Thus, only
cheap improvements to attributes make sense.  This patchset is currently at
+3 net lines, this certainly counts as cheap.

On the other hand, displaying the _glyph_ better would be worth some effort.
Nicolas' uniscr that's already in Greg's tree did most of the work, what
remains is a way to load and store a font with more glyphs (needs a sparse
representation).


Meow!
-- 
// If you believe in so-called "intellectual property", please immediately
// cease using counterfeit alphabets.  Instead, contact the nearest temple
// of Amon, whose priests will provide you with scribal services for all
// your writing needs, for Reasonable And Non-Discriminatory prices.


Re: [PATCH 0/6] vt: no blinking on console, 256/24-bit color improvements

2018-07-21 Thread Adam Borowski
On Sat, Jul 21, 2018 at 09:43:19AM +0200, Greg Kroah-Hartman wrote:
> On Wed, Jul 18, 2018 at 05:01:52AM +0200, Adam Borowski wrote:
> > Here's a patchset with two entangled improvements:
> > 
> > * it'd be good to get rid of blinking where possible.  Even CGA (thus VGA)
> >   allows disabling it, rendering such characters with a bright background
> >   instead.
> > * due to my error, 256-color mode uses a much darker palette for conversion,
> >   resulting in behaving inconsistently with 24-bit mode.
> > 
> > The new code uses bright backgrounds when possible, enabled with \e[100m or
> > \e[48;m.
> > 
> > Despite the whole idea following a VGA capability, this patchset doesn't
> > change vgacon yet, just fbcon.  [Not breaking nvidia-proprietary.]
> > 
> > Thus, let's enable unblinking on fbcon for now.  We can flip that bit (in
> > register 0x10) later.
> > 
> > This fixes the display of catimg and similar tools.
> 
> I've applied the first patch, as it was obvious :)
> 
> For the rest, can you make it a config option as Alan said?  And I
> agree, we don't care about breaking nvidia systems, go ahead :)

The only thing such an option would be able to set is disabling blinking on
vgacon, which is not yet implemented.  fbcon never blinks, and the only case
it can't display bright background is 1-bit black

I probably should change the line in fbcon:
+   vc->vc_unblinking = vc->vc_can_do_color;
to always use 1, but it doesn't matter either way.  Only difference would be
matching the variable name ("unblinking" vs "shows_bright_bg").

Console drivers can interpret that bit of attribute as:
* blinking (vc_unblinking = 0)
* bright background (vc_unblinking = 1)
* neither (result is ignored down the pipeline)

The matrix is:
color b
---+---+--
fbbright bg neither
mda  N/A blink
newport  ?bright bg? ?N/A?
sti   ?neither??
vga  CGA/VGA: hardware switch (write to port)  MDA: blink
sisusb??

Technically, every console can be made to blink by drawing/clearing affected
characters a few times per second, but that'd be quite a waste of coding
time and kernel size.  There's a reason browsers dropped support for 
and text-decoration:blink.

This patchset doesn't drop blink in any case that was supported before,
though.  I did the easy part used by most people first (fbcon), so it can be
reviewed/merged.  Doing the hard part is quite pointless if this is NAKed
(but obviously doesn't require actual merging).

But perhaps by "option" you mean letting the user drop blink with no
replacement, which might also be a good idea if configurable?


Meow!
-- 
// If you believe in so-called "intellectual property", please immediately
// cease using counterfeit alphabets.  Instead, contact the nearest temple
// of Amon, whose priests will provide you with scribal services for all
// your writing needs, for Reasonable And Non-Discriminatory prices.
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 0/6] vt: no blinking on console, 256/24-bit color improvements

2018-07-21 Thread Adam Borowski
On Sat, Jul 21, 2018 at 09:43:19AM +0200, Greg Kroah-Hartman wrote:
> On Wed, Jul 18, 2018 at 05:01:52AM +0200, Adam Borowski wrote:
> > Here's a patchset with two entangled improvements:
> > 
> > * it'd be good to get rid of blinking where possible.  Even CGA (thus VGA)
> >   allows disabling it, rendering such characters with a bright background
> >   instead.
> > * due to my error, 256-color mode uses a much darker palette for conversion,
> >   resulting in behaving inconsistently with 24-bit mode.
> > 
> > The new code uses bright backgrounds when possible, enabled with \e[100m or
> > \e[48;m.
> > 
> > Despite the whole idea following a VGA capability, this patchset doesn't
> > change vgacon yet, just fbcon.  [Not breaking nvidia-proprietary.]
> > 
> > Thus, let's enable unblinking on fbcon for now.  We can flip that bit (in
> > register 0x10) later.
> > 
> > This fixes the display of catimg and similar tools.
> 
> I've applied the first patch, as it was obvious :)
> 
> For the rest, can you make it a config option as Alan said?  And I
> agree, we don't care about breaking nvidia systems, go ahead :)

The only thing such an option would be able to set is disabling blinking on
vgacon, which is not yet implemented.  fbcon never blinks, and the only case
it can't display bright background is 1-bit black

I probably should change the line in fbcon:
+   vc->vc_unblinking = vc->vc_can_do_color;
to always use 1, but it doesn't matter either way.  Only difference would be
matching the variable name ("unblinking" vs "shows_bright_bg").

Console drivers can interpret that bit of attribute as:
* blinking (vc_unblinking = 0)
* bright background (vc_unblinking = 1)
* neither (result is ignored down the pipeline)

The matrix is:
color b
---+---+--
fbbright bg neither
mda  N/A blink
newport  ?bright bg? ?N/A?
sti   ?neither??
vga  CGA/VGA: hardware switch (write to port)  MDA: blink
sisusb??

Technically, every console can be made to blink by drawing/clearing affected
characters a few times per second, but that'd be quite a waste of coding
time and kernel size.  There's a reason browsers dropped support for 
and text-decoration:blink.

This patchset doesn't drop blink in any case that was supported before,
though.  I did the easy part used by most people first (fbcon), so it can be
reviewed/merged.  Doing the hard part is quite pointless if this is NAKed
(but obviously doesn't require actual merging).

But perhaps by "option" you mean letting the user drop blink with no
replacement, which might also be a good idea if configurable?


Meow!
-- 
// If you believe in so-called "intellectual property", please immediately
// cease using counterfeit alphabets.  Instead, contact the nearest temple
// of Amon, whose priests will provide you with scribal services for all
// your writing needs, for Reasonable And Non-Discriminatory prices.
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 0/6] vt: no blinking on console, 256/24-bit color improvements

2018-07-19 Thread Adam Borowski
On Thu, Jul 19, 2018 at 11:47:49AM +0100, Alan Cox wrote:
> On Wed, 18 Jul 2018 05:01:52 +0200
> Adam Borowski  wrote:
> 
> > Hi!
> > Here's a patchset with two entangled improvements:
> > 
> > * it'd be good to get rid of blinking where possible.  Even CGA (thus VGA)
> >   allows disabling it, rendering such characters with a bright background
> >   instead.
> 
> That's a matter of taste so needs to configurable. Changing the default
> ought I think to be its own patch as it's a separate discussion to having
> the choice there and easy to make.

For vgacon yeah, you have a good point, as it can support either (and to be
exact, exactly one of the two as they share a hardware bit).  Only reason to
not have this configurable would be avoiding bloating the kernel with knobs
hardly anyone flips -- but you can already set minutiae like replacement
color for underline/italic/dim.  Thus you're right that if/when killing
blink on vgacon is implemented, it probably should be settable.

This is not an option on fbcon, though -- it can't blink (doable but I'm not
seeing anyone wanting to implement that) and already interprets that
attribute bit the way VGA would.  Thus, this patch merely makes vt behave a
way to match what the driver does.  This fixes some visual corruption in
certain user programs.

> For the palette why does it needs changing and exactly what standards
> document defines 'right', especially given we don't do ICC in console
> mode ? Have you tested the values used against multiple monitor types and
> cards with a light meter ?

You don't need a light meter for a difference of 53-out-of-256.  Most
desktop environments don't do ICC out of the box either, but there's an
assumption that whatever your monitor does, it gives the same result for the
same input (in the same lighting conditions).

That patch's purpose is:
* behave consistently between two APIs to set the same thing (\e[38;5;m vs
  \e[38;2;m) in a way that matches other terminals
* in case a future driver has better color handling we'd be different from
  other terminals

By the way, you can have ICC on console: "apt install vtgamma"
(https://github.com/kilobyte/vtgamma).

> BTW visibly breaking the Nvidia crud is also fine. They'll then actually
> bother to fix it and uually quite soon.

Yeah, but they haven't fixed 512-glyph yet (at least the last time I
looked), broken since day one.  In any case, this patchset doesn't support
vgacon yet, thus this is moot for now.  I picked the easy case (fbcon which
is always unblinking) over writing stuff to ports, we can update other
drivers later.

And vgacon has code that looks like it can do CGA and MDA (redundant with
mdacon?), either of which I haven't used in... quite a while.  Might be
tricky getting access to such hardware to test...


Meow!
-- 
// If you believe in so-called "intellectual property", please immediately
// cease using counterfeit alphabets.  Instead, contact the nearest temple
// of Amon, whose priests will provide you with scribal services for all
// your writing needs, for Reasonable And Non-Discriminatory prices.


Re: [PATCH 0/6] vt: no blinking on console, 256/24-bit color improvements

2018-07-19 Thread Adam Borowski
On Thu, Jul 19, 2018 at 11:47:49AM +0100, Alan Cox wrote:
> On Wed, 18 Jul 2018 05:01:52 +0200
> Adam Borowski  wrote:
> 
> > Hi!
> > Here's a patchset with two entangled improvements:
> > 
> > * it'd be good to get rid of blinking where possible.  Even CGA (thus VGA)
> >   allows disabling it, rendering such characters with a bright background
> >   instead.
> 
> That's a matter of taste so needs to configurable. Changing the default
> ought I think to be its own patch as it's a separate discussion to having
> the choice there and easy to make.

For vgacon yeah, you have a good point, as it can support either (and to be
exact, exactly one of the two as they share a hardware bit).  Only reason to
not have this configurable would be avoiding bloating the kernel with knobs
hardly anyone flips -- but you can already set minutiae like replacement
color for underline/italic/dim.  Thus you're right that if/when killing
blink on vgacon is implemented, it probably should be settable.

This is not an option on fbcon, though -- it can't blink (doable but I'm not
seeing anyone wanting to implement that) and already interprets that
attribute bit the way VGA would.  Thus, this patch merely makes vt behave a
way to match what the driver does.  This fixes some visual corruption in
certain user programs.

> For the palette why does it needs changing and exactly what standards
> document defines 'right', especially given we don't do ICC in console
> mode ? Have you tested the values used against multiple monitor types and
> cards with a light meter ?

You don't need a light meter for a difference of 53-out-of-256.  Most
desktop environments don't do ICC out of the box either, but there's an
assumption that whatever your monitor does, it gives the same result for the
same input (in the same lighting conditions).

That patch's purpose is:
* behave consistently between two APIs to set the same thing (\e[38;5;m vs
  \e[38;2;m) in a way that matches other terminals
* in case a future driver has better color handling we'd be different from
  other terminals

By the way, you can have ICC on console: "apt install vtgamma"
(https://github.com/kilobyte/vtgamma).

> BTW visibly breaking the Nvidia crud is also fine. They'll then actually
> bother to fix it and uually quite soon.

Yeah, but they haven't fixed 512-glyph yet (at least the last time I
looked), broken since day one.  In any case, this patchset doesn't support
vgacon yet, thus this is moot for now.  I picked the easy case (fbcon which
is always unblinking) over writing stuff to ports, we can update other
drivers later.

And vgacon has code that looks like it can do CGA and MDA (redundant with
mdacon?), either of which I haven't used in... quite a while.  Might be
tricky getting access to such hardware to test...


Meow!
-- 
// If you believe in so-called "intellectual property", please immediately
// cease using counterfeit alphabets.  Instead, contact the nearest temple
// of Amon, whose priests will provide you with scribal services for all
// your writing needs, for Reasonable And Non-Discriminatory prices.


[PATCH 3/6] vt: let \e[100m use bright background if unblinking

2018-07-17 Thread Adam Borowski
Let's keep \e[5m setting this bit, it's a nice way to convey the
information, and it preserves old behaviour.  Some other terminals
that can't or don't want to blink do so as well.

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/vt.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
index 45057bbf6f74..4096093c8cd2 100644
--- a/drivers/tty/vt/vt.c
+++ b/drivers/tty/vt/vt.c
@@ -1709,6 +1709,8 @@ static void csi_m(struct vc_data *vc)
if (vc->vc_par[i] >= 90 && vc->vc_par[i] <= 107) {
if (vc->vc_par[i] < 100)
vc->vc_intensity = 2;
+   else if (vc->vc_unblinking)
+   vc->vc_blink = 1;
vc->vc_par[i] -= 60;
}
if (vc->vc_par[i] >= 30 && vc->vc_par[i] <= 37)
-- 
2.18.0

--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 3/6] vt: let \e[100m use bright background if unblinking

2018-07-17 Thread Adam Borowski
Let's keep \e[5m setting this bit, it's a nice way to convey the
information, and it preserves old behaviour.  Some other terminals
that can't or don't want to blink do so as well.

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/vt.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
index 45057bbf6f74..4096093c8cd2 100644
--- a/drivers/tty/vt/vt.c
+++ b/drivers/tty/vt/vt.c
@@ -1709,6 +1709,8 @@ static void csi_m(struct vc_data *vc)
if (vc->vc_par[i] >= 90 && vc->vc_par[i] <= 107) {
if (vc->vc_par[i] < 100)
vc->vc_intensity = 2;
+   else if (vc->vc_unblinking)
+   vc->vc_blink = 1;
vc->vc_par[i] -= 60;
}
if (vc->vc_par[i] >= 30 && vc->vc_par[i] <= 37)
-- 
2.18.0

--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 1/6] vt: drop unused struct vt_struct

2018-07-17 Thread Adam Borowski
Hasn't been ever used within historic (ie, git) times.

Signed-off-by: Adam Borowski 
---
 include/linux/console_struct.h | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/include/linux/console_struct.h b/include/linux/console_struct.h
index 2c8d3239899b..fea64f2692a0 100644
--- a/include/linux/console_struct.h
+++ b/include/linux/console_struct.h
@@ -17,7 +17,6 @@
 #include 
 #include 
 
-struct vt_struct;
 struct uni_pagedir;
 struct uni_screen;
 
@@ -150,7 +149,7 @@ struct vc {
struct vc_data *d;
struct work_struct SAK_work;
 
-   /* might add  scrmem, vt_struct, kbd  at some time,
+   /* might add  scrmem, kbd  at some time,
   to have everything in one place - the disadvantage
   would be that vc_cons etc can no longer be static */
 };
-- 
2.18.0



[PATCH 2/6] vt: add console flag "unblinking"

2018-07-17 Thread Adam Borowski
Marks consoles that interpret the blink bit by showing bright background
instead.  Doesn't matter if the console can't do either.

For now, turn it on for fbcon when in color mode.  Vgacon can also do so
but requires setting appropriate VGA register (bit 3 of AMCR).  I don't
know other consoles: newport looks like it shows bright bg, sti can't do
either, mda appears to blink, etc -- but confirmation would be needed.

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/vt.c  | 1 +
 drivers/video/fbdev/core/fbcon.c | 1 +
 include/linux/console_struct.h   | 1 +
 3 files changed, 3 insertions(+)

diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
index 846dfedb657d..45057bbf6f74 100644
--- a/drivers/tty/vt/vt.c
+++ b/drivers/tty/vt/vt.c
@@ -998,6 +998,7 @@ static void visual_init(struct vc_data *vc, int num, int 
init)
vc->vc_hi_font_mask = 0;
vc->vc_complement_mask = 0;
vc->vc_can_do_color = 0;
+   vc->vc_unblinking = 0;
vc->vc_panic_force_write = false;
vc->vc_cur_blink_ms = DEFAULT_CURSOR_BLINK_MS;
vc->vc_sw->con_init(vc, init);
diff --git a/drivers/video/fbdev/core/fbcon.c b/drivers/video/fbdev/core/fbcon.c
index c910e74d46ff..4c67254f1ec4 100644
--- a/drivers/video/fbdev/core/fbcon.c
+++ b/drivers/video/fbdev/core/fbcon.c
@@ -1092,6 +1092,7 @@ static void fbcon_init(struct vc_data *vc, int init)
 
vc->vc_panic_force_write = !!(info->flags & FBINFO_CAN_FORCE_OUTPUT);
vc->vc_can_do_color = (fb_get_color_depth(>var, >fix)!=1);
+   vc->vc_unblinking = vc->vc_can_do_color;
vc->vc_complement_mask = vc->vc_can_do_color ? 0x7700 : 0x0800;
if (charcnt == 256) {
vc->vc_hi_font_mask = 0;
diff --git a/include/linux/console_struct.h b/include/linux/console_struct.h
index fea64f2692a0..f94b28a6bd2d 100644
--- a/include/linux/console_struct.h
+++ b/include/linux/console_struct.h
@@ -122,6 +122,7 @@ struct vc_data {
unsigned intvc_ques : 1;
unsigned intvc_need_wrap: 1;
unsigned intvc_can_do_color : 1;
+   unsigned intvc_unblinking   : 1;/* shows bright bg for blink */
unsigned intvc_report_mouse : 2;
unsigned char   vc_utf  : 1;/* Unicode UTF-8 encoding */
unsigned char   vc_utf_count;
-- 
2.18.0



[PATCH 5/6] vt: compensate for brightening the 256-color palette

2018-07-17 Thread Adam Borowski
The algorithm for 256-to-16 conversion was designed with wrong input
palette but actually tuned on mainstream GUI terminals.  This resulted in
something that works well only for data we convert ourselves (ie, 256 not
24-bit).

As the change is non-linear, I did not bother replicating it exactly, thus
there are some differences, among others:
* values very close to black go to 0 (black) rather than 8 (dark grey)
* grayscale ramp is more even

A comparison of the old vs new vs FreeBSD's teken is at:
https://github.com/kilobyte/colorkernel

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/vt.c | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
index 8c61caafdf3c..c777f4c91df0 100644
--- a/drivers/tty/vt/vt.c
+++ b/drivers/tty/vt/vt.c
@@ -1559,17 +1559,17 @@ static void rgb_foreground(struct vc_data *vc, const 
struct rgb *c)
 {
u8 hue = 0, max = max3(c->r, c->g, c->b);
 
-   if (c->r > max / 2)
+   if (c->r > max / 2 + 32)
hue |= 4;
-   if (c->g > max / 2)
+   if (c->g > max / 2 + 32)
hue |= 2;
-   if (c->b > max / 2)
+   if (c->b > max / 2 + 32)
hue |= 1;
 
-   if (hue == 7 && max <= 0x55) {
+   if (hue == 7 && max <= 0x70) {
hue = 0;
vc->vc_intensity = 2;
-   } else if (max > 0xaa)
+   } else if (max > 0xc0)
vc->vc_intensity = 2;
else
vc->vc_intensity = 1;
-- 
2.18.0



[PATCH 4/6] vt: change 256-color palette to match all(?) modern terminals

2018-07-17 Thread Adam Borowski
Turns out that osso-xterm which I based upon uses something a lot different
from apparently any other terminal -- they all use identical shades, much
brighter than what I copied:

Old: 00 2a 55 7f aa d4
New: 00 5f 87 af d7 ff

This did hardly matter as we immediately shoehorn the colors into only 16
values, but recently 24-bit codes turned from an oddity to something
widespread, thus it's better to handle 256 vs 24-bit consistently.

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/vt.c | 9 ++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
index 4096093c8cd2..8c61caafdf3c 100644
--- a/drivers/tty/vt/vt.c
+++ b/drivers/tty/vt/vt.c
@@ -1545,9 +1545,12 @@ static void rgb_from_256(int i, struct rgb *c)
c->g = i&2 ? 0xff : 0x55;
c->b = i&4 ? 0xff : 0x55;
} else if (i < 232) {   /* 6x6x6 colour cube. */
-   c->r = (i - 16) / 36 * 85 / 2;
-   c->g = (i - 16) / 6 % 6 * 85 / 2;
-   c->b = (i - 16) % 6 * 85 / 2;
+   int r = (i - 16) / 36;
+   int g = (i - 16) / 6 % 6;
+   int b = (i - 16) % 6;
+   c->r = r ? r * 0x28 + 0x37 : 0;
+   c->g = g ? g * 0x28 + 0x37 : 0;
+   c->b = b ? b * 0x28 + 0x37 : 0;
} else  /* Grayscale ramp. */
c->r = c->g = c->b = i * 10 - 2312;
 }
-- 
2.18.0

--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 6/6] vt: support bright backgrounds for \e[48m if unblinking

2018-07-17 Thread Adam Borowski
This improves schemes used by fancy new programs.  For example, it lets
bright powerline segments match, and fixes images shown by catimg being
striped to the point of unreadability.

Handling of 8-color backgrounds uses stripped 16-color value instead of a
dedicated formula; this works worse for dark and better for bright inputs.

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/vt.c | 38 +-
 1 file changed, 17 insertions(+), 21 deletions(-)

diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
index c777f4c91df0..7fcb0ff2dccf 100644
--- a/drivers/tty/vt/vt.c
+++ b/drivers/tty/vt/vt.c
@@ -1555,7 +1555,7 @@ static void rgb_from_256(int i, struct rgb *c)
c->r = c->g = c->b = i * 10 - 2312;
 }
 
-static void rgb_foreground(struct vc_data *vc, const struct rgb *c)
+static u8 rgb_to_16(const struct rgb *c)
 {
u8 hue = 0, max = max3(c->r, c->g, c->b);
 
@@ -1566,22 +1566,12 @@ static void rgb_foreground(struct vc_data *vc, const 
struct rgb *c)
if (c->b > max / 2 + 32)
hue |= 1;
 
-   if (hue == 7 && max <= 0x70) {
-   hue = 0;
-   vc->vc_intensity = 2;
-   } else if (max > 0xc0)
-   vc->vc_intensity = 2;
+   if (hue == 7 && max <= 0x70)
+   return 8;
+   if (max > 0xc0)
+   return hue | 8;
else
-   vc->vc_intensity = 1;
-
-   vc->vc_color = (vc->vc_color & 0xf0) | hue;
-}
-
-static void rgb_background(struct vc_data *vc, const struct rgb *c)
-{
-   /* For backgrounds, err on the dark side. */
-   vc->vc_color = (vc->vc_color & 0x0f)
-   | (c->r&0x80) >> 1 | (c->g&0x80) >> 2 | (c->b&0x80) >> 3;
+   return hue;
 }
 
 /*
@@ -1593,10 +1583,10 @@ static void rgb_background(struct vc_data *vc, const 
struct rgb *c)
  * Subcommands 3 (CMY) and 4 (CMYK) are so insane there's no point in
  * supporting them.
  */
-static int vc_t416_color(struct vc_data *vc, int i,
-   void(*set_color)(struct vc_data *vc, const struct rgb *c))
+static int vc_t416_color(struct vc_data *vc, int i, int bgshift)
 {
struct rgb c;
+   u8 v;
 
i++;
if (i > vc->vc_npar)
@@ -1615,7 +1605,13 @@ static int vc_t416_color(struct vc_data *vc, int i,
} else
return i;
 
-   set_color(vc, );
+   v = rgb_to_16();
+
+   vc->vc_color = (vc->vc_color & (0xf0 >> bgshift)) | v << bgshift;
+   if (!bgshift)
+   vc->vc_intensity = (v & 8 >> 4) + 1;
+   else if (vc->vc_unblinking)
+   vc->vc_blink = v & 8 >> 4;
 
return i;
 }
@@ -1695,10 +1691,10 @@ static void csi_m(struct vc_data *vc)
vc->vc_reverse = 0;
break;
case 38:
-   i = vc_t416_color(vc, i, rgb_foreground);
+   i = vc_t416_color(vc, i, 0);
break;
case 48:
-   i = vc_t416_color(vc, i, rgb_background);
+   i = vc_t416_color(vc, i, 4);
break;
case 39:
vc->vc_color = (vc->vc_def_color & 0x0f) |
-- 
2.18.0

--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 1/6] vt: drop unused struct vt_struct

2018-07-17 Thread Adam Borowski
Hasn't been ever used within historic (ie, git) times.

Signed-off-by: Adam Borowski 
---
 include/linux/console_struct.h | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/include/linux/console_struct.h b/include/linux/console_struct.h
index 2c8d3239899b..fea64f2692a0 100644
--- a/include/linux/console_struct.h
+++ b/include/linux/console_struct.h
@@ -17,7 +17,6 @@
 #include 
 #include 
 
-struct vt_struct;
 struct uni_pagedir;
 struct uni_screen;
 
@@ -150,7 +149,7 @@ struct vc {
struct vc_data *d;
struct work_struct SAK_work;
 
-   /* might add  scrmem, vt_struct, kbd  at some time,
+   /* might add  scrmem, kbd  at some time,
   to have everything in one place - the disadvantage
   would be that vc_cons etc can no longer be static */
 };
-- 
2.18.0



[PATCH 2/6] vt: add console flag "unblinking"

2018-07-17 Thread Adam Borowski
Marks consoles that interpret the blink bit by showing bright background
instead.  Doesn't matter if the console can't do either.

For now, turn it on for fbcon when in color mode.  Vgacon can also do so
but requires setting appropriate VGA register (bit 3 of AMCR).  I don't
know other consoles: newport looks like it shows bright bg, sti can't do
either, mda appears to blink, etc -- but confirmation would be needed.

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/vt.c  | 1 +
 drivers/video/fbdev/core/fbcon.c | 1 +
 include/linux/console_struct.h   | 1 +
 3 files changed, 3 insertions(+)

diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
index 846dfedb657d..45057bbf6f74 100644
--- a/drivers/tty/vt/vt.c
+++ b/drivers/tty/vt/vt.c
@@ -998,6 +998,7 @@ static void visual_init(struct vc_data *vc, int num, int 
init)
vc->vc_hi_font_mask = 0;
vc->vc_complement_mask = 0;
vc->vc_can_do_color = 0;
+   vc->vc_unblinking = 0;
vc->vc_panic_force_write = false;
vc->vc_cur_blink_ms = DEFAULT_CURSOR_BLINK_MS;
vc->vc_sw->con_init(vc, init);
diff --git a/drivers/video/fbdev/core/fbcon.c b/drivers/video/fbdev/core/fbcon.c
index c910e74d46ff..4c67254f1ec4 100644
--- a/drivers/video/fbdev/core/fbcon.c
+++ b/drivers/video/fbdev/core/fbcon.c
@@ -1092,6 +1092,7 @@ static void fbcon_init(struct vc_data *vc, int init)
 
vc->vc_panic_force_write = !!(info->flags & FBINFO_CAN_FORCE_OUTPUT);
vc->vc_can_do_color = (fb_get_color_depth(>var, >fix)!=1);
+   vc->vc_unblinking = vc->vc_can_do_color;
vc->vc_complement_mask = vc->vc_can_do_color ? 0x7700 : 0x0800;
if (charcnt == 256) {
vc->vc_hi_font_mask = 0;
diff --git a/include/linux/console_struct.h b/include/linux/console_struct.h
index fea64f2692a0..f94b28a6bd2d 100644
--- a/include/linux/console_struct.h
+++ b/include/linux/console_struct.h
@@ -122,6 +122,7 @@ struct vc_data {
unsigned intvc_ques : 1;
unsigned intvc_need_wrap: 1;
unsigned intvc_can_do_color : 1;
+   unsigned intvc_unblinking   : 1;/* shows bright bg for blink */
unsigned intvc_report_mouse : 2;
unsigned char   vc_utf  : 1;/* Unicode UTF-8 encoding */
unsigned char   vc_utf_count;
-- 
2.18.0



[PATCH 5/6] vt: compensate for brightening the 256-color palette

2018-07-17 Thread Adam Borowski
The algorithm for 256-to-16 conversion was designed with wrong input
palette but actually tuned on mainstream GUI terminals.  This resulted in
something that works well only for data we convert ourselves (ie, 256 not
24-bit).

As the change is non-linear, I did not bother replicating it exactly, thus
there are some differences, among others:
* values very close to black go to 0 (black) rather than 8 (dark grey)
* grayscale ramp is more even

A comparison of the old vs new vs FreeBSD's teken is at:
https://github.com/kilobyte/colorkernel

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/vt.c | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
index 8c61caafdf3c..c777f4c91df0 100644
--- a/drivers/tty/vt/vt.c
+++ b/drivers/tty/vt/vt.c
@@ -1559,17 +1559,17 @@ static void rgb_foreground(struct vc_data *vc, const 
struct rgb *c)
 {
u8 hue = 0, max = max3(c->r, c->g, c->b);
 
-   if (c->r > max / 2)
+   if (c->r > max / 2 + 32)
hue |= 4;
-   if (c->g > max / 2)
+   if (c->g > max / 2 + 32)
hue |= 2;
-   if (c->b > max / 2)
+   if (c->b > max / 2 + 32)
hue |= 1;
 
-   if (hue == 7 && max <= 0x55) {
+   if (hue == 7 && max <= 0x70) {
hue = 0;
vc->vc_intensity = 2;
-   } else if (max > 0xaa)
+   } else if (max > 0xc0)
vc->vc_intensity = 2;
else
vc->vc_intensity = 1;
-- 
2.18.0



[PATCH 4/6] vt: change 256-color palette to match all(?) modern terminals

2018-07-17 Thread Adam Borowski
Turns out that osso-xterm which I based upon uses something a lot different
from apparently any other terminal -- they all use identical shades, much
brighter than what I copied:

Old: 00 2a 55 7f aa d4
New: 00 5f 87 af d7 ff

This did hardly matter as we immediately shoehorn the colors into only 16
values, but recently 24-bit codes turned from an oddity to something
widespread, thus it's better to handle 256 vs 24-bit consistently.

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/vt.c | 9 ++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
index 4096093c8cd2..8c61caafdf3c 100644
--- a/drivers/tty/vt/vt.c
+++ b/drivers/tty/vt/vt.c
@@ -1545,9 +1545,12 @@ static void rgb_from_256(int i, struct rgb *c)
c->g = i&2 ? 0xff : 0x55;
c->b = i&4 ? 0xff : 0x55;
} else if (i < 232) {   /* 6x6x6 colour cube. */
-   c->r = (i - 16) / 36 * 85 / 2;
-   c->g = (i - 16) / 6 % 6 * 85 / 2;
-   c->b = (i - 16) % 6 * 85 / 2;
+   int r = (i - 16) / 36;
+   int g = (i - 16) / 6 % 6;
+   int b = (i - 16) % 6;
+   c->r = r ? r * 0x28 + 0x37 : 0;
+   c->g = g ? g * 0x28 + 0x37 : 0;
+   c->b = b ? b * 0x28 + 0x37 : 0;
} else  /* Grayscale ramp. */
c->r = c->g = c->b = i * 10 - 2312;
 }
-- 
2.18.0

--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 6/6] vt: support bright backgrounds for \e[48m if unblinking

2018-07-17 Thread Adam Borowski
This improves schemes used by fancy new programs.  For example, it lets
bright powerline segments match, and fixes images shown by catimg being
striped to the point of unreadability.

Handling of 8-color backgrounds uses stripped 16-color value instead of a
dedicated formula; this works worse for dark and better for bright inputs.

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/vt.c | 38 +-
 1 file changed, 17 insertions(+), 21 deletions(-)

diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
index c777f4c91df0..7fcb0ff2dccf 100644
--- a/drivers/tty/vt/vt.c
+++ b/drivers/tty/vt/vt.c
@@ -1555,7 +1555,7 @@ static void rgb_from_256(int i, struct rgb *c)
c->r = c->g = c->b = i * 10 - 2312;
 }
 
-static void rgb_foreground(struct vc_data *vc, const struct rgb *c)
+static u8 rgb_to_16(const struct rgb *c)
 {
u8 hue = 0, max = max3(c->r, c->g, c->b);
 
@@ -1566,22 +1566,12 @@ static void rgb_foreground(struct vc_data *vc, const 
struct rgb *c)
if (c->b > max / 2 + 32)
hue |= 1;
 
-   if (hue == 7 && max <= 0x70) {
-   hue = 0;
-   vc->vc_intensity = 2;
-   } else if (max > 0xc0)
-   vc->vc_intensity = 2;
+   if (hue == 7 && max <= 0x70)
+   return 8;
+   if (max > 0xc0)
+   return hue | 8;
else
-   vc->vc_intensity = 1;
-
-   vc->vc_color = (vc->vc_color & 0xf0) | hue;
-}
-
-static void rgb_background(struct vc_data *vc, const struct rgb *c)
-{
-   /* For backgrounds, err on the dark side. */
-   vc->vc_color = (vc->vc_color & 0x0f)
-   | (c->r&0x80) >> 1 | (c->g&0x80) >> 2 | (c->b&0x80) >> 3;
+   return hue;
 }
 
 /*
@@ -1593,10 +1583,10 @@ static void rgb_background(struct vc_data *vc, const 
struct rgb *c)
  * Subcommands 3 (CMY) and 4 (CMYK) are so insane there's no point in
  * supporting them.
  */
-static int vc_t416_color(struct vc_data *vc, int i,
-   void(*set_color)(struct vc_data *vc, const struct rgb *c))
+static int vc_t416_color(struct vc_data *vc, int i, int bgshift)
 {
struct rgb c;
+   u8 v;
 
i++;
if (i > vc->vc_npar)
@@ -1615,7 +1605,13 @@ static int vc_t416_color(struct vc_data *vc, int i,
} else
return i;
 
-   set_color(vc, );
+   v = rgb_to_16();
+
+   vc->vc_color = (vc->vc_color & (0xf0 >> bgshift)) | v << bgshift;
+   if (!bgshift)
+   vc->vc_intensity = (v & 8 >> 4) + 1;
+   else if (vc->vc_unblinking)
+   vc->vc_blink = v & 8 >> 4;
 
return i;
 }
@@ -1695,10 +1691,10 @@ static void csi_m(struct vc_data *vc)
vc->vc_reverse = 0;
break;
case 38:
-   i = vc_t416_color(vc, i, rgb_foreground);
+   i = vc_t416_color(vc, i, 0);
break;
case 48:
-   i = vc_t416_color(vc, i, rgb_background);
+   i = vc_t416_color(vc, i, 4);
break;
case 39:
vc->vc_color = (vc->vc_def_color & 0x0f) |
-- 
2.18.0

--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 0/6] vt: no blinking on console, 256/24-bit color improvements

2018-07-17 Thread Adam Borowski
Hi!
Here's a patchset with two entangled improvements:

* it'd be good to get rid of blinking where possible.  Even CGA (thus VGA)
  allows disabling it, rendering such characters with a bright background
  instead.
* due to my error, 256-color mode uses a much darker palette for conversion,
  resulting in behaving inconsistently with 24-bit mode.

The new code uses bright backgrounds when possible, enabled with \e[100m or
\e[48;m.

Despite the whole idea following a VGA capability, this patchset doesn't
change vgacon yet, just fbcon.  The reason being: ~80% of x86 users have an
nVidia chip, which means nouveau or nvidia-proprietary.  Nouveau implies
fbcon, nvidia-proprietary fails to properly restore text flags (as evidenced
by 512 glyph mode turning to 256 on switch from graphics).  You don't care
about the proprietary driver, but let's not break it pointlessly, and as
both nVidia cards I own work only with nouveau, I don't want to touch what I
can't test.

Thus, let's enable unblinking on fbcon for now.  We can flip that bit (in
register 0x10) later.

This fixes the display of catimg and similar tools.


Diffstat:
 drivers/tty/vt/vt.c  | 56 
+---
 drivers/video/fbdev/core/fbcon.c |  1 +
 include/linux/console_struct.h   |  4 ++--
 3 files changed, 32 insertions(+), 29 deletions(-)

-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢠⠒⠀⣿⡁ A dumb species has no way to open a tuna can.
⢿⡄⠘⠷⠚⠋⠀ A smart species invents a can opener.
⠈⠳⣄ A master species delegates.


[PATCH 0/6] vt: no blinking on console, 256/24-bit color improvements

2018-07-17 Thread Adam Borowski
Hi!
Here's a patchset with two entangled improvements:

* it'd be good to get rid of blinking where possible.  Even CGA (thus VGA)
  allows disabling it, rendering such characters with a bright background
  instead.
* due to my error, 256-color mode uses a much darker palette for conversion,
  resulting in behaving inconsistently with 24-bit mode.

The new code uses bright backgrounds when possible, enabled with \e[100m or
\e[48;m.

Despite the whole idea following a VGA capability, this patchset doesn't
change vgacon yet, just fbcon.  The reason being: ~80% of x86 users have an
nVidia chip, which means nouveau or nvidia-proprietary.  Nouveau implies
fbcon, nvidia-proprietary fails to properly restore text flags (as evidenced
by 512 glyph mode turning to 256 on switch from graphics).  You don't care
about the proprietary driver, but let's not break it pointlessly, and as
both nVidia cards I own work only with nouveau, I don't want to touch what I
can't test.

Thus, let's enable unblinking on fbcon for now.  We can flip that bit (in
register 0x10) later.

This fixes the display of catimg and similar tools.


Diffstat:
 drivers/tty/vt/vt.c  | 56 
+---
 drivers/video/fbdev/core/fbcon.c |  1 +
 include/linux/console_struct.h   |  4 ++--
 3 files changed, 32 insertions(+), 29 deletions(-)

-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢠⠒⠀⣿⡁ A dumb species has no way to open a tuna can.
⢿⡄⠘⠷⠚⠋⠀ A smart species invents a can opener.
⠈⠳⣄ A master species delegates.


[PATCH 3/3] vt: selection: take screen contents from uniscr if available

2018-07-17 Thread Adam Borowski
This preserves whatever was written even if we can't currently display the
given glyph.  Mouse paste won't corrupt any character of wcwidth() == 1
anymore.

Note that for now uniscr doesn't get allocated until something reads
/dev/vcsuN for that console, making this code dormant for most users.

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/selection.c | 11 +++
 drivers/tty/vt/vt.c| 10 ++
 include/linux/selection.h  |  1 +
 3 files changed, 18 insertions(+), 4 deletions(-)

diff --git a/drivers/tty/vt/selection.c b/drivers/tty/vt/selection.c
index 69ca337d3220..07496c711d7d 100644
--- a/drivers/tty/vt/selection.c
+++ b/drivers/tty/vt/selection.c
@@ -57,11 +57,13 @@ static inline void highlight_pointer(const int where)
complement_pos(sel_cons, where);
 }
 
-static u16
+static u32
 sel_pos(int n)
 {
+   if (use_unicode)
+   return screen_glyph_unicode(sel_cons, n / 2);
return inverse_translate(sel_cons, screen_glyph(sel_cons, n),
-   use_unicode);
+   0);
 }
 
 /**
@@ -90,7 +92,8 @@ static u32 inwordLut[]={
   0x07FE, /* lowercase */
 };
 
-static inline int inword(const u16 c) {
+static inline int inword(const u32 c)
+{
return c > 0x7f || (( inwordLut[c>>5] >> (c & 0x1F) ) & 1);
 }
 
@@ -167,7 +170,7 @@ int set_selection(const struct tiocl_selection __user *sel, 
struct tty_struct *t
struct tiocl_selection v;
char *bp, *obp;
int i, ps, pe, multiplier;
-   u16 c;
+   u32 c;
int mode;
 
poke_blanked_console();
diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
index 7fcb0ff2dccf..19da4c0b4b8e 100644
--- a/drivers/tty/vt/vt.c
+++ b/drivers/tty/vt/vt.c
@@ -4545,6 +4545,16 @@ u16 screen_glyph(struct vc_data *vc, int offset)
 }
 EXPORT_SYMBOL_GPL(screen_glyph);
 
+u32 screen_glyph_unicode(struct vc_data *vc, int n)
+{
+   struct uni_screen *uniscr = get_vc_uniscr(vc);
+
+   if (uniscr)
+   return uniscr->lines[n / vc->vc_cols][n % vc->vc_cols];
+   return inverse_translate(vc, screen_glyph(vc, n * 2), 1);
+}
+EXPORT_SYMBOL_GPL(screen_glyph_unicode);
+
 /* used by vcs - note the word offset */
 unsigned short *screen_pos(struct vc_data *vc, int w_offset, int viewed)
 {
diff --git a/include/linux/selection.h b/include/linux/selection.h
index 067d2e99c79f..a8f5b97b216f 100644
--- a/include/linux/selection.h
+++ b/include/linux/selection.h
@@ -32,6 +32,7 @@ extern unsigned char default_blu[];
 
 extern unsigned short *screen_pos(struct vc_data *vc, int w_offset, int 
viewed);
 extern u16 screen_glyph(struct vc_data *vc, int offset);
+extern u32 screen_glyph_unicode(struct vc_data *vc, int offset);
 extern void complement_pos(struct vc_data *vc, int offset);
 extern void invert_screen(struct vc_data *vc, int offset, int count, int 
shift);
 
-- 
2.18.0

--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 3/3] vt: selection: take screen contents from uniscr if available

2018-07-17 Thread Adam Borowski
This preserves whatever was written even if we can't currently display the
given glyph.  Mouse paste won't corrupt any character of wcwidth() == 1
anymore.

Note that for now uniscr doesn't get allocated until something reads
/dev/vcsuN for that console, making this code dormant for most users.

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/selection.c | 11 +++
 drivers/tty/vt/vt.c| 10 ++
 include/linux/selection.h  |  1 +
 3 files changed, 18 insertions(+), 4 deletions(-)

diff --git a/drivers/tty/vt/selection.c b/drivers/tty/vt/selection.c
index 69ca337d3220..07496c711d7d 100644
--- a/drivers/tty/vt/selection.c
+++ b/drivers/tty/vt/selection.c
@@ -57,11 +57,13 @@ static inline void highlight_pointer(const int where)
complement_pos(sel_cons, where);
 }
 
-static u16
+static u32
 sel_pos(int n)
 {
+   if (use_unicode)
+   return screen_glyph_unicode(sel_cons, n / 2);
return inverse_translate(sel_cons, screen_glyph(sel_cons, n),
-   use_unicode);
+   0);
 }
 
 /**
@@ -90,7 +92,8 @@ static u32 inwordLut[]={
   0x07FE, /* lowercase */
 };
 
-static inline int inword(const u16 c) {
+static inline int inword(const u32 c)
+{
return c > 0x7f || (( inwordLut[c>>5] >> (c & 0x1F) ) & 1);
 }
 
@@ -167,7 +170,7 @@ int set_selection(const struct tiocl_selection __user *sel, 
struct tty_struct *t
struct tiocl_selection v;
char *bp, *obp;
int i, ps, pe, multiplier;
-   u16 c;
+   u32 c;
int mode;
 
poke_blanked_console();
diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
index 7fcb0ff2dccf..19da4c0b4b8e 100644
--- a/drivers/tty/vt/vt.c
+++ b/drivers/tty/vt/vt.c
@@ -4545,6 +4545,16 @@ u16 screen_glyph(struct vc_data *vc, int offset)
 }
 EXPORT_SYMBOL_GPL(screen_glyph);
 
+u32 screen_glyph_unicode(struct vc_data *vc, int n)
+{
+   struct uni_screen *uniscr = get_vc_uniscr(vc);
+
+   if (uniscr)
+   return uniscr->lines[n / vc->vc_cols][n % vc->vc_cols];
+   return inverse_translate(vc, screen_glyph(vc, n * 2), 1);
+}
+EXPORT_SYMBOL_GPL(screen_glyph_unicode);
+
 /* used by vcs - note the word offset */
 unsigned short *screen_pos(struct vc_data *vc, int w_offset, int viewed)
 {
diff --git a/include/linux/selection.h b/include/linux/selection.h
index 067d2e99c79f..a8f5b97b216f 100644
--- a/include/linux/selection.h
+++ b/include/linux/selection.h
@@ -32,6 +32,7 @@ extern unsigned char default_blu[];
 
 extern unsigned short *screen_pos(struct vc_data *vc, int w_offset, int 
viewed);
 extern u16 screen_glyph(struct vc_data *vc, int offset);
+extern u32 screen_glyph_unicode(struct vc_data *vc, int offset);
 extern void complement_pos(struct vc_data *vc, int offset);
 extern void invert_screen(struct vc_data *vc, int offset, int count, int 
shift);
 
-- 
2.18.0

--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 2/3] vt: selection: handle storing of characters above U+FFFF

2018-07-17 Thread Adam Borowski
Those above U+10 get replaced with U+FFFD.

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/selection.c | 23 ++-
 1 file changed, 18 insertions(+), 5 deletions(-)

diff --git a/drivers/tty/vt/selection.c b/drivers/tty/vt/selection.c
index 34e7110f310d..69ca337d3220 100644
--- a/drivers/tty/vt/selection.c
+++ b/drivers/tty/vt/selection.c
@@ -116,8 +116,8 @@ static inline int atedge(const int p, int size_row)
return (!(p % size_row) || !((p + 2) % size_row));
 }
 
-/* stores the char in UTF8 and returns the number of bytes used (1-3) */
-static int store_utf8(u16 c, char *p)
+/* stores the char in UTF8 and returns the number of bytes used (1-4) */
+static int store_utf8(u32 c, char *p)
 {
if (c < 0x80) {
/*  0*** */
@@ -128,13 +128,26 @@ static int store_utf8(u16 c, char *p)
p[0] = 0xc0 | (c >> 6);
p[1] = 0x80 | (c & 0x3f);
return 2;
-   } else {
+   } else if (c < 0x1) {
/* 1110 10** 10** */
p[0] = 0xe0 | (c >> 12);
p[1] = 0x80 | ((c >> 6) & 0x3f);
p[2] = 0x80 | (c & 0x3f);
return 3;
-   }
+   } else if (c < 0x11) {
+   /* 0*** 10** 10** 10** */
+   p[0] = 0xf0 | (c >> 18);
+   p[1] = 0x80 | ((c >> 12) & 0x3f);
+   p[2] = 0x80 | ((c >> 6) & 0x3f);
+   p[3] = 0x80 | (c & 0x3f);
+   return 4;
+   } else {
+   /* outside Unicode, replace with U+FFFD */
+   p[0] = 0xef;
+   p[1] = 0xbf;
+   p[2] = 0xbd;
+   return 3;
+   }
 }
 
 /**
@@ -273,7 +286,7 @@ int set_selection(const struct tiocl_selection __user *sel, 
struct tty_struct *t
sel_end = new_sel_end;
 
/* Allocate a new buffer before freeing the old one ... */
-   multiplier = use_unicode ? 3 : 1;  /* chars can take up to 3 bytes */
+   multiplier = use_unicode ? 4 : 1;  /* chars can take up to 4 bytes */
bp = kmalloc_array((sel_end - sel_start) / 2 + 1, multiplier,
   GFP_KERNEL);
if (!bp) {
-- 
2.18.0



[PATCH 1/3] vt: don't reinvent min()

2018-07-17 Thread Adam Borowski
All the helper function saved us was a cast.

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/selection.c | 14 --
 1 file changed, 4 insertions(+), 10 deletions(-)

diff --git a/drivers/tty/vt/selection.c b/drivers/tty/vt/selection.c
index 90ea1cc52b7a..34e7110f310d 100644
--- a/drivers/tty/vt/selection.c
+++ b/drivers/tty/vt/selection.c
@@ -116,12 +116,6 @@ static inline int atedge(const int p, int size_row)
return (!(p % size_row) || !((p + 2) % size_row));
 }
 
-/* constrain v such that v <= u */
-static inline unsigned short limit(const unsigned short v, const unsigned 
short u)
-{
-   return (v > u) ? u : v;
-}
-
 /* stores the char in UTF8 and returns the number of bytes used (1-3) */
 static int store_utf8(u16 c, char *p)
 {
@@ -167,10 +161,10 @@ int set_selection(const struct tiocl_selection __user 
*sel, struct tty_struct *t
if (copy_from_user(, sel, sizeof(*sel)))
return -EFAULT;
 
-   v.xs = limit(v.xs - 1, vc->vc_cols - 1);
-   v.ys = limit(v.ys - 1, vc->vc_rows - 1);
-   v.xe = limit(v.xe - 1, vc->vc_cols - 1);
-   v.ye = limit(v.ye - 1, vc->vc_rows - 1);
+   v.xs = min_t(u16, v.xs - 1, vc->vc_cols - 1);
+   v.ys = min_t(u16, v.ys - 1, vc->vc_rows - 1);
+   v.xe = min_t(u16, v.xe - 1, vc->vc_cols - 1);
+   v.ye = min_t(u16, v.ye - 1, vc->vc_rows - 1);
ps = v.ys * vc->vc_size_row + (v.xs << 1);
pe = v.ye * vc->vc_size_row + (v.xe << 1);
 
-- 
2.18.0

--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 2/3] vt: selection: handle storing of characters above U+FFFF

2018-07-17 Thread Adam Borowski
Those above U+10 get replaced with U+FFFD.

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/selection.c | 23 ++-
 1 file changed, 18 insertions(+), 5 deletions(-)

diff --git a/drivers/tty/vt/selection.c b/drivers/tty/vt/selection.c
index 34e7110f310d..69ca337d3220 100644
--- a/drivers/tty/vt/selection.c
+++ b/drivers/tty/vt/selection.c
@@ -116,8 +116,8 @@ static inline int atedge(const int p, int size_row)
return (!(p % size_row) || !((p + 2) % size_row));
 }
 
-/* stores the char in UTF8 and returns the number of bytes used (1-3) */
-static int store_utf8(u16 c, char *p)
+/* stores the char in UTF8 and returns the number of bytes used (1-4) */
+static int store_utf8(u32 c, char *p)
 {
if (c < 0x80) {
/*  0*** */
@@ -128,13 +128,26 @@ static int store_utf8(u16 c, char *p)
p[0] = 0xc0 | (c >> 6);
p[1] = 0x80 | (c & 0x3f);
return 2;
-   } else {
+   } else if (c < 0x1) {
/* 1110 10** 10** */
p[0] = 0xe0 | (c >> 12);
p[1] = 0x80 | ((c >> 6) & 0x3f);
p[2] = 0x80 | (c & 0x3f);
return 3;
-   }
+   } else if (c < 0x11) {
+   /* 0*** 10** 10** 10** */
+   p[0] = 0xf0 | (c >> 18);
+   p[1] = 0x80 | ((c >> 12) & 0x3f);
+   p[2] = 0x80 | ((c >> 6) & 0x3f);
+   p[3] = 0x80 | (c & 0x3f);
+   return 4;
+   } else {
+   /* outside Unicode, replace with U+FFFD */
+   p[0] = 0xef;
+   p[1] = 0xbf;
+   p[2] = 0xbd;
+   return 3;
+   }
 }
 
 /**
@@ -273,7 +286,7 @@ int set_selection(const struct tiocl_selection __user *sel, 
struct tty_struct *t
sel_end = new_sel_end;
 
/* Allocate a new buffer before freeing the old one ... */
-   multiplier = use_unicode ? 3 : 1;  /* chars can take up to 3 bytes */
+   multiplier = use_unicode ? 4 : 1;  /* chars can take up to 4 bytes */
bp = kmalloc_array((sel_end - sel_start) / 2 + 1, multiplier,
   GFP_KERNEL);
if (!bp) {
-- 
2.18.0



[PATCH 1/3] vt: don't reinvent min()

2018-07-17 Thread Adam Borowski
All the helper function saved us was a cast.

Signed-off-by: Adam Borowski 
---
 drivers/tty/vt/selection.c | 14 --
 1 file changed, 4 insertions(+), 10 deletions(-)

diff --git a/drivers/tty/vt/selection.c b/drivers/tty/vt/selection.c
index 90ea1cc52b7a..34e7110f310d 100644
--- a/drivers/tty/vt/selection.c
+++ b/drivers/tty/vt/selection.c
@@ -116,12 +116,6 @@ static inline int atedge(const int p, int size_row)
return (!(p % size_row) || !((p + 2) % size_row));
 }
 
-/* constrain v such that v <= u */
-static inline unsigned short limit(const unsigned short v, const unsigned 
short u)
-{
-   return (v > u) ? u : v;
-}
-
 /* stores the char in UTF8 and returns the number of bytes used (1-3) */
 static int store_utf8(u16 c, char *p)
 {
@@ -167,10 +161,10 @@ int set_selection(const struct tiocl_selection __user 
*sel, struct tty_struct *t
if (copy_from_user(, sel, sizeof(*sel)))
return -EFAULT;
 
-   v.xs = limit(v.xs - 1, vc->vc_cols - 1);
-   v.ys = limit(v.ys - 1, vc->vc_rows - 1);
-   v.xe = limit(v.xe - 1, vc->vc_cols - 1);
-   v.ye = limit(v.ye - 1, vc->vc_rows - 1);
+   v.xs = min_t(u16, v.xs - 1, vc->vc_cols - 1);
+   v.ys = min_t(u16, v.ys - 1, vc->vc_rows - 1);
+   v.xe = min_t(u16, v.xe - 1, vc->vc_cols - 1);
+   v.ye = min_t(u16, v.ye - 1, vc->vc_rows - 1);
ps = v.ys * vc->vc_size_row + (v.xs << 1);
pe = v.ye * vc->vc_size_row + (v.xe << 1);
 
-- 
2.18.0

--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 0/3] use unicode for vt mouse paste

2018-07-17 Thread Adam Borowski
Hi!
Based on Nicolas' nice work (in tty-next), let's avoid corrupting characters
that have been copy+pasted via mouse selection.  The uniscr array holds
their original identity even if they got mangled by glyph conversion.
The glyph conversion lossily turns similar-looking characters into a
representation, and everyone else into a replacement character.

There's no proper handling for CJK (yet?) but anything of wcwidth()==1 will
work fine.

The whole thing doesn't get enabled until something reads from /dev/vcsu for
that console, but let's test this code first before enabling it widely.


Diffstat:
 drivers/tty/vt/selection.c | 48 
+---
 drivers/tty/vt/vt.c| 10 ++
 include/linux/selection.h  |  1 +
 3 files changed, 40 insertions(+), 19 deletions(-)


-- 
// If you believe in so-called "intellectual property", please immediately
// cease using counterfeit alphabets.  Instead, contact the nearest temple
// of Amon, whose priests will provide you with scribal services for all
// your writing needs, for Reasonable And Non-Discriminatory prices.


[PATCH 0/3] use unicode for vt mouse paste

2018-07-17 Thread Adam Borowski
Hi!
Based on Nicolas' nice work (in tty-next), let's avoid corrupting characters
that have been copy+pasted via mouse selection.  The uniscr array holds
their original identity even if they got mangled by glyph conversion.
The glyph conversion lossily turns similar-looking characters into a
representation, and everyone else into a replacement character.

There's no proper handling for CJK (yet?) but anything of wcwidth()==1 will
work fine.

The whole thing doesn't get enabled until something reads from /dev/vcsu for
that console, but let's test this code first before enabling it widely.


Diffstat:
 drivers/tty/vt/selection.c | 48 
+---
 drivers/tty/vt/vt.c| 10 ++
 include/linux/selection.h  |  1 +
 3 files changed, 40 insertions(+), 19 deletions(-)


-- 
// If you believe in so-called "intellectual property", please immediately
// cease using counterfeit alphabets.  Instead, contact the nearest temple
// of Amon, whose priests will provide you with scribal services for all
// your writing needs, for Reasonable And Non-Discriminatory prices.


Re: [PATCH 1/3] vt: avoid a VLA in the unicode screen scroll function

2018-07-17 Thread Adam Borowski
On Tue, Jul 17, 2018 at 09:02:40PM -0400, Nicolas Pitre wrote:
> The nr argument is typically small: most often nr == 1. However this
> could be abused with a very large explicit scroll in a resized screen.
> Make the code scroll lines one at a time in all cases to avoid the VLA.
> Anything smarter is most likely not warranted here.

Even though nr can be 32767 at most, your new version is O(nr*nr) for no
reason.  Instead of O(n) memory or O(n²) time, a variant of the original
that copies values one at a time would be shorter and faster.

> Requested-by: Kees Cook 
> Signed-off-by: Nicolas Pitre 
> ---
>  drivers/tty/vt/vt.c | 18 ++
>  1 file changed, 10 insertions(+), 8 deletions(-)
> 
> diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
> index 2d14bb195d..03e79f7787 100644
> --- a/drivers/tty/vt/vt.c
> +++ b/drivers/tty/vt/vt.c
> @@ -433,20 +433,22 @@ static void vc_uniscr_scroll(struct vc_data *vc, 
> unsigned int t, unsigned int b,
>  
>   if (uniscr) {
>   unsigned int s, d, rescue, clear;
> - char32_t *save[nr];
>  
>   s = clear = t;
> - d = t + nr;
> - rescue = b - nr;
> + d = t + 1;
> + rescue = b - 1;
>   if (dir == SM_UP) {
>   swap(s, d);
>   swap(clear, rescue);
>   }
> - memcpy(save, uniscr->lines + rescue, nr * sizeof(*save));
> - memmove(uniscr->lines + d, uniscr->lines + s,
> - (b - t - nr) * sizeof(*uniscr->lines));
> - memcpy(uniscr->lines + clear, save, nr * sizeof(*save));
> - vc_uniscr_clear_lines(vc, clear, nr);
> + while (nr--) {
> + char32_t *tmp;
> + tmp = uniscr->lines[rescue];
> + memmove(uniscr->lines + d, uniscr->lines + s,
> + (b - t - 1) * sizeof(*uniscr->lines));
> + uniscr->lines[clear] = tmp;
> + vc_uniscr_clear_lines(vc, clear, 1);
> + }
>   }
>  }

What the function does is rotating an array (slice [t..b) here), by nr if
SM_DOWN or by -nr ie (b - t - nr) if SM_UP.  A nice problem that almost every
"code interview questions" book includes :)

Please say if you don't have time for such games, I've just refreshed what's
a good answer. :þ


Meow.
-- 
// If you believe in so-called "intellectual property", please immediately
// cease using counterfeit alphabets.  Instead, contact the nearest temple
// of Amon, whose priests will provide you with scribal services for all
// your writing needs, for Reasonable And Non-Discriminatory prices.
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 1/3] vt: avoid a VLA in the unicode screen scroll function

2018-07-17 Thread Adam Borowski
On Tue, Jul 17, 2018 at 09:02:40PM -0400, Nicolas Pitre wrote:
> The nr argument is typically small: most often nr == 1. However this
> could be abused with a very large explicit scroll in a resized screen.
> Make the code scroll lines one at a time in all cases to avoid the VLA.
> Anything smarter is most likely not warranted here.

Even though nr can be 32767 at most, your new version is O(nr*nr) for no
reason.  Instead of O(n) memory or O(n²) time, a variant of the original
that copies values one at a time would be shorter and faster.

> Requested-by: Kees Cook 
> Signed-off-by: Nicolas Pitre 
> ---
>  drivers/tty/vt/vt.c | 18 ++
>  1 file changed, 10 insertions(+), 8 deletions(-)
> 
> diff --git a/drivers/tty/vt/vt.c b/drivers/tty/vt/vt.c
> index 2d14bb195d..03e79f7787 100644
> --- a/drivers/tty/vt/vt.c
> +++ b/drivers/tty/vt/vt.c
> @@ -433,20 +433,22 @@ static void vc_uniscr_scroll(struct vc_data *vc, 
> unsigned int t, unsigned int b,
>  
>   if (uniscr) {
>   unsigned int s, d, rescue, clear;
> - char32_t *save[nr];
>  
>   s = clear = t;
> - d = t + nr;
> - rescue = b - nr;
> + d = t + 1;
> + rescue = b - 1;
>   if (dir == SM_UP) {
>   swap(s, d);
>   swap(clear, rescue);
>   }
> - memcpy(save, uniscr->lines + rescue, nr * sizeof(*save));
> - memmove(uniscr->lines + d, uniscr->lines + s,
> - (b - t - nr) * sizeof(*uniscr->lines));
> - memcpy(uniscr->lines + clear, save, nr * sizeof(*save));
> - vc_uniscr_clear_lines(vc, clear, nr);
> + while (nr--) {
> + char32_t *tmp;
> + tmp = uniscr->lines[rescue];
> + memmove(uniscr->lines + d, uniscr->lines + s,
> + (b - t - 1) * sizeof(*uniscr->lines));
> + uniscr->lines[clear] = tmp;
> + vc_uniscr_clear_lines(vc, clear, 1);
> + }
>   }
>  }

What the function does is rotating an array (slice [t..b) here), by nr if
SM_DOWN or by -nr ie (b - t - nr) if SM_UP.  A nice problem that almost every
"code interview questions" book includes :)

Please say if you don't have time for such games, I've just refreshed what's
a good answer. :þ


Meow.
-- 
// If you believe in so-called "intellectual property", please immediately
// cease using counterfeit alphabets.  Instead, contact the nearest temple
// of Amon, whose priests will provide you with scribal services for all
// your writing needs, for Reasonable And Non-Discriminatory prices.
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH] .gitignore: add ZSTD-compressed files

2018-07-13 Thread Adam Borowski
For now, that's arch/x86/boot/compressed/vmlinux.bin.zst but probably more
will come, thus let's be consistent with all other compressors.

Signed-off-by: Adam Borowski 
---
 .gitignore | 1 +
 1 file changed, 1 insertion(+)

diff --git a/.gitignore b/.gitignore
index 97ba6b79834c..0d09cf1c053c 100644
--- a/.gitignore
+++ b/.gitignore
@@ -42,6 +42,7 @@
 *.tab.[ch]
 *.tar
 *.xz
+*.zst
 Module.symvers
 modules.builtin
 
-- 
2.18.0



[PATCH] .gitignore: add ZSTD-compressed files

2018-07-13 Thread Adam Borowski
For now, that's arch/x86/boot/compressed/vmlinux.bin.zst but probably more
will come, thus let's be consistent with all other compressors.

Signed-off-by: Adam Borowski 
---
 .gitignore | 1 +
 1 file changed, 1 insertion(+)

diff --git a/.gitignore b/.gitignore
index 97ba6b79834c..0d09cf1c053c 100644
--- a/.gitignore
+++ b/.gitignore
@@ -42,6 +42,7 @@
 *.tab.[ch]
 *.tar
 *.xz
+*.zst
 Module.symvers
 modules.builtin
 
-- 
2.18.0



Re: [PATCH v3 1/3] vt: preserve unicode values corresponding to screen characters

2018-07-11 Thread Adam Borowski
On Wed, Jul 11, 2018 at 01:39:56PM -0700, Kees Cook wrote:
> On Wed, Jul 11, 2018 at 2:18 AM, Greg Kroah-Hartman
>  wrote:
> > On Tue, Jul 10, 2018 at 05:52:01PM -0700, Kees Cook wrote:
> >> On Tue, Jun 26, 2018 at 8:56 PM, Nicolas Pitre  
> >> wrote:
> >> > +++ b/drivers/tty/vt/vt.c
> >> > [...]
> >> > +static void vc_uniscr_scroll(struct vc_data *vc, unsigned int t, 
> >> > unsigned int b,
> >> > +enum con_scroll dir, unsigned int nr)
> >> > +{
> >> > +   struct uni_screen *uniscr = get_vc_uniscr(vc);
> >> > +
> >> > +   if (uniscr) {
> >> > +   unsigned int s, d, rescue, clear;
> >> > +   char32_t *save[nr];
> >>
> >> Can you adjust this to avoid the VLA here? I've almost gotten all VLAs
> >> removed from the kernel[1], and this is introducing a new one. :)
> >
> Yup, that's fine. (It's how I noticed it: linux-next VLA build tests.)
> I'm just hoping it can get solved before the merge window opens. :)

This one is actually nasty: max console height is 32767, if you resize it
that big then issue a large scroll request, boom it goes.

> There are still a bunch of VLAs I'm chipping away at, but this was a
> newly added one, so I was hoping Nicolas (when he's back from
> vacation) will have ideas on how to best avoid it.

Nicolas: what about just moving line pointers one at a time?  Rotating an
array slice in-place isn't that slow -- and optimizing that much when
reasonable sizes don't exceed 100ish (depending on how good your eyes are)
is quite ridiculous.  Thanks to your change, we don't need to move actual
contents, just line pointers -- that's fast enough.


Meow!
-- 
// If you believe in so-called "intellectual property", please immediately
// cease using counterfeit alphabets.  Instead, contact the nearest temple
// of Amon, whose priests will provide you with scribal services for all
// your writing needs, for Reasonable And Non-Discriminatory prices.
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH v3 1/3] vt: preserve unicode values corresponding to screen characters

2018-07-11 Thread Adam Borowski
On Wed, Jul 11, 2018 at 01:39:56PM -0700, Kees Cook wrote:
> On Wed, Jul 11, 2018 at 2:18 AM, Greg Kroah-Hartman
>  wrote:
> > On Tue, Jul 10, 2018 at 05:52:01PM -0700, Kees Cook wrote:
> >> On Tue, Jun 26, 2018 at 8:56 PM, Nicolas Pitre  
> >> wrote:
> >> > +++ b/drivers/tty/vt/vt.c
> >> > [...]
> >> > +static void vc_uniscr_scroll(struct vc_data *vc, unsigned int t, 
> >> > unsigned int b,
> >> > +enum con_scroll dir, unsigned int nr)
> >> > +{
> >> > +   struct uni_screen *uniscr = get_vc_uniscr(vc);
> >> > +
> >> > +   if (uniscr) {
> >> > +   unsigned int s, d, rescue, clear;
> >> > +   char32_t *save[nr];
> >>
> >> Can you adjust this to avoid the VLA here? I've almost gotten all VLAs
> >> removed from the kernel[1], and this is introducing a new one. :)
> >
> Yup, that's fine. (It's how I noticed it: linux-next VLA build tests.)
> I'm just hoping it can get solved before the merge window opens. :)

This one is actually nasty: max console height is 32767, if you resize it
that big then issue a large scroll request, boom it goes.

> There are still a bunch of VLAs I'm chipping away at, but this was a
> newly added one, so I was hoping Nicolas (when he's back from
> vacation) will have ideas on how to best avoid it.

Nicolas: what about just moving line pointers one at a time?  Rotating an
array slice in-place isn't that slow -- and optimizing that much when
reasonable sizes don't exceed 100ish (depending on how good your eyes are)
is quite ridiculous.  Thanks to your change, we don't need to move actual
contents, just line pointers -- that's fast enough.


Meow!
-- 
// If you believe in so-called "intellectual property", please immediately
// cease using counterfeit alphabets.  Instead, contact the nearest temple
// of Amon, whose priests will provide you with scribal services for all
// your writing needs, for Reasonable And Non-Discriminatory prices.
--
To unsubscribe from this list: send the line "unsubscribe linux-console" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH resend] scripts: teach extract-vmlinux about LZ4 and ZSTD

2018-07-06 Thread Adam Borowski
Note that the LZ4 signature is different than that of modern LZ4 as we
use the "legacy" format which suffers from some downsides like inability
to disable compression.

Signed-off-by: Adam Borowski 
---
The first time this was sent I managed to screw up both the subject and
scissors line, so it's likely whatever filter you're using didn't notice
the patch inside a thread.

ZSTD compression for vmlinux is not yet in, but being able to unpack it
can't hurt.  LZ4 is in for quite a while.


 scripts/extract-vmlinux | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/scripts/extract-vmlinux b/scripts/extract-vmlinux
index 5061abcc2540..e6239f39abad 100755
--- a/scripts/extract-vmlinux
+++ b/scripts/extract-vmlinux
@@ -57,6 +57,8 @@ try_decompress '\3757zXZ\000' abcde unxz
 try_decompress 'BZh'  xybunzip2
 try_decompress '\135\0\0\0'   xxx   unlzma
 try_decompress '\211\114\132' xy'lzop -d'
+try_decompress '\002!L\030'   xxx   'lz4 -d'
+try_decompress '(\265/\375'   xxx   unzstd
 
 # Bail out:
 echo "$me: Cannot find vmlinux." >&2
-- 
2.18.0



  1   2   3   4   5   >