Re: [PATCH] Unexport LANG env variable

[Michal Marek]

Dne 14.6.2017 v 14:40 Alexey Brodkin napsal(a):
> In those cases when we parse output of standard utilities like readelf
> etc we rely on a particular sentences. For example for ARC we extract
> an entry-point from vmlinux like that:
> -->8
> readelf -h vmlinux | grep "Entry point address" | grep -o 0x.*
> -->8
> 
> And in case LANG is set to anything other than en_XX we're getting
> nothing and subsequent execution of mkimage utility fails.
> 
> Probably there're more cases like that but given people rarely
> use non-English locales on their dev machines problems like the one
> above are not very visible.

I'm all for this change but the *.po files in the gcc tree must have
been created for a reason:

  https://github.com/gcc-mirror/gcc/tree/master/gcc/po

:)

Michal

Re: [PATCH] Unexport LANG env variable

[Michal Marek]

Dne 14.6.2017 v 14:40 Alexey Brodkin napsal(a):
> In those cases when we parse output of standard utilities like readelf
> etc we rely on a particular sentences. For example for ARC we extract
> an entry-point from vmlinux like that:
> -->8
> readelf -h vmlinux | grep "Entry point address" | grep -o 0x.*
> -->8
> 
> And in case LANG is set to anything other than en_XX we're getting
> nothing and subsequent execution of mkimage utility fails.
> 
> Probably there're more cases like that but given people rarely
> use non-English locales on their dev machines problems like the one
> above are not very visible.

I'm all for this change but the *.po files in the gcc tree must have
been created for a reason:

  https://github.com/gcc-mirror/gcc/tree/master/gcc/po

:)

Michal

Re: [PATCH v2 1/4] kbuild: Add cc-option-raw macro

[Michal Marek]

Dne 14.6.2017 v 09:31 Arnd Bergmann napsal(a):
> On Wed, Jun 14, 2017 at 3:27 AM, Masahiro Yamada
> <yamada.masah...@socionext.com> wrote:
>> 2017-06-14 8:08 GMT+09:00 Matthias Kaehlcke <m...@chromium.org>:
>>> cc-option uses KBUILD_CFLAGS and KBUILD_CPPFLAGS when it determines
>>> whether an option is supported or not. This is fine for options used to
>>> build the kernel itself, however some components like the x86 boot code
>>> use a different set of flags.
>>>
>>> Add the new macro cc-option-raw which serves the same purpose as
>>> cc-option but has additional parameters. One parameter is the compiler
>>> with which the check should be performed, the other the compiler options
>>> to be used instead KBUILD_C*FLAGS. The compiler parameter allows other
>>> macros like hostcc-option to be implemented on top of cc-option-raw.
>>>
>>> Also rework cc-option to make use of cc-option-raw.
>>>
>>> Suggested-by: Arnd Bergmann <a...@arndb.de>
>>> Suggested-by: Masahiro Yamada <yamada.masah...@socionext.com>
>>> Signed-off-by: Matthias Kaehlcke <m...@chromium.org>
>>> ---
>>> Changes in v2:
>>> - Changed macro name from Add cc-option-no-kbuild to cc-option-raw
>>> - Added compiler as parameter to the macro
>>> - Reworked cc-option to make use of cc-option-raw
>>> - Updated commit message
>>>
>>>  scripts/Kbuild.include | 9 +++--
>>>  1 file changed, 7 insertions(+), 2 deletions(-)
>>>
>>
>> Acked-by: Masahiro Yamada <yamada.masah...@socionext.com>
>> (if nothing better pops up)
> 
> Acked-by: Arnd Bergmann <a...@arndb.de>

Acked-by: Michal Marek <mma...@suse.com>


> Regarding the naming, __cc-option might be better than cc-option-raw,
> but the current version is fine too.

I have no strong opinion either way :).

Michal

Re: [PATCH v2 1/4] kbuild: Add cc-option-raw macro

[Michal Marek]

Dne 14.6.2017 v 09:31 Arnd Bergmann napsal(a):
> On Wed, Jun 14, 2017 at 3:27 AM, Masahiro Yamada
>  wrote:
>> 2017-06-14 8:08 GMT+09:00 Matthias Kaehlcke :
>>> cc-option uses KBUILD_CFLAGS and KBUILD_CPPFLAGS when it determines
>>> whether an option is supported or not. This is fine for options used to
>>> build the kernel itself, however some components like the x86 boot code
>>> use a different set of flags.
>>>
>>> Add the new macro cc-option-raw which serves the same purpose as
>>> cc-option but has additional parameters. One parameter is the compiler
>>> with which the check should be performed, the other the compiler options
>>> to be used instead KBUILD_C*FLAGS. The compiler parameter allows other
>>> macros like hostcc-option to be implemented on top of cc-option-raw.
>>>
>>> Also rework cc-option to make use of cc-option-raw.
>>>
>>> Suggested-by: Arnd Bergmann 
>>> Suggested-by: Masahiro Yamada 
>>> Signed-off-by: Matthias Kaehlcke 
>>> ---
>>> Changes in v2:
>>> - Changed macro name from Add cc-option-no-kbuild to cc-option-raw
>>> - Added compiler as parameter to the macro
>>> - Reworked cc-option to make use of cc-option-raw
>>> - Updated commit message
>>>
>>>  scripts/Kbuild.include | 9 +++--
>>>  1 file changed, 7 insertions(+), 2 deletions(-)
>>>
>>
>> Acked-by: Masahiro Yamada 
>> (if nothing better pops up)
> 
> Acked-by: Arnd Bergmann 

Acked-by: Michal Marek 


> Regarding the naming, __cc-option might be better than cc-option-raw,
> but the current version is fine too.

I have no strong opinion either way :).

Michal

Re: [PATCH v2] Kbuild: tiny correction on `make help`

[Michal Marek]

Dne 14.6.2017 v 09:08 Cao jin napsal(a):
> 
> 
> On 06/14/2017 09:15 AM, Masahiro Yamada wrote:
>> 2017-06-13 18:52 GMT+09:00 Cao jin :
>>> Yamada-san,
>>>
>>>   I have a question on the current top level Makefile. There are
>>> following lines, I have trouble to understand "firstword x$(MAKEFLAGS)"
>>> in 2nd line, why there is a 'x' there?
>>>
>>>
>>>   ifneq ($(filter 4.%,$(MAKE_VERSION)),)  # make-4
>>>   ifneq ($(filter %s ,$(firstword x$(MAKEFLAGS))),)
>>
>>
>> I do not understand the 'x' either.
>>
>> Michal suggested the code according to the following:
>> https://patchwork.kernel.org/patch/3291301/
>> (I hope Michal can help you.)
>>
> 
> It does helps.
> 
>>
>> My first guess was, it was added just in case $(MAKEFLAGS) is empty.
>> Actually $(firstword ) seems to work with empty argument, though.
>>
> 
> I see, I guess you are saying it was added in case there is no
> single-letter flags in $(MAKEFLAGS) while it has "--debug=jobs", that
> make sense to me. Thanks very much, Yamada-san.

Well, it more than three years ago. But yes, I believe that was the
intention:

make --foo=s

x$(MAKEFLAGS) => x --foo=s
$(firstword x$(MAKEFLAGS)) => x

Michal

Re: [PATCH v2] Kbuild: tiny correction on `make help`

[Michal Marek]

Dne 14.6.2017 v 09:08 Cao jin napsal(a):
> 
> 
> On 06/14/2017 09:15 AM, Masahiro Yamada wrote:
>> 2017-06-13 18:52 GMT+09:00 Cao jin :
>>> Yamada-san,
>>>
>>>   I have a question on the current top level Makefile. There are
>>> following lines, I have trouble to understand "firstword x$(MAKEFLAGS)"
>>> in 2nd line, why there is a 'x' there?
>>>
>>>
>>>   ifneq ($(filter 4.%,$(MAKE_VERSION)),)  # make-4
>>>   ifneq ($(filter %s ,$(firstword x$(MAKEFLAGS))),)
>>
>>
>> I do not understand the 'x' either.
>>
>> Michal suggested the code according to the following:
>> https://patchwork.kernel.org/patch/3291301/
>> (I hope Michal can help you.)
>>
> 
> It does helps.
> 
>>
>> My first guess was, it was added just in case $(MAKEFLAGS) is empty.
>> Actually $(firstword ) seems to work with empty argument, though.
>>
> 
> I see, I guess you are saying it was added in case there is no
> single-letter flags in $(MAKEFLAGS) while it has "--debug=jobs", that
> make sense to me. Thanks very much, Yamada-san.

Well, it more than three years ago. But yes, I believe that was the
intention:

make --foo=s

x$(MAKEFLAGS) => x --foo=s
$(firstword x$(MAKEFLAGS)) => x

Michal

Re: [PATCH 1/3] kbuild: Add cc-option-no-kbuild macro

[Michal Marek]

On 2017-06-13 02:55, Matthias Kaehlcke wrote:
> cc-option uses KBUILD_CFLAGS and KBUILD_CPPFLAGS when it determines
> whether an option is supported or not. This is fine for options used to
> build the kernel itself, however some components like the x86 boot code
> use a different set of flags.
> 
> Add the new macro cc-option-no-kbuild which does the same as cc-option
> except that it has an additional parameter with the compiler options
> which are used instead of KBUILD_CFLAGS and KBUILD_CPPFLAGS.
> 
> Signed-off-by: Matthias Kaehlcke 
> ---
>  scripts/Kbuild.include | 5 +
>  1 file changed, 5 insertions(+)
> 
> diff --git a/scripts/Kbuild.include b/scripts/Kbuild.include
> index 61f87a99bf0a..d9fdc740105f 100644
> --- a/scripts/Kbuild.include
> +++ b/scripts/Kbuild.include
> @@ -128,6 +128,11 @@ cc-option-yn = $(call try-run,\
>  cc-option-align = $(subst -functions=0,,\
>   $(call cc-option,-falign-functions=0,-malign-functions=0))
>  
> +# cc-option-no-kbuild
> +# Usage: cflags-no-kbuild-y += $(call cc-option-no-kbuild, flags>,-march=winchip-c6,-march=i586)
> +cc-option-no-kbuild = $(call try-run,\
> + $(CC) -Werror $(filter-out $(GCC_PLUGINS_CFLAGS),$(1)) $(2) -c -x c 
> /dev/null -o "$$TMP",$(2),$(3))

As this is a version of cc-option with an extrra argument, how about
implementing cc-option as a shorthand for cc-option-no-kbuild? It would
make it more obvious what cc-option-no-kbuild does differently (it's
probably just me, but I was unable to infer the semantics from its name).

Michal

Re: [PATCH 1/3] kbuild: Add cc-option-no-kbuild macro

[Michal Marek]

On 2017-06-13 02:55, Matthias Kaehlcke wrote:
> cc-option uses KBUILD_CFLAGS and KBUILD_CPPFLAGS when it determines
> whether an option is supported or not. This is fine for options used to
> build the kernel itself, however some components like the x86 boot code
> use a different set of flags.
> 
> Add the new macro cc-option-no-kbuild which does the same as cc-option
> except that it has an additional parameter with the compiler options
> which are used instead of KBUILD_CFLAGS and KBUILD_CPPFLAGS.
> 
> Signed-off-by: Matthias Kaehlcke 
> ---
>  scripts/Kbuild.include | 5 +
>  1 file changed, 5 insertions(+)
> 
> diff --git a/scripts/Kbuild.include b/scripts/Kbuild.include
> index 61f87a99bf0a..d9fdc740105f 100644
> --- a/scripts/Kbuild.include
> +++ b/scripts/Kbuild.include
> @@ -128,6 +128,11 @@ cc-option-yn = $(call try-run,\
>  cc-option-align = $(subst -functions=0,,\
>   $(call cc-option,-falign-functions=0,-malign-functions=0))
>  
> +# cc-option-no-kbuild
> +# Usage: cflags-no-kbuild-y += $(call cc-option-no-kbuild, flags>,-march=winchip-c6,-march=i586)
> +cc-option-no-kbuild = $(call try-run,\
> + $(CC) -Werror $(filter-out $(GCC_PLUGINS_CFLAGS),$(1)) $(2) -c -x c 
> /dev/null -o "$$TMP",$(2),$(3))

As this is a version of cc-option with an extrra argument, how about
implementing cc-option as a shorthand for cc-option-no-kbuild? It would
make it more obvious what cc-option-no-kbuild does differently (it's
probably just me, but I was unable to infer the semantics from its name).

Michal

Re: [PATCH] kbuild: simplify silent build (-s) detection

[Michal Marek]

On 2017-05-19 06:28, Masahiro Yamada wrote:
> This allows to detect -s option without checking GNU Make version.
> 
> As commit e36aaea28972 ("kbuild: Fix silent builds with make-4")
> pointed out, GNU Make 4.x changed the way/order it presents the
> command line options into MAKEFLAGS.
> 
> In Make 3.8x, 's' is always be the first in a group of short options.
> The group could be prefixed with '-'.
> 
> In Make 4.x, 's' is always the last in a group of short options.
> 
> As commit e6ac89fabd03 ("kbuild: Correctly deal with make options
> which contain an 's'") addressed, we also need to deal with long
> options that end with 's', like --warn-undefined-variables.
> 
> Test cases:
> 
> [1] command line input:make --silent
>  -> MAKEFLAGS for Make 3.8x:s
>  -> MAKEFLAGS for Make 4.x :s
> 
> [2] command line input:make -srR
>  -> MAKEFLAGS for Make 3.8x:sRr
>  -> MAKEFLAGS for Make 4.x :rRs
> 
> [3] command line input:make -s -rR --warn-undefined-variables
>  -> MAKEFLAGS for Make 3.8x:--warn-undefined-variables -sRr
>  -> MAKEFLAGS for Make 4.x :rRs --warn-undefined-variables
> 
> We can take care of them, by filtering out long options (--%),
> then matching -s% s% %s patterns.

Good idea, that's a much cleaner way.

Michal

Re: [PATCH] kbuild: simplify silent build (-s) detection

[Michal Marek]

On 2017-05-19 06:28, Masahiro Yamada wrote:
> This allows to detect -s option without checking GNU Make version.
> 
> As commit e36aaea28972 ("kbuild: Fix silent builds with make-4")
> pointed out, GNU Make 4.x changed the way/order it presents the
> command line options into MAKEFLAGS.
> 
> In Make 3.8x, 's' is always be the first in a group of short options.
> The group could be prefixed with '-'.
> 
> In Make 4.x, 's' is always the last in a group of short options.
> 
> As commit e6ac89fabd03 ("kbuild: Correctly deal with make options
> which contain an 's'") addressed, we also need to deal with long
> options that end with 's', like --warn-undefined-variables.
> 
> Test cases:
> 
> [1] command line input:make --silent
>  -> MAKEFLAGS for Make 3.8x:s
>  -> MAKEFLAGS for Make 4.x :s
> 
> [2] command line input:make -srR
>  -> MAKEFLAGS for Make 3.8x:sRr
>  -> MAKEFLAGS for Make 4.x :rRs
> 
> [3] command line input:make -s -rR --warn-undefined-variables
>  -> MAKEFLAGS for Make 3.8x:--warn-undefined-variables -sRr
>  -> MAKEFLAGS for Make 4.x :rRs --warn-undefined-variables
> 
> We can take care of them, by filtering out long options (--%),
> then matching -s% s% %s patterns.

Good idea, that's a much cleaner way.

Michal

Re: [RFC] Increase Minimal GNU Make version for Linux Kernel from 3.80 to 3.81

[Michal Marek]

On 2017-05-03 08:46, Masahiro Yamada wrote:
> Hello Linus and Kbuild developers.
> 
> 
> Documentation/process/changes.rst says the minimal version
> of GNU Make is 3.80, but actually building the kernel
> with this version has been broken for a long time.
> 
> Specifically, it got broken by commit c8589d1e9e01 (i.e. Linux 3.18).
> Sorry, it's me who broke it.
> 
> Here is my excuse:
> - It is almost 3 years since then, but nobody complained about it.
> - GNU Make 3.80 is almost 15 years old.
>   (Even GNU Make 3.81 was released in 2006.)
> - People seldom test their makefiles on such old GNU Make version,
>   so they often use some features that are not supported by version 3.80.

Agreed. It's not just the kbuild change that broke it, but as you say,
new make features tend creep into random Makefiles. So I'm fine
adjusting the documentation to match reality.

Adding Jan Beulich to Cc, who to fix these cases in the past.

Michal

Re: [RFC] Increase Minimal GNU Make version for Linux Kernel from 3.80 to 3.81

[Michal Marek]

On 2017-05-03 08:46, Masahiro Yamada wrote:
> Hello Linus and Kbuild developers.
> 
> 
> Documentation/process/changes.rst says the minimal version
> of GNU Make is 3.80, but actually building the kernel
> with this version has been broken for a long time.
> 
> Specifically, it got broken by commit c8589d1e9e01 (i.e. Linux 3.18).
> Sorry, it's me who broke it.
> 
> Here is my excuse:
> - It is almost 3 years since then, but nobody complained about it.
> - GNU Make 3.80 is almost 15 years old.
>   (Even GNU Make 3.81 was released in 2006.)
> - People seldom test their makefiles on such old GNU Make version,
>   so they often use some features that are not supported by version 3.80.

Agreed. It's not just the kbuild change that broke it, but as you say,
new make features tend creep into random Makefiles. So I'm fine
adjusting the documentation to match reality.

Adding Jan Beulich to Cc, who to fix these cases in the past.

Michal

Re: [RFC-b PATCH] kbuild: redefine __FILE__ as relative path from $(srctree) if possible

[Michal Marek]

Dne 21.4.2017 v 21:36 Masahiro Yamada napsal(a):
> Since Kbuild runs in the objtree, __FILE__ can be a very long path
> depending of $(srctree).
> 
> Commit 9da0763bdd82 ("kbuild: Use relative path when building in a
> subdir of the source tree") made the situation better for cases
> where objtree is a child of srctree.  ($(srctree) is "..")
> 
> For other cases of out-of-tree build, filenames in WARN_ON() etc.
> are still an absolute path.  It also means the kernel image depends
> on where it was built.
> 
> Here, the idea is to redefine __FILE__ as the relative path from
> $(srctree), but doing so causes a compiler warning:
>   warning: "__FILE__" redefined [-Wbuiltin-macro-redefined]
> 
> We can suppress it with -Wno-builtin-macro-redefined.  However,
> this option is not recognized by old compilers.  So, __FILE__
> is re-defined only when the option is supported.

I actually like the KBUILD_FILE approach more, even if it means patching
various users.


> Please note __FILE__ is always an absolute path for external
> modules.  We can strip KBUILD_EXTMOD from the path if we want,
> but I am not doing that.  It would make it difficult to figure out
> the module in question in case of WARN_ON().

Agreed. OOT modules are potential source of trouble, so let them stand out.

Michal

Re: [RFC-b PATCH] kbuild: redefine __FILE__ as relative path from $(srctree) if possible

[Michal Marek]

Dne 21.4.2017 v 21:36 Masahiro Yamada napsal(a):
> Since Kbuild runs in the objtree, __FILE__ can be a very long path
> depending of $(srctree).
> 
> Commit 9da0763bdd82 ("kbuild: Use relative path when building in a
> subdir of the source tree") made the situation better for cases
> where objtree is a child of srctree.  ($(srctree) is "..")
> 
> For other cases of out-of-tree build, filenames in WARN_ON() etc.
> are still an absolute path.  It also means the kernel image depends
> on where it was built.
> 
> Here, the idea is to redefine __FILE__ as the relative path from
> $(srctree), but doing so causes a compiler warning:
>   warning: "__FILE__" redefined [-Wbuiltin-macro-redefined]
> 
> We can suppress it with -Wno-builtin-macro-redefined.  However,
> this option is not recognized by old compilers.  So, __FILE__
> is re-defined only when the option is supported.

I actually like the KBUILD_FILE approach more, even if it means patching
various users.


> Please note __FILE__ is always an absolute path for external
> modules.  We can strip KBUILD_EXTMOD from the path if we want,
> but I am not doing that.  It would make it difficult to figure out
> the module in question in case of WARN_ON().

Agreed. OOT modules are potential source of trouble, so let them stand out.

Michal

Re: [PATCH 0/5] Fix dead URLs to ftp.kernel.org

[Michal Marek]

Dne 27.3.2017 v 07:47 SeongJae Park napsal(a):
> URLs to `ftp.kernel.org` exist here and there though `ftp.kernel.org` is
> already dead [0].  This patchset fixes those URLs to use `www.kernel.org`
> instead.
> 
> The change is splitted into multiple patches for independent review and merge
> of each maintainer, though the change is trivial.
> 
> [0] https://www.kernel.org/shutting-down-ftp-services.html
> 
> SeongJae Park (5):
>   MAINTAINERS: Fix a dead URL to ftp.kernel.org
>   drivers/block: Fix a dead URL to ftp.kernel.org
>   drivers/md: Fix a dead URL to ftp.kernel.org
>   fs/autofs4: Fix a dead URL to ftp.kernel.org
>   scripts: Fix dead URLs to ftp.kernel.org

I guess the change be sent in a single patch to the trivial tree.

Michal

Re: [PATCH 0/5] Fix dead URLs to ftp.kernel.org

[Michal Marek]

Dne 27.3.2017 v 07:47 SeongJae Park napsal(a):
> URLs to `ftp.kernel.org` exist here and there though `ftp.kernel.org` is
> already dead [0].  This patchset fixes those URLs to use `www.kernel.org`
> instead.
> 
> The change is splitted into multiple patches for independent review and merge
> of each maintainer, though the change is trivial.
> 
> [0] https://www.kernel.org/shutting-down-ftp-services.html
> 
> SeongJae Park (5):
>   MAINTAINERS: Fix a dead URL to ftp.kernel.org
>   drivers/block: Fix a dead URL to ftp.kernel.org
>   drivers/md: Fix a dead URL to ftp.kernel.org
>   fs/autofs4: Fix a dead URL to ftp.kernel.org
>   scripts: Fix dead URLs to ftp.kernel.org

I guess the change be sent in a single patch to the trivial tree.

Michal

Re: [PATCH] x86/boot: Support uncompressed kernel

[Michal Marek]

Dne 27.3.2017 v 09:58 Sebastian Andrzej Siewior napsal(a):
> On 2017-03-24 13:35:40 [+0800], Chao Peng wrote:
>>
> kernel   kernel sizetime in decompress_kernel
> compressed (gzip)3.3M   53ms
> uncompressed 14M3ms

>> Exactly, LZ4 is the fastest. It takes 16ms to complete the
>> decompression. Still sounds a little longer when compared to
>> uncompressed kernel.
> 
> Are we seriously talking here about one-time improvement of 13ms
> boot time?

If the use case is launching new VM instances continuously, then
compressing the kernel image is about as useful as compressing /bin/bash.

Michal

Re: [PATCH] x86/boot: Support uncompressed kernel

[Michal Marek]

Dne 27.3.2017 v 09:58 Sebastian Andrzej Siewior napsal(a):
> On 2017-03-24 13:35:40 [+0800], Chao Peng wrote:
>>
> kernel   kernel sizetime in decompress_kernel
> compressed (gzip)3.3M   53ms
> uncompressed 14M3ms

>> Exactly, LZ4 is the fastest. It takes 16ms to complete the
>> decompression. Still sounds a little longer when compared to
>> uncompressed kernel.
> 
> Are we seriously talking here about one-time improvement of 13ms
> boot time?

If the use case is launching new VM instances continuously, then
compressing the kernel image is about as useful as compressing /bin/bash.

Michal

Re: [PATCH] x86/boot: Support uncompressed kernel

[Michal Marek]

On 2017-03-23 13:51, Chao Peng wrote:
> Compressed kernel has its own drawback: uncompressing takes time. Even
> though the time is short enough to ignore for most cases but for cases that
> time is critical this is still a big number. In our on-going optimization
> for kernel boot time, the measured overall kernel boot time is ~90ms while
> the uncompressing takes ~50ms with gzip.
> 
> The patch adds a 'CONFIG_KERNEL_RAW' configure choice so the built binary
> can have no uncompressing at all. The experiment shows:
> 
> kernel   kernel sizetime in decompress_kernel
> compressed (gzip)3.3M   53ms
> uncompressed 14M3ms
> 
> Signed-off-by: Chao Peng 
> ---
>  arch/x86/boot/compressed/Makefile |  3 +++
>  arch/x86/boot/compressed/misc.c   | 14 ++
>  init/Kconfig  |  7 +++
>  scripts/Makefile.lib  |  8 
>  4 files changed, 32 insertions(+)
> 
> diff --git a/arch/x86/boot/compressed/Makefile 
> b/arch/x86/boot/compressed/Makefile
> index f9ce75d..fc0e1c0 100644
> --- a/arch/x86/boot/compressed/Makefile
> +++ b/arch/x86/boot/compressed/Makefile
> @@ -73,6 +73,8 @@ $(obj)/vmlinux.relocs: vmlinux FORCE
>  vmlinux.bin.all-y := $(obj)/vmlinux.bin
>  vmlinux.bin.all-$(CONFIG_X86_NEED_RELOCS) += $(obj)/vmlinux.relocs
>  
> +$(obj)/vmlinux.bin.raw: $(vmlinux.bin.all-y) FORCE
> + $(call if_changed,raw)
>  $(obj)/vmlinux.bin.gz: $(vmlinux.bin.all-y) FORCE
>   $(call if_changed,gzip)
>  $(obj)/vmlinux.bin.bz2: $(vmlinux.bin.all-y) FORCE
> @@ -86,6 +88,7 @@ $(obj)/vmlinux.bin.lzo: $(vmlinux.bin.all-y) FORCE
>  $(obj)/vmlinux.bin.lz4: $(vmlinux.bin.all-y) FORCE
>   $(call if_changed,lz4)
>  
> +suffix-$(CONFIG_KERNEL_RAW)  := raw
>  suffix-$(CONFIG_KERNEL_GZIP) := gz
>  suffix-$(CONFIG_KERNEL_BZIP2):= bz2
>  suffix-$(CONFIG_KERNEL_LZMA) := lzma
> diff --git a/arch/x86/boot/compressed/misc.c b/arch/x86/boot/compressed/misc.c
> index 79dac17..fb3cd43 100644
> --- a/arch/x86/boot/compressed/misc.c
> +++ b/arch/x86/boot/compressed/misc.c
> @@ -123,6 +123,20 @@ static char *vidmem;
>  static int vidport;
>  static int lines, cols;
>  
> +#ifdef CONFIG_KERNEL_RAW
> +#include 
> +static int __decompress(unsigned char *buf, long len,
> + long (*fill)(void*, unsigned long),
> + long (*flush)(void*, unsigned long),
> + unsigned char *outbuf, long olen,
> + long *pos,
> + void (*error)(char *x))
> +{
> + memcpy(outbuf, buf, olen);
> + return 0;
> +}
> +#endif
> +
>  #ifdef CONFIG_KERNEL_GZIP
>  #include "../../../../lib/decompress_inflate.c"
>  #endif
> diff --git a/init/Kconfig b/init/Kconfig
> index 2232080..1db2ea2 100644
> --- a/init/Kconfig
> +++ b/init/Kconfig
> @@ -137,6 +137,13 @@ choice
>  
> If in doubt, select 'gzip'
>  
> +config KERNEL_RAW
> + bool "RAW"
> + help
> +   No compression. It creates much bigger kernel and uses much more
> +   space (disk/memory) than other choices. It can be useful when
> +   decompression speed is the most concern while space is not a problem.

This needs to depend on a HAVE_KERNEL_RAW config that is selected by the
architectures that implement this target (x86).

Michal

Re: [PATCH] x86/boot: Support uncompressed kernel

[Michal Marek]

On 2017-03-23 13:51, Chao Peng wrote:
> Compressed kernel has its own drawback: uncompressing takes time. Even
> though the time is short enough to ignore for most cases but for cases that
> time is critical this is still a big number. In our on-going optimization
> for kernel boot time, the measured overall kernel boot time is ~90ms while
> the uncompressing takes ~50ms with gzip.
> 
> The patch adds a 'CONFIG_KERNEL_RAW' configure choice so the built binary
> can have no uncompressing at all. The experiment shows:
> 
> kernel   kernel sizetime in decompress_kernel
> compressed (gzip)3.3M   53ms
> uncompressed 14M3ms
> 
> Signed-off-by: Chao Peng 
> ---
>  arch/x86/boot/compressed/Makefile |  3 +++
>  arch/x86/boot/compressed/misc.c   | 14 ++
>  init/Kconfig  |  7 +++
>  scripts/Makefile.lib  |  8 
>  4 files changed, 32 insertions(+)
> 
> diff --git a/arch/x86/boot/compressed/Makefile 
> b/arch/x86/boot/compressed/Makefile
> index f9ce75d..fc0e1c0 100644
> --- a/arch/x86/boot/compressed/Makefile
> +++ b/arch/x86/boot/compressed/Makefile
> @@ -73,6 +73,8 @@ $(obj)/vmlinux.relocs: vmlinux FORCE
>  vmlinux.bin.all-y := $(obj)/vmlinux.bin
>  vmlinux.bin.all-$(CONFIG_X86_NEED_RELOCS) += $(obj)/vmlinux.relocs
>  
> +$(obj)/vmlinux.bin.raw: $(vmlinux.bin.all-y) FORCE
> + $(call if_changed,raw)
>  $(obj)/vmlinux.bin.gz: $(vmlinux.bin.all-y) FORCE
>   $(call if_changed,gzip)
>  $(obj)/vmlinux.bin.bz2: $(vmlinux.bin.all-y) FORCE
> @@ -86,6 +88,7 @@ $(obj)/vmlinux.bin.lzo: $(vmlinux.bin.all-y) FORCE
>  $(obj)/vmlinux.bin.lz4: $(vmlinux.bin.all-y) FORCE
>   $(call if_changed,lz4)
>  
> +suffix-$(CONFIG_KERNEL_RAW)  := raw
>  suffix-$(CONFIG_KERNEL_GZIP) := gz
>  suffix-$(CONFIG_KERNEL_BZIP2):= bz2
>  suffix-$(CONFIG_KERNEL_LZMA) := lzma
> diff --git a/arch/x86/boot/compressed/misc.c b/arch/x86/boot/compressed/misc.c
> index 79dac17..fb3cd43 100644
> --- a/arch/x86/boot/compressed/misc.c
> +++ b/arch/x86/boot/compressed/misc.c
> @@ -123,6 +123,20 @@ static char *vidmem;
>  static int vidport;
>  static int lines, cols;
>  
> +#ifdef CONFIG_KERNEL_RAW
> +#include 
> +static int __decompress(unsigned char *buf, long len,
> + long (*fill)(void*, unsigned long),
> + long (*flush)(void*, unsigned long),
> + unsigned char *outbuf, long olen,
> + long *pos,
> + void (*error)(char *x))
> +{
> + memcpy(outbuf, buf, olen);
> + return 0;
> +}
> +#endif
> +
>  #ifdef CONFIG_KERNEL_GZIP
>  #include "../../../../lib/decompress_inflate.c"
>  #endif
> diff --git a/init/Kconfig b/init/Kconfig
> index 2232080..1db2ea2 100644
> --- a/init/Kconfig
> +++ b/init/Kconfig
> @@ -137,6 +137,13 @@ choice
>  
> If in doubt, select 'gzip'
>  
> +config KERNEL_RAW
> + bool "RAW"
> + help
> +   No compression. It creates much bigger kernel and uses much more
> +   space (disk/memory) than other choices. It can be useful when
> +   decompression speed is the most concern while space is not a problem.

This needs to depend on a HAVE_KERNEL_RAW config that is selected by the
architectures that implement this target (x86).

Michal

Re: kbuild/mkspec: Fix architectures where KBUILD_IMAGE isn't a full path

[Michal Marek]

Dne 15.3.2017 v 15:51 Tom Rini napsal(a):
> I found https://patchwork.kernel.org/patch/9442211/ today and I see that
> it was brought up again just before I sent my patch.  I just want to
> point out that 9442211 doesn't address arm, arc, and sh and they will
> still not have a functional rpm build target.  This is at least probably
> important for arm.  Of course all of those could also be addressed with
> a patch similar to 9442211, and my main concern is making sure
> everything gets fixed one way or another.  Thanks!

The patch is 1/6 of a series that also covers other architectures, but
patchwork won't should you the rest of the thread. Have a look here for
instance: https://lkml.org/lkml/2016/11/22/750.

Michal

Re: kbuild/mkspec: Fix architectures where KBUILD_IMAGE isn't a full path

[Michal Marek]

Dne 15.3.2017 v 15:51 Tom Rini napsal(a):
> I found https://patchwork.kernel.org/patch/9442211/ today and I see that
> it was brought up again just before I sent my patch.  I just want to
> point out that 9442211 doesn't address arm, arc, and sh and they will
> still not have a functional rpm build target.  This is at least probably
> important for arm.  Of course all of those could also be addressed with
> a patch similar to 9442211, and my main concern is making sure
> everything gets fixed one way or another.  Thanks!

The patch is 1/6 of a series that also covers other architectures, but
patchwork won't should you the rest of the thread. Have a look here for
instance: https://lkml.org/lkml/2016/11/22/750.

Michal

Re: Kbuild maintainership

[Michal Marek]

On 2017-03-11 06:45, Masahiro Yamada wrote:
> Hi Michal,
> 
> 2017-03-10 19:15 GMT+09:00 Michal Marek <mma...@suse.com>:
>> On 2017-03-10 10:17, Stephen Rothwell wrote:
>>> I assume that I will get a request to change the kbuild-current and
>>> kbuilt trees in linux-next soon.  In the meantime, should I remove the
>>> current ones?
>>
>> There is one genksyms fix in kbuild.git#kbuild which is not in mainline.
>> Masahiro, can you please pull it into your tree and give Stephen the url
>> and branches?
> 
> 
> Pulled into the (new)  linux-kbuild/kbuild

Thank you!

Michal

Re: Kbuild maintainership

[Michal Marek]

On 2017-03-11 06:45, Masahiro Yamada wrote:
> Hi Michal,
> 
> 2017-03-10 19:15 GMT+09:00 Michal Marek :
>> On 2017-03-10 10:17, Stephen Rothwell wrote:
>>> I assume that I will get a request to change the kbuild-current and
>>> kbuilt trees in linux-next soon.  In the meantime, should I remove the
>>> current ones?
>>
>> There is one genksyms fix in kbuild.git#kbuild which is not in mainline.
>> Masahiro, can you please pull it into your tree and give Stephen the url
>> and branches?
> 
> 
> Pulled into the (new)  linux-kbuild/kbuild

Thank you!

Michal

Re: [PATCH 1/6] arm64: Use full path in KBUILD_IMAGE definition

[Michal Marek]

Dne 10.3.2017 v 18:58 Will Deacon napsal(a):
> Hi Michael,
> 
> On Tue, Nov 22, 2016 at 10:34:29PM +0100, Michal Marek wrote:
>> The KBUILD_IMAGE variable is used by the rpm and deb-pkg targets, which
>> expect it to point to the image file in the build directory. The
>> builddeb script has a workaround for architectures which only provide
>> the basename, but let's provide a clean interface for packaging tools.
>>
>> Cc: Catalin Marinas <catalin.mari...@arm.com>
>> Cc: Will Deacon <will.dea...@arm.com>
>> Cc: linux-arm-ker...@lists.infradead.org
>> Signed-off-by: Michal Marek <mma...@suse.com>
>> ---
>>  arch/arm64/Makefile | 6 +++---
>>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> What's the plan for this patch? I'm happy to merge it,

Great, please merge it.


> but I don't see
> the rest of the series queued anywhere.

I'll see what remains and submit the rest via the kbuild tree.

Thanks,
Michal

Re: [PATCH 1/6] arm64: Use full path in KBUILD_IMAGE definition

[Michal Marek]

Dne 10.3.2017 v 18:58 Will Deacon napsal(a):
> Hi Michael,
> 
> On Tue, Nov 22, 2016 at 10:34:29PM +0100, Michal Marek wrote:
>> The KBUILD_IMAGE variable is used by the rpm and deb-pkg targets, which
>> expect it to point to the image file in the build directory. The
>> builddeb script has a workaround for architectures which only provide
>> the basename, but let's provide a clean interface for packaging tools.
>>
>> Cc: Catalin Marinas 
>> Cc: Will Deacon 
>> Cc: linux-arm-ker...@lists.infradead.org
>> Signed-off-by: Michal Marek 
>> ---
>>  arch/arm64/Makefile | 6 +++---
>>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> What's the plan for this patch? I'm happy to merge it,

Great, please merge it.


> but I don't see
> the rest of the series queued anywhere.

I'll see what remains and submit the rest via the kbuild tree.

Thanks,
Michal

Re: Kbuild maintainership

[Michal Marek]

On 2017-03-10 10:17, Stephen Rothwell wrote:
> I assume that I will get a request to change the kbuild-current and
> kbuilt trees in linux-next soon.  In the meantime, should I remove the
> current ones?

There is one genksyms fix in kbuild.git#kbuild which is not in mainline.
Masahiro, can you please pull it into your tree and give Stephen the url
and branches?

Thanks,
Michal

Re: Kbuild maintainership

[Michal Marek]

On 2017-03-10 10:17, Stephen Rothwell wrote:
> I assume that I will get a request to change the kbuild-current and
> kbuilt trees in linux-next soon.  In the meantime, should I remove the
> current ones?

There is one genksyms fix in kbuild.git#kbuild which is not in mainline.
Masahiro, can you please pull it into your tree and give Stephen the url
and branches?

Thanks,
Michal

Re: Kbuild maintainership

[Michal Marek]

Dne 10.3.2017 v 03:07 Masahiro Yamada napsal(a):
> Hello Kbuild contributors,
> 
> 
> As you may notice, Michal has been busy since the beginning
> of this year, and it has been getting difficult for him to
> work on Kbuild on his regular basis.

Yes, that's unfortunately the case.


> Michal sent me an email a few days ago, and we discussed this locally.
> Then, I decided to be a new maintainer of Kbuild. [1]

Thanks a lot for this and I wish you good luck with this new challenge.

Michal

Re: Kbuild maintainership

[Michal Marek]

Dne 10.3.2017 v 03:07 Masahiro Yamada napsal(a):
> Hello Kbuild contributors,
> 
> 
> As you may notice, Michal has been busy since the beginning
> of this year, and it has been getting difficult for him to
> work on Kbuild on his regular basis.

Yes, that's unfortunately the case.


> Michal sent me an email a few days ago, and we discussed this locally.
> Then, I decided to be a new maintainer of Kbuild. [1]

Thanks a lot for this and I wish you good luck with this new challenge.

Michal

[GIT PULL] kbuild fix for v4.10-rc3

[Michal Marek]

Hi Linus,

The asm-prototypes.h file added in the last merge window results in
invalid code with CONFIG_KMEMCHECK=y. The net result is that genksyms
segfaults. This pull request fixes the header, the genksyms fix is in my
kbuild branch for 4.11.

Thanks,
Michal


The following changes since commit 7ce7d89f48834cefece7804d38fc5d85382edf77:

  Linux 4.10-rc1 (2016-12-25 16:13:08 -0800)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/mmarek/kbuild.git rc-fixes

for you to fetch changes up to c7858bf16c0b2cc62f475f31e6df28c3a68da1d6:

  asm-prototypes: Clear any CPP defines before declaring the functions 
(2017-01-05 12:57:42 +0100)


Michal Marek (1):
  asm-prototypes: Clear any CPP defines before declaring the functions

 include/asm-generic/asm-prototypes.h | 6 ++
 1 file changed, 6 insertions(+)

[GIT PULL] kbuild fix for v4.10-rc3

[Michal Marek]

Hi Linus,

The asm-prototypes.h file added in the last merge window results in
invalid code with CONFIG_KMEMCHECK=y. The net result is that genksyms
segfaults. This pull request fixes the header, the genksyms fix is in my
kbuild branch for 4.11.

Thanks,
Michal


The following changes since commit 7ce7d89f48834cefece7804d38fc5d85382edf77:

  Linux 4.10-rc1 (2016-12-25 16:13:08 -0800)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/mmarek/kbuild.git rc-fixes

for you to fetch changes up to c7858bf16c0b2cc62f475f31e6df28c3a68da1d6:

  asm-prototypes: Clear any CPP defines before declaring the functions 
(2017-01-05 12:57:42 +0100)


Michal Marek (1):
  asm-prototypes: Clear any CPP defines before declaring the functions

 include/asm-generic/asm-prototypes.h | 6 ++
 1 file changed, 6 insertions(+)

Re: [PATCH 1/3] asm-prototypes: Clear any CPP defines before declaring the functions

[Michal Marek]

On 2017-01-05 10:33, Borislav Petkov wrote:
> On Tue, Jan 03, 2017 at 03:14:54PM +0100, Michal Marek wrote:
>> During real build, this manifests itself by genksyms segfaulting.
>>
>> Reported-by: Borislav Petkov <b...@alien8.de>
> 
> ... -and-tested-by: ...
> 
> For all three.
> 
> Thanks!

Thanks for testing. I added 1/3 to kbuild.git#rc-fixes for -rc3 and the
remaining two to kbuild.git#kbuild for the next merge window.

Michal

Re: [PATCH 1/3] asm-prototypes: Clear any CPP defines before declaring the functions

[Michal Marek]

On 2017-01-05 10:33, Borislav Petkov wrote:
> On Tue, Jan 03, 2017 at 03:14:54PM +0100, Michal Marek wrote:
>> During real build, this manifests itself by genksyms segfaulting.
>>
>> Reported-by: Borislav Petkov 
> 
> ... -and-tested-by: ...
> 
> For all three.
> 
> Thanks!

Thanks for testing. I added 1/3 to kbuild.git#rc-fixes for -rc3 and the
remaining two to kbuild.git#kbuild for the next merge window.

Michal

[PATCH 2/3] genksyms: Fix segfault with invalid declarations

[Michal Marek]

Do not try to recover too early and segfault when parsing invalid
declarations such as

echo 'int (int);' | scripts/genksyms/genksyms
echo 'int a, (int);' | scripts/genksyms/genksyms
echo 'extern void *__inline_memcpy((void *), (const void *), 
(__kernel_size_t));' | scripts/genksyms/genksyms

The last one was a real-life bug with
include/asm-generic/asm-prototypes.h on x86_64.

Reported-by: Borislav Petkov <b...@alien8.de>
Signed-off-by: Michal Marek <mma...@suse.com>
---
 scripts/genksyms/parse.y | 2 --
 1 file changed, 2 deletions(-)

diff --git a/scripts/genksyms/parse.y b/scripts/genksyms/parse.y
index 4fba255e54ae..00a6d7e54971 100644
--- a/scripts/genksyms/parse.y
+++ b/scripts/genksyms/parse.y
@@ -322,8 +322,6 @@ direct_declarator:
{ $$ = $2; }
| '(' declarator ')'
{ $$ = $3; }
-   | '(' error ')'
-   { $$ = $3; }
;
 
 /* Nested declarators differ from regular declarators in that they do
-- 
2.10.2

[PATCH 2/3] genksyms: Fix segfault with invalid declarations

[Michal Marek]

Do not try to recover too early and segfault when parsing invalid
declarations such as

echo 'int (int);' | scripts/genksyms/genksyms
echo 'int a, (int);' | scripts/genksyms/genksyms
echo 'extern void *__inline_memcpy((void *), (const void *), 
(__kernel_size_t));' | scripts/genksyms/genksyms

The last one was a real-life bug with
include/asm-generic/asm-prototypes.h on x86_64.

Reported-by: Borislav Petkov 
Signed-off-by: Michal Marek 
---
 scripts/genksyms/parse.y | 2 --
 1 file changed, 2 deletions(-)

diff --git a/scripts/genksyms/parse.y b/scripts/genksyms/parse.y
index 4fba255e54ae..00a6d7e54971 100644
--- a/scripts/genksyms/parse.y
+++ b/scripts/genksyms/parse.y
@@ -322,8 +322,6 @@ direct_declarator:
{ $$ = $2; }
| '(' declarator ')'
{ $$ = $3; }
-   | '(' error ')'
-   { $$ = $3; }
;
 
 /* Nested declarators differ from regular declarators in that they do
-- 
2.10.2

[PATCH 3/3] genksyms: Regenerate parser

[Michal Marek]

Regenerate the parser after 309d4cef4667 ("genksyms: Fix segfault with
invalid declarations").

Reported-by: Reported-by: Borislav Petkov <b...@alien8.de>
Signed-off-by: Michal Marek <mma...@suse.com>
---
 scripts/genksyms/parse.tab.c_shipped | 474 +--
 1 file changed, 234 insertions(+), 240 deletions(-)

diff --git a/scripts/genksyms/parse.tab.c_shipped 
b/scripts/genksyms/parse.tab.c_shipped
index 69148d30ca3f..d02258bafe7b 100644
--- a/scripts/genksyms/parse.tab.c_shipped
+++ b/scripts/genksyms/parse.tab.c_shipped
@@ -440,16 +440,16 @@ union yyalloc
 /* YYFINAL -- State number of the termination state.  */
 #define YYFINAL  4
 /* YYLAST -- Last index in YYTABLE.  */
-#define YYLAST   524
+#define YYLAST   522
 
 /* YYNTOKENS -- Number of terminals.  */
 #define YYNTOKENS  55
 /* YYNNTS -- Number of nonterminals.  */
 #define YYNNTS  49
 /* YYNRULES -- Number of rules.  */
-#define YYNRULES  134
+#define YYNRULES  133
 /* YYNRULES -- Number of states.  */
-#define YYNSTATES  189
+#define YYNSTATES  187
 
 /* YYTRANSLATE(YYLEX) -- Bison symbol number corresponding to YYLEX.  */
 #define YYUNDEFTOK  2
@@ -506,13 +506,13 @@ static const yytype_uint16 yyprhs[] =
   97,   101,   105,   109,   112,   115,   118,   120,   122,   124,
  126,   128,   130,   132,   134,   136,   138,   140,   142,   145,
  146,   148,   150,   153,   155,   157,   159,   161,   164,   166,
- 168,   170,   175,   180,   183,   187,   191,   194,   196,   198,
- 200,   205,   210,   213,   217,   221,   224,   226,   230,   231,
- 233,   235,   239,   242,   245,   247,   248,   250,   252,   257,
- 262,   265,   269,   273,   277,   278,   280,   283,   287,   291,
- 292,   294,   296,   299,   303,   306,   307,   309,   311,   315,
- 318,   321,   323,   326,   327,   330,   334,   339,   341,   345,
- 347,   351,   354,   355,   357
+ 168,   170,   175,   180,   183,   187,   190,   192,   194,   196,
+ 201,   206,   209,   213,   217,   220,   222,   226,   227,   229,
+ 231,   235,   238,   241,   243,   244,   246,   248,   253,   258,
+ 261,   265,   269,   273,   274,   276,   279,   283,   287,   288,
+ 290,   292,   295,   299,   302,   303,   305,   307,   311,   314,
+ 317,   319,   322,   323,   326,   330,   335,   337,   341,   343,
+ 347,   350,   351,   353
 };
 
 /* YYRHS -- A `-1'-separated list of the rules' RHS.  */
@@ -536,25 +536,24 @@ static const yytype_int8 yyrhs[] =
   74,75,-1, 8,-1,27,-1,32,-1,18,
   -1,72,76,-1,77,-1,39,-1,43,-1,
   77,49,80,50,-1,77,49, 1,50,-1,
-  77,35,-1,49,76,50,-1,49, 1,50,
-  -1,72,78,-1,79,-1,39,-1,43,-1,
-  79,49,80,50,-1,79,49, 1,50,-1,
-  79,35,-1,49,78,50,-1,49, 1,50,
-  -1,81,38,-1,81,-1,82,48,38,-1,
-  -1,82,-1,83,-1,82,48,83,-1,67,
-  84,-1,72,84,-1,85,-1,-1,39,-1,
-  43,-1,85,49,80,50,-1,85,49, 1,
-  50,-1,85,35,-1,49,84,50,-1,49,
-   1,50,-1,66,76,34,-1,-1,88,-1,
-  52,36,-1,53,90,47,-1,53, 1,47,
-  -1,-1,91,-1,92,-1,91,92,-1,66,
-  93,46,-1, 1,46,-1,-1,94,-1,95,
-  -1,94,48,95,-1,78,97,-1,39,96,
-  -1,96,-1,54,36,-1,-1,97,32,-1,
-  53,99,47,-1,53,99,48,47,-1,   100,
-  -1,99,48,   100,-1,39,-1,39,52,36,
-  -1,31,46,-1,-1,31,-1,30,49,39,
-  50,46,-1
+  77,35,-1,49,76,50,-1,72,78,-1,
+  79,-1,39,-1,43,-1,79,49,80,50,
+  -1,79,49, 1,50,-1,79,35,-1,49,
+  78,50,-1,49, 1,50,-1,81,38,-1,
+  81,-1,82,48,38,-1,-1,82,-1,83,
+  -1,82,48,83,-1,67,84,-1,72,84,
+  -1,85,-1,-1,39,-1,43,-1,85,49,
+  80,50,-1,85,49, 1,50,-1,85,35,
+  -1,49,84,50,-1,49, 1,50,-1,66,
+  76,34,-1,-1,88,-1,52,36,-1,53,
+  90,47,-1,53, 1,47,-1,-1,91,-1,
+  92,-1,91,92,-1,66,93,46,-1,  

[PATCH 3/3] genksyms: Regenerate parser

[Michal Marek]

Regenerate the parser after 309d4cef4667 ("genksyms: Fix segfault with
invalid declarations").

Reported-by: Reported-by: Borislav Petkov 
Signed-off-by: Michal Marek 
---
 scripts/genksyms/parse.tab.c_shipped | 474 +--
 1 file changed, 234 insertions(+), 240 deletions(-)

diff --git a/scripts/genksyms/parse.tab.c_shipped 
b/scripts/genksyms/parse.tab.c_shipped
index 69148d30ca3f..d02258bafe7b 100644
--- a/scripts/genksyms/parse.tab.c_shipped
+++ b/scripts/genksyms/parse.tab.c_shipped
@@ -440,16 +440,16 @@ union yyalloc
 /* YYFINAL -- State number of the termination state.  */
 #define YYFINAL  4
 /* YYLAST -- Last index in YYTABLE.  */
-#define YYLAST   524
+#define YYLAST   522
 
 /* YYNTOKENS -- Number of terminals.  */
 #define YYNTOKENS  55
 /* YYNNTS -- Number of nonterminals.  */
 #define YYNNTS  49
 /* YYNRULES -- Number of rules.  */
-#define YYNRULES  134
+#define YYNRULES  133
 /* YYNRULES -- Number of states.  */
-#define YYNSTATES  189
+#define YYNSTATES  187
 
 /* YYTRANSLATE(YYLEX) -- Bison symbol number corresponding to YYLEX.  */
 #define YYUNDEFTOK  2
@@ -506,13 +506,13 @@ static const yytype_uint16 yyprhs[] =
   97,   101,   105,   109,   112,   115,   118,   120,   122,   124,
  126,   128,   130,   132,   134,   136,   138,   140,   142,   145,
  146,   148,   150,   153,   155,   157,   159,   161,   164,   166,
- 168,   170,   175,   180,   183,   187,   191,   194,   196,   198,
- 200,   205,   210,   213,   217,   221,   224,   226,   230,   231,
- 233,   235,   239,   242,   245,   247,   248,   250,   252,   257,
- 262,   265,   269,   273,   277,   278,   280,   283,   287,   291,
- 292,   294,   296,   299,   303,   306,   307,   309,   311,   315,
- 318,   321,   323,   326,   327,   330,   334,   339,   341,   345,
- 347,   351,   354,   355,   357
+ 168,   170,   175,   180,   183,   187,   190,   192,   194,   196,
+ 201,   206,   209,   213,   217,   220,   222,   226,   227,   229,
+ 231,   235,   238,   241,   243,   244,   246,   248,   253,   258,
+ 261,   265,   269,   273,   274,   276,   279,   283,   287,   288,
+ 290,   292,   295,   299,   302,   303,   305,   307,   311,   314,
+ 317,   319,   322,   323,   326,   330,   335,   337,   341,   343,
+ 347,   350,   351,   353
 };
 
 /* YYRHS -- A `-1'-separated list of the rules' RHS.  */
@@ -536,25 +536,24 @@ static const yytype_int8 yyrhs[] =
   74,75,-1, 8,-1,27,-1,32,-1,18,
   -1,72,76,-1,77,-1,39,-1,43,-1,
   77,49,80,50,-1,77,49, 1,50,-1,
-  77,35,-1,49,76,50,-1,49, 1,50,
-  -1,72,78,-1,79,-1,39,-1,43,-1,
-  79,49,80,50,-1,79,49, 1,50,-1,
-  79,35,-1,49,78,50,-1,49, 1,50,
-  -1,81,38,-1,81,-1,82,48,38,-1,
-  -1,82,-1,83,-1,82,48,83,-1,67,
-  84,-1,72,84,-1,85,-1,-1,39,-1,
-  43,-1,85,49,80,50,-1,85,49, 1,
-  50,-1,85,35,-1,49,84,50,-1,49,
-   1,50,-1,66,76,34,-1,-1,88,-1,
-  52,36,-1,53,90,47,-1,53, 1,47,
-  -1,-1,91,-1,92,-1,91,92,-1,66,
-  93,46,-1, 1,46,-1,-1,94,-1,95,
-  -1,94,48,95,-1,78,97,-1,39,96,
-  -1,96,-1,54,36,-1,-1,97,32,-1,
-  53,99,47,-1,53,99,48,47,-1,   100,
-  -1,99,48,   100,-1,39,-1,39,52,36,
-  -1,31,46,-1,-1,31,-1,30,49,39,
-  50,46,-1
+  77,35,-1,49,76,50,-1,72,78,-1,
+  79,-1,39,-1,43,-1,79,49,80,50,
+  -1,79,49, 1,50,-1,79,35,-1,49,
+  78,50,-1,49, 1,50,-1,81,38,-1,
+  81,-1,82,48,38,-1,-1,82,-1,83,
+  -1,82,48,83,-1,67,84,-1,72,84,
+  -1,85,-1,-1,39,-1,43,-1,85,49,
+  80,50,-1,85,49, 1,50,-1,85,35,
+  -1,49,84,50,-1,49, 1,50,-1,66,
+  76,34,-1,-1,88,-1,52,36,-1,53,
+  90,47,-1,53, 1,47,-1,-1,91,-1,
+  92,-1,91,92,-1,66,93,46,-1, 1,
+  46,-1,-1,94,-1,   

[PATCH 1/3] asm-prototypes: Clear any CPP defines before declaring the functions

[Michal Marek]

The asm-prototypes.h file is used to provide dummy function declarations
for genksyms, when processing asm files with EXPORT_SYMBOL. Make sure
that any architecture defines get out of our way. x86 currently has an
issue with memcpy on 64bit with CONFIG_KMEMCHECK=y and with
memset/__memset on 32bit:

$ cat init/test.c
#include 
$ make -s init/test.o
$ make -s init/test.o
In file included from ./arch/x86/include/asm/string.h:4:0,
 from ./include/linux/string.h:18,
 from ./include/linux/bitmap.h:8,
 from ./include/linux/cpumask.h:11,
 from ./arch/x86/include/asm/cpumask.h:4,
 from ./arch/x86/include/asm/msr.h:10,
 from ./arch/x86/include/asm/processor.h:20,
 from ./arch/x86/include/asm/cpufeature.h:4,
 from ./arch/x86/include/asm/thread_info.h:52,
 from ./include/linux/thread_info.h:25,
 from ./arch/x86/include/asm/preempt.h:6,
 from ./include/linux/preempt.h:59,
 from ./include/linux/spinlock.h:50,
 from ./include/linux/seqlock.h:35,
 from ./include/linux/time.h:5,
 from ./include/uapi/linux/timex.h:56,
 from ./include/linux/timex.h:56,
 from ./include/linux/sched.h:19,
 from ./include/linux/uaccess.h:4,
 from ./arch/x86/include/asm/asm-prototypes.h:2,
 from init/test.c:1:
./arch/x86/include/asm/string_64.h:52:47: error: expected declaration 
specifiers or ‘...’ before ‘(’ token
 #define memcpy(dst, src, len) __inline_memcpy((dst), (src), (len))
 ./include/asm-generic/asm-prototypes.h:6:14: note: in expansion of 
macro ‘memcpy’
  extern void *memcpy(void *, const void *, __kernel_size_t);

   ^
...

During real build, this manifests itself by genksyms segfaulting.

Reported-by: Borislav Petkov <b...@alien8.de>
Cc: Adam Borowski <kilob...@angband.pl>
Signed-off-by: Michal Marek <mma...@suse.com>
---
 include/asm-generic/asm-prototypes.h | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/include/asm-generic/asm-prototypes.h 
b/include/asm-generic/asm-prototypes.h
index df13637e4017..939869c772b1 100644
--- a/include/asm-generic/asm-prototypes.h
+++ b/include/asm-generic/asm-prototypes.h
@@ -1,7 +1,13 @@
 #include 
+#undef __memset
 extern void *__memset(void *, int, __kernel_size_t);
+#undef __memcpy
 extern void *__memcpy(void *, const void *, __kernel_size_t);
+#undef __memmove
 extern void *__memmove(void *, const void *, __kernel_size_t);
+#undef memset
 extern void *memset(void *, int, __kernel_size_t);
+#undef memcpy
 extern void *memcpy(void *, const void *, __kernel_size_t);
+#undef memmove
 extern void *memmove(void *, const void *, __kernel_size_t);
-- 
2.10.2

[PATCH 1/3] asm-prototypes: Clear any CPP defines before declaring the functions

[Michal Marek]

The asm-prototypes.h file is used to provide dummy function declarations
for genksyms, when processing asm files with EXPORT_SYMBOL. Make sure
that any architecture defines get out of our way. x86 currently has an
issue with memcpy on 64bit with CONFIG_KMEMCHECK=y and with
memset/__memset on 32bit:

$ cat init/test.c
#include 
$ make -s init/test.o
$ make -s init/test.o
In file included from ./arch/x86/include/asm/string.h:4:0,
 from ./include/linux/string.h:18,
 from ./include/linux/bitmap.h:8,
 from ./include/linux/cpumask.h:11,
 from ./arch/x86/include/asm/cpumask.h:4,
 from ./arch/x86/include/asm/msr.h:10,
 from ./arch/x86/include/asm/processor.h:20,
 from ./arch/x86/include/asm/cpufeature.h:4,
 from ./arch/x86/include/asm/thread_info.h:52,
 from ./include/linux/thread_info.h:25,
 from ./arch/x86/include/asm/preempt.h:6,
 from ./include/linux/preempt.h:59,
 from ./include/linux/spinlock.h:50,
 from ./include/linux/seqlock.h:35,
 from ./include/linux/time.h:5,
 from ./include/uapi/linux/timex.h:56,
 from ./include/linux/timex.h:56,
 from ./include/linux/sched.h:19,
 from ./include/linux/uaccess.h:4,
 from ./arch/x86/include/asm/asm-prototypes.h:2,
 from init/test.c:1:
./arch/x86/include/asm/string_64.h:52:47: error: expected declaration 
specifiers or ‘...’ before ‘(’ token
 #define memcpy(dst, src, len) __inline_memcpy((dst), (src), (len))
 ./include/asm-generic/asm-prototypes.h:6:14: note: in expansion of 
macro ‘memcpy’
  extern void *memcpy(void *, const void *, __kernel_size_t);

   ^
...

During real build, this manifests itself by genksyms segfaulting.

Reported-by: Borislav Petkov 
Cc: Adam Borowski 
Signed-off-by: Michal Marek 
---
 include/asm-generic/asm-prototypes.h | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/include/asm-generic/asm-prototypes.h 
b/include/asm-generic/asm-prototypes.h
index df13637e4017..939869c772b1 100644
--- a/include/asm-generic/asm-prototypes.h
+++ b/include/asm-generic/asm-prototypes.h
@@ -1,7 +1,13 @@
 #include 
+#undef __memset
 extern void *__memset(void *, int, __kernel_size_t);
+#undef __memcpy
 extern void *__memcpy(void *, const void *, __kernel_size_t);
+#undef __memmove
 extern void *__memmove(void *, const void *, __kernel_size_t);
+#undef memset
 extern void *memset(void *, int, __kernel_size_t);
+#undef memcpy
 extern void *memcpy(void *, const void *, __kernel_size_t);
+#undef memmove
 extern void *memmove(void *, const void *, __kernel_size_t);
-- 
2.10.2

Re: default 0 if KASAN expression not working in kbuild

[Michal Marek]

On 2016-12-15 19:09, Nikolay Borisov wrote:
> Hello,
> 
> I was doing some kasan-related debugging and when I enabled it I started
> getting warnings for large stackframes. So CONFIG_FRAME_WARN has :
> 
> int "Warn for stack frames larger than (needs gcc 4.4)"
> range 0 8192
> default 0 if KASAN
> default 2048 if GCC_PLUGIN_LATENT_ENTROPY
> default 1024 if !64BIT
> default 2048 if 64BIT
> 
> This means that frame_warns should effectively be disabled when kasan is
> enabled. However in my case this is not the situation.
> http://sprunge.us/FiGf here is the config file. It does have
> CONFIG_KASAN=y and CONFIG_FRAME_WARN=1024 . And even this is erroneous
> since it's a 64bit kernel, so it should be 2k. I haven't manually set
> the limit to 1k either.

I guess the problem is that it's an option with a propt, so deviating
from the default value is legitimate. And most likely the
value is computed during menuconfig startup and a subsequent change of
CONFIG_KASAN does not change it (because 2048 or 1048 are also valid).

Michal

Re: default 0 if KASAN expression not working in kbuild

[Michal Marek]

On 2016-12-15 19:09, Nikolay Borisov wrote:
> Hello,
> 
> I was doing some kasan-related debugging and when I enabled it I started
> getting warnings for large stackframes. So CONFIG_FRAME_WARN has :
> 
> int "Warn for stack frames larger than (needs gcc 4.4)"
> range 0 8192
> default 0 if KASAN
> default 2048 if GCC_PLUGIN_LATENT_ENTROPY
> default 1024 if !64BIT
> default 2048 if 64BIT
> 
> This means that frame_warns should effectively be disabled when kasan is
> enabled. However in my case this is not the situation.
> http://sprunge.us/FiGf here is the config file. It does have
> CONFIG_KASAN=y and CONFIG_FRAME_WARN=1024 . And even this is erroneous
> since it's a 64bit kernel, so it should be 2k. I haven't manually set
> the limit to 1k either.

I guess the problem is that it's an option with a propt, so deviating
from the default value is legitimate. And most likely the
value is computed during menuconfig startup and a subsequent change of
CONFIG_KASAN does not change it (because 2048 or 1048 are also valid).

Michal

Re: [PATCH] scripts: checkincludes: add exit message for no duplicates found

[Michal Marek]

On 2016-12-18 13:35, Cheah Kok Cheong wrote:
> If no duplicates found, inform user.
> 
> Signed-off-by: Cheah Kok Cheong 
> ---
>  scripts/checkincludes.pl | 13 +
>  1 file changed, 13 insertions(+)
> 
> diff --git a/scripts/checkincludes.pl b/scripts/checkincludes.pl
> index 97b2c61..2bf234c 100755
> --- a/scripts/checkincludes.pl
> +++ b/scripts/checkincludes.pl
> @@ -37,6 +37,13 @@ if ($#ARGV >= 1) {
>   }
>  }
>  
> +my $dup_counter = 0;
> +
> +sub nodup_count {
> + ++$dup_counter;
> + return;
> +}

I don't think it's necessary to have a function only to increment a
variable.

Michal

Re: [PATCH] scripts: checkincludes: add exit message for no duplicates found

[Michal Marek]

On 2016-12-18 13:35, Cheah Kok Cheong wrote:
> If no duplicates found, inform user.
> 
> Signed-off-by: Cheah Kok Cheong 
> ---
>  scripts/checkincludes.pl | 13 +
>  1 file changed, 13 insertions(+)
> 
> diff --git a/scripts/checkincludes.pl b/scripts/checkincludes.pl
> index 97b2c61..2bf234c 100755
> --- a/scripts/checkincludes.pl
> +++ b/scripts/checkincludes.pl
> @@ -37,6 +37,13 @@ if ($#ARGV >= 1) {
>   }
>  }
>  
> +my $dup_counter = 0;
> +
> +sub nodup_count {
> + ++$dup_counter;
> + return;
> +}

I don't think it's necessary to have a function only to increment a
variable.

Michal

[GIT PULL] kconfig update for v4.10-rc1

[Michal Marek]

Hi Linus,

please pull these kconfig bits for v4.10-rc1:
- make xconfig gui fixes
- make nconfig fix for options with long prompts
- fix make nconfig warning when pkg-config forces -D_GNU_SOURCE

Michal

The following changes since commit 1001354ca34179f3db924eb66672442a173147dc:

  Linux 4.9-rc1 (2016-10-15 12:17:50 -0700)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/mmarek/kbuild.git kconfig

for you to fetch changes up to 83c3a1bad224189f22ca2c1955337c3478bd3ab2:

  xconfig: fix missing suboption and help panels on first run (2016-12-11 
21:40:54 +0100)


Ben Hutchings (1):
  kconfig/nconf: Fix hang when editing symbol with a long prompt

Boris Barbulovski (2):
  xconfig: fix 'Show Debug' functionality
  xconfig: fix missing suboption and help panels on first run

Cheah Kok Cheong (1):
  Scripts: kconfig: nconf: fix _GNU_SOURCE redefined warning

 scripts/kconfig/nconf.c |  2 ++
 scripts/kconfig/nconf.gui.c | 15 +++
 scripts/kconfig/qconf.cc| 19 ++-
 3 files changed, 27 insertions(+), 9 deletions(-)

[GIT PULL] kbuild updates for v4.10-rc1

[Michal Marek]

Hi Linus,

please pull these kbuild updates for v4.10-rc1:
- Prototypes for x86 asm-exported symbols (Adam Borowski) and a warning
  about missing CRCs (Nick Piggin)
- asm-exports fix for LTO (Nicolas Pitre)
- Thin archives improvements (Nick Piggin)
- Linker script fix for CONFIG_LD_DEAD_CODE_DATA_ELIMINATION (Nick
  Piggin)
- genksyms support for __builtin_va_list keyword
- misc minor fixes

Michal

The following changes since commit 1001354ca34179f3db924eb66672442a173147dc:

  Linux 4.9-rc1 (2016-10-15 12:17:50 -0700)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/mmarek/kbuild.git kbuild

for you to fetch changes up to 334bb773876403eae3457d81be0b8ea70f8e4ccc:

  x86/kbuild: enable modversions for symbols exported from asm (2016-12-14 
00:35:35 +0100)


Adam Borowski (1):
  x86/kbuild: enable modversions for symbols exported from asm

Ard Biesheuvel (1):
  scripts/kallsyms: remove last remnants of --page-offset option

Michal Marek (1):
  genksyms: Regenerate parser

Nicholas Piggin (7):
  kbuild: kallsyms allow 3-pass generation if symbols size has changed
  kbuild: thin archives for multi-y targets
  kbuild/genksyms: handle va_list type
  kbuild: improve linker compatibility with lib-ksyms.o build
  kbuild: keep data tables through dead code elimination
  kbuild: modpost warn if export version crc is missing
  kbuild: minor improvement for thin archives build

Nicolas Pitre (2):
  kbuild: cmd_export_list: tighten the sed script
  kbuild: fix scripts/adjust_autoksyms.sh* for the no modules case

Uwe Kleine-König (1):
  make use of make variable CURDIR instead of calling pwd

 arch/x86/include/asm/asm-prototypes.h  |  16 +
 include/asm-generic/asm-prototypes.h   |   7 +
 include/asm-generic/vmlinux.lds.h  |  68 +--
 samples/bpf/Makefile   |   4 +-
 samples/connector/Makefile |   2 +-
 scripts/Makefile.build |  17 +-
 scripts/adjust_autoksyms.sh|   1 +
 scripts/genksyms/keywords.gperf|   1 +
 scripts/genksyms/keywords.hash.c_shipped   |  59 +--
 scripts/genksyms/parse.tab.c_shipped   | 676 +++--
 scripts/genksyms/parse.tab.h_shipped   |  33 +-
 scripts/genksyms/parse.y   |   2 +
 scripts/kallsyms.c |   1 -
 scripts/link-vmlinux.sh|  37 +-
 scripts/mod/modpost.c  |   6 +
 tools/build/Makefile   |   2 +-
 tools/gpio/Makefile|   2 +-
 tools/lib/api/Makefile |   2 +-
 tools/lib/bpf/Makefile |   2 +-
 tools/lib/lockdep/Makefile |   2 +-
 tools/lib/subcmd/Makefile  |   2 +-
 tools/lib/traceevent/Makefile  |   2 +-
 tools/objtool/Makefile |   4 +-
 tools/perf/Makefile.perf   |   2 +-
 tools/perf/tests/make  |   2 +-
 tools/power/cpupower/Makefile  |   3 -
 tools/power/cpupower/debug/kernel/Makefile |   3 +-
 27 files changed, 503 insertions(+), 455 deletions(-)
 create mode 100644 arch/x86/include/asm/asm-prototypes.h
 create mode 100644 include/asm-generic/asm-prototypes.h

[GIT PULL] kconfig update for v4.10-rc1

[Michal Marek]

Hi Linus,

please pull these kconfig bits for v4.10-rc1:
- make xconfig gui fixes
- make nconfig fix for options with long prompts
- fix make nconfig warning when pkg-config forces -D_GNU_SOURCE

Michal

The following changes since commit 1001354ca34179f3db924eb66672442a173147dc:

  Linux 4.9-rc1 (2016-10-15 12:17:50 -0700)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/mmarek/kbuild.git kconfig

for you to fetch changes up to 83c3a1bad224189f22ca2c1955337c3478bd3ab2:

  xconfig: fix missing suboption and help panels on first run (2016-12-11 
21:40:54 +0100)


Ben Hutchings (1):
  kconfig/nconf: Fix hang when editing symbol with a long prompt

Boris Barbulovski (2):
  xconfig: fix 'Show Debug' functionality
  xconfig: fix missing suboption and help panels on first run

Cheah Kok Cheong (1):
  Scripts: kconfig: nconf: fix _GNU_SOURCE redefined warning

 scripts/kconfig/nconf.c |  2 ++
 scripts/kconfig/nconf.gui.c | 15 +++
 scripts/kconfig/qconf.cc| 19 ++-
 3 files changed, 27 insertions(+), 9 deletions(-)

[GIT PULL] kbuild updates for v4.10-rc1

[Michal Marek]

Hi Linus,

please pull these kbuild updates for v4.10-rc1:
- Prototypes for x86 asm-exported symbols (Adam Borowski) and a warning
  about missing CRCs (Nick Piggin)
- asm-exports fix for LTO (Nicolas Pitre)
- Thin archives improvements (Nick Piggin)
- Linker script fix for CONFIG_LD_DEAD_CODE_DATA_ELIMINATION (Nick
  Piggin)
- genksyms support for __builtin_va_list keyword
- misc minor fixes

Michal

The following changes since commit 1001354ca34179f3db924eb66672442a173147dc:

  Linux 4.9-rc1 (2016-10-15 12:17:50 -0700)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/mmarek/kbuild.git kbuild

for you to fetch changes up to 334bb773876403eae3457d81be0b8ea70f8e4ccc:

  x86/kbuild: enable modversions for symbols exported from asm (2016-12-14 
00:35:35 +0100)


Adam Borowski (1):
  x86/kbuild: enable modversions for symbols exported from asm

Ard Biesheuvel (1):
  scripts/kallsyms: remove last remnants of --page-offset option

Michal Marek (1):
  genksyms: Regenerate parser

Nicholas Piggin (7):
  kbuild: kallsyms allow 3-pass generation if symbols size has changed
  kbuild: thin archives for multi-y targets
  kbuild/genksyms: handle va_list type
  kbuild: improve linker compatibility with lib-ksyms.o build
  kbuild: keep data tables through dead code elimination
  kbuild: modpost warn if export version crc is missing
  kbuild: minor improvement for thin archives build

Nicolas Pitre (2):
  kbuild: cmd_export_list: tighten the sed script
  kbuild: fix scripts/adjust_autoksyms.sh* for the no modules case

Uwe Kleine-König (1):
  make use of make variable CURDIR instead of calling pwd

 arch/x86/include/asm/asm-prototypes.h  |  16 +
 include/asm-generic/asm-prototypes.h   |   7 +
 include/asm-generic/vmlinux.lds.h  |  68 +--
 samples/bpf/Makefile   |   4 +-
 samples/connector/Makefile |   2 +-
 scripts/Makefile.build |  17 +-
 scripts/adjust_autoksyms.sh|   1 +
 scripts/genksyms/keywords.gperf|   1 +
 scripts/genksyms/keywords.hash.c_shipped   |  59 +--
 scripts/genksyms/parse.tab.c_shipped   | 676 +++--
 scripts/genksyms/parse.tab.h_shipped   |  33 +-
 scripts/genksyms/parse.y   |   2 +
 scripts/kallsyms.c |   1 -
 scripts/link-vmlinux.sh|  37 +-
 scripts/mod/modpost.c  |   6 +
 tools/build/Makefile   |   2 +-
 tools/gpio/Makefile|   2 +-
 tools/lib/api/Makefile |   2 +-
 tools/lib/bpf/Makefile |   2 +-
 tools/lib/lockdep/Makefile |   2 +-
 tools/lib/subcmd/Makefile  |   2 +-
 tools/lib/traceevent/Makefile  |   2 +-
 tools/objtool/Makefile |   4 +-
 tools/perf/Makefile.perf   |   2 +-
 tools/perf/tests/make  |   2 +-
 tools/power/cpupower/Makefile  |   3 -
 tools/power/cpupower/debug/kernel/Makefile |   3 +-
 27 files changed, 503 insertions(+), 455 deletions(-)
 create mode 100644 arch/x86/include/asm/asm-prototypes.h
 create mode 100644 include/asm-generic/asm-prototypes.h

[GIT PULL] kbuild misc updates for v4.10-rc1

[Michal Marek]

Hi Linus,

please pull these non-core kbuild bits for v4.10-rc1:
- One new coccinelle check and improvements to irqf_oneshot.cocci
- make rpm POSIX compatibility fix
- make deb-pkg arm64 cross-compiling fix. I forgot to send this one
  during the v4.9 rc-phase, therefor the pull request is based on -rc6
  and not -rc1

Thanks,
Michal

The following changes since commit c6a385539175ebc603da53aafb7753d39089f32e:

  kbuild: Steal gcc's pie from the very beginning (2016-11-15 17:08:14 +0100)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/mmarek/kbuild.git misc

for you to fetch changes up to ff5b1ce6b22080379fd1ae9076ff2f5ca6519861:

  Merge branch 'kbuild/rc-fixes' into kbuild/misc (2016-12-11 21:20:34 +0100)


Adam Borowski (1):
  builddeb: fix cross-building to arm64 producing host-arch debs

Andrew F. Davis (1):
  Coccinelle: Add misc/boolconv.cocci

Anton Tikhomirov (1):
  kbuild/mkspec: avoid using brace expansion

Michal Marek (1):
  Merge branch 'kbuild/rc-fixes' into kbuild/misc

Vaishali Thakkar (3):
  Coccinelle: misc: Improve the matching of rules
  Coccinelle: misc: Improve the result given by context mode
  Coccinelle: misc: Add support for devm variant in all modes

 scripts/coccinelle/misc/boolconv.cocci | 90 ++
 scripts/coccinelle/misc/irqf_oneshot.cocci | 52 +++--
 scripts/package/builddeb   |  2 +-
 scripts/package/mkspec |  3 +-
 4 files changed, 129 insertions(+), 18 deletions(-)
 create mode 100644 scripts/coccinelle/misc/boolconv.cocci

[GIT PULL] kbuild misc updates for v4.10-rc1

[Michal Marek]

Hi Linus,

please pull these non-core kbuild bits for v4.10-rc1:
- One new coccinelle check and improvements to irqf_oneshot.cocci
- make rpm POSIX compatibility fix
- make deb-pkg arm64 cross-compiling fix. I forgot to send this one
  during the v4.9 rc-phase, therefor the pull request is based on -rc6
  and not -rc1

Thanks,
Michal

The following changes since commit c6a385539175ebc603da53aafb7753d39089f32e:

  kbuild: Steal gcc's pie from the very beginning (2016-11-15 17:08:14 +0100)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/mmarek/kbuild.git misc

for you to fetch changes up to ff5b1ce6b22080379fd1ae9076ff2f5ca6519861:

  Merge branch 'kbuild/rc-fixes' into kbuild/misc (2016-12-11 21:20:34 +0100)


Adam Borowski (1):
  builddeb: fix cross-building to arm64 producing host-arch debs

Andrew F. Davis (1):
  Coccinelle: Add misc/boolconv.cocci

Anton Tikhomirov (1):
  kbuild/mkspec: avoid using brace expansion

Michal Marek (1):
  Merge branch 'kbuild/rc-fixes' into kbuild/misc

Vaishali Thakkar (3):
  Coccinelle: misc: Improve the matching of rules
  Coccinelle: misc: Improve the result given by context mode
  Coccinelle: misc: Add support for devm variant in all modes

 scripts/coccinelle/misc/boolconv.cocci | 90 ++
 scripts/coccinelle/misc/irqf_oneshot.cocci | 52 +++--
 scripts/package/builddeb   |  2 +-
 scripts/package/mkspec |  3 +-
 4 files changed, 129 insertions(+), 18 deletions(-)
 create mode 100644 scripts/coccinelle/misc/boolconv.cocci

Re: [PATCH] x86/kbuild: enable modversions for symbols exported from asm

[Michal Marek]

On 2016-12-14 10:15, Michal Marek wrote:
> A minimal example would be
> 
> t1.c:
> struct s1;
> struct s2 {
>   int i;
> }
> struct s3 {
>   struct s1 *ptr1;
>   struct s2 *ptr2;
> }
> void foo(struct s3*);
> EXPORT_SYMBOL(foo);
> 
> t2.c:
> struct s1 {
>   int j;
> }
> struct s2;
> struct s3 {
>   struct s1 *ptr1;
>   struct s2 *ptr2;
> }
> void foo(struct s3*);
> EXPORT_SYMBOL(foo);

Note that the above, if passed to genksyms verbatim, would result in
genksyms treating all the types as internal. Here is a complete
example including linemarkers:

$ cat t1.i
# 1 "t1.c"
# 1 ""
# 1 ""
# 1 "t1.c"
# 1 "t1.h" 1
# 1 "t.h" 1
struct s1;
struct s2;
struct s3 {
 struct s1 *ptr1;
 struct s2 *ptr2;
};
# 2 "t1.h" 2
struct s2 {
 int i;
};
# 2 "t1.c" 2
void foo(struct s3 *s) { }
EXPORT_SYMBOL(foo);

$ cat t2.i
# 1 "t2.c"
# 1 ""
# 1 ""
# 1 "t2.c"
# 1 "t2.h" 1
# 1 "t.h" 1
struct s1;
struct s2;
struct s3 {
 struct s1 *ptr1;
 struct s2 *ptr2;
};
# 2 "t2.h" 2
struct s1 {
 int j;
};
# 2 "t2.c" 2
void foo(struct s3 *s) { }
EXPORT_SYMBOL(foo);

$ ./scripts/genksyms/genksyms -D 
__crc_foo = 0xf731cef8 ;

$ ./scripts/genksyms/genksyms -D 
__crc_foo = 0xc925dae5 ;

Michal

Re: [PATCH] x86/kbuild: enable modversions for symbols exported from asm

[Michal Marek]

On 2016-12-14 10:15, Michal Marek wrote:
> A minimal example would be
> 
> t1.c:
> struct s1;
> struct s2 {
>   int i;
> }
> struct s3 {
>   struct s1 *ptr1;
>   struct s2 *ptr2;
> }
> void foo(struct s3*);
> EXPORT_SYMBOL(foo);
> 
> t2.c:
> struct s1 {
>   int j;
> }
> struct s2;
> struct s3 {
>   struct s1 *ptr1;
>   struct s2 *ptr2;
> }
> void foo(struct s3*);
> EXPORT_SYMBOL(foo);

Note that the above, if passed to genksyms verbatim, would result in
genksyms treating all the types as internal. Here is a complete
example including linemarkers:

$ cat t1.i
# 1 "t1.c"
# 1 ""
# 1 ""
# 1 "t1.c"
# 1 "t1.h" 1
# 1 "t.h" 1
struct s1;
struct s2;
struct s3 {
 struct s1 *ptr1;
 struct s2 *ptr2;
};
# 2 "t1.h" 2
struct s2 {
 int i;
};
# 2 "t1.c" 2
void foo(struct s3 *s) { }
EXPORT_SYMBOL(foo);

$ cat t2.i
# 1 "t2.c"
# 1 ""
# 1 ""
# 1 "t2.c"
# 1 "t2.h" 1
# 1 "t.h" 1
struct s1;
struct s2;
struct s3 {
 struct s1 *ptr1;
 struct s2 *ptr2;
};
# 2 "t2.h" 2
struct s1 {
 int j;
};
# 2 "t2.c" 2
void foo(struct s3 *s) { }
EXPORT_SYMBOL(foo);

$ ./scripts/genksyms/genksyms -D 
__crc_foo = 0xf731cef8 ;

$ ./scripts/genksyms/genksyms -D 
__crc_foo = 0xc925dae5 ;

Michal

Re: [PATCH] x86/kbuild: enable modversions for symbols exported from asm

[Michal Marek]

On 2016-12-14 11:02, Dodji Seketeli wrote:
> Michal Marek <mma...@suse.com> a écrit:
> 
>>> Libabigail does a "whole binary" analysis of types.
>>>
>>> So, consider the point of use of the type 'struct s1*'.  Even if 'struct
>>> s' is just forward-declared at that point, the declaration of struct s1
>>> is "resolved" to its definition.  Even if the definition comes later in
>>> the binary.
>>
>> But there isn't any definition of struct s1 in t1.o. Does abidiff
>> "steal" the definition from the other object file? That would be
>> legitimate, I'm just curious.
> 
> If there is another translation unit in the *same* binary that defines
> struct s1, then yes, it's "stolen", as you say.
> 
> But if in the entire binary, struct s1 is just declared (not defined),
> then it'll compare equal to any struct s1 that is defined in the
> *second* binary.

That makes sense, thanks.

Michal

Re: [PATCH] x86/kbuild: enable modversions for symbols exported from asm

[Michal Marek]

On 2016-12-14 11:02, Dodji Seketeli wrote:
> Michal Marek  a écrit:
> 
>>> Libabigail does a "whole binary" analysis of types.
>>>
>>> So, consider the point of use of the type 'struct s1*'.  Even if 'struct
>>> s' is just forward-declared at that point, the declaration of struct s1
>>> is "resolved" to its definition.  Even if the definition comes later in
>>> the binary.
>>
>> But there isn't any definition of struct s1 in t1.o. Does abidiff
>> "steal" the definition from the other object file? That would be
>> legitimate, I'm just curious.
> 
> If there is another translation unit in the *same* binary that defines
> struct s1, then yes, it's "stolen", as you say.
> 
> But if in the entire binary, struct s1 is just declared (not defined),
> then it'll compare equal to any struct s1 that is defined in the
> *second* binary.

That makes sense, thanks.

Michal

Re: [PATCH] x86/kbuild: enable modversions for symbols exported from asm

[Michal Marek]

On 2016-12-14 10:36, Dodji Seketeli wrote:
> Michal Marek <mma...@suse.com> a écrit:
> 
> [...]
> 
>> A minimal example would be
>>
>> t1.c:
>> struct s1;
>> struct s2 {
>>  int i;
>> }
>> struct s3 {
>>  struct s1 *ptr1;
>>  struct s2 *ptr2;
>> }
>> void foo(struct s3*);
>> EXPORT_SYMBOL(foo);
>>
>> t2.c:
>> struct s1 {
>>  int j;
>> }
>> struct s2;
>> struct s3 {
>>  struct s1 *ptr1;
>>  struct s2 *ptr2;
>> }
>> void foo(struct s3*);
>> EXPORT_SYMBOL(foo);
>>
>> genksyms expands this to
>> void foo ( struct s3 { struct s1 { UNKNOWN } * ptr1 ; struct s2 { int i ; } 
>> * ptr2 ; } * )
>>
>> or
>>
>> void foo ( struct s3 { struct s1 { int j ; } * ptr1 ; struct s2 { UNKNOWN } 
>> * ptr2 ; } * )
>> respectively.
> 
> Thanks, I have built an independant test case from this:
> 
> $ cat t1.c
> struct s1;
> struct s2 {
>   int i;
> };
> struct s3 {
>   struct s1 *ptr1;
>   struct s2 *ptr2;
> };
> void foo(struct s3*);
> $ cat t2.c
> struct s1 {
>   int j;
> };
> struct s2;
> struct s3 {
>   struct s1 *ptr1;
>   struct s2 *ptr2;
> };
> void foo(struct s3*);
> $ gcc -g -c t1.c
> $ gcc -g -c t2.c
> $ abidiff t1.o t2.o
> $ 
> 
> So, as you see here, abidiff considers t1.o and t2.o has having the same
> ABI, so it considers the two foo functions to be equivalent.

Wow. That sounds too good to be true.


>> The types are the same, but their visibility in the different
>> compilation units differs.
> 
> I see, for genksyms, the order of declarations matters, especially when
> forward declarations are involved.
> 
> Libabigail does a "whole binary" analysis of types.
> 
> So, consider the point of use of the type 'struct s1*'.  Even if 'struct
> s' is just forward-declared at that point, the declaration of struct s1
> is "resolved" to its definition.  Even if the definition comes later in
> the binary.

But there isn't any definition of struct s1 in t1.o. Does abidiff
"steal" the definition from the other object file? That would be
legitimate, I'm just curious.

Thanks,
Michal

Re: [PATCH] x86/kbuild: enable modversions for symbols exported from asm

[Michal Marek]

On 2016-12-14 10:36, Dodji Seketeli wrote:
> Michal Marek  a écrit:
> 
> [...]
> 
>> A minimal example would be
>>
>> t1.c:
>> struct s1;
>> struct s2 {
>>  int i;
>> }
>> struct s3 {
>>  struct s1 *ptr1;
>>  struct s2 *ptr2;
>> }
>> void foo(struct s3*);
>> EXPORT_SYMBOL(foo);
>>
>> t2.c:
>> struct s1 {
>>  int j;
>> }
>> struct s2;
>> struct s3 {
>>  struct s1 *ptr1;
>>  struct s2 *ptr2;
>> }
>> void foo(struct s3*);
>> EXPORT_SYMBOL(foo);
>>
>> genksyms expands this to
>> void foo ( struct s3 { struct s1 { UNKNOWN } * ptr1 ; struct s2 { int i ; } 
>> * ptr2 ; } * )
>>
>> or
>>
>> void foo ( struct s3 { struct s1 { int j ; } * ptr1 ; struct s2 { UNKNOWN } 
>> * ptr2 ; } * )
>> respectively.
> 
> Thanks, I have built an independant test case from this:
> 
> $ cat t1.c
> struct s1;
> struct s2 {
>   int i;
> };
> struct s3 {
>   struct s1 *ptr1;
>   struct s2 *ptr2;
> };
> void foo(struct s3*);
> $ cat t2.c
> struct s1 {
>   int j;
> };
> struct s2;
> struct s3 {
>   struct s1 *ptr1;
>   struct s2 *ptr2;
> };
> void foo(struct s3*);
> $ gcc -g -c t1.c
> $ gcc -g -c t2.c
> $ abidiff t1.o t2.o
> $ 
> 
> So, as you see here, abidiff considers t1.o and t2.o has having the same
> ABI, so it considers the two foo functions to be equivalent.

Wow. That sounds too good to be true.


>> The types are the same, but their visibility in the different
>> compilation units differs.
> 
> I see, for genksyms, the order of declarations matters, especially when
> forward declarations are involved.
> 
> Libabigail does a "whole binary" analysis of types.
> 
> So, consider the point of use of the type 'struct s1*'.  Even if 'struct
> s' is just forward-declared at that point, the declaration of struct s1
> is "resolved" to its definition.  Even if the definition comes later in
> the binary.

But there isn't any definition of struct s1 in t1.o. Does abidiff
"steal" the definition from the other object file? That would be
legitimate, I'm just curious.

Thanks,
Michal

Re: [PATCH] x86/kbuild: enable modversions for symbols exported from asm

[Michal Marek]

On 2016-12-14 09:58, Dodji Seketeli wrote:
> Michal Marek <mma...@suse.com> a écrit:
> 
> [...]
> 
>> Does the abidiff tool handle the case when an exported symbol is moved
>> between .c files? This is always a mess with genksyms, because the two
>> .c files have different includes and thus the type expansion stops at
>> different points. So typically the move needs to be reverted as a
>> workaround.
> 
> Let's consider the function:
> 
>   'void foo(struct S*);'
> 
> If two ELF binaries contain a definition of that function foo which ELF
> symbol is exported, if the type struct S hasn't changed, and if the only
> difference between the ELF binaries is that foo was defined in the
> translation unit a.c in the first binary and in b.c in the second
> binary, then the comparison engine of libabigail (which is the library
> that abidiff uses) will consider the declarations of the two foo
> functions as being equal -- no matter what include file comes before the
> definition point of foo in a.c and b.c.  If it does not, then it's a bug
> that ought to be fixed.
> 
> If you feel that I haven't understood your question, then I guess a
> minimal standalone example (in the form of C source code) that
> illustrates your use case could be helpful to me.

A minimal example would be

t1.c:
struct s1;
struct s2 {
int i;
}
struct s3 {
struct s1 *ptr1;
struct s2 *ptr2;
}
void foo(struct s3*);
EXPORT_SYMBOL(foo);

t2.c:
struct s1 {
int j;
}
struct s2;
struct s3 {
struct s1 *ptr1;
struct s2 *ptr2;
}
void foo(struct s3*);
EXPORT_SYMBOL(foo);

genksyms expands this to
void foo ( struct s3 { struct s1 { UNKNOWN } * ptr1 ; struct s2 { int i ; } * 
ptr2 ; } * )

or

void foo ( struct s3 { struct s1 { int j ; } * ptr1 ; struct s2 { UNKNOWN } * 
ptr2 ; } * )

respectively. The types are the same, but their visibility in the
different compilation units differs.

Michal

Re: [PATCH] x86/kbuild: enable modversions for symbols exported from asm

[Michal Marek]

On 2016-12-14 09:58, Dodji Seketeli wrote:
> Michal Marek  a écrit:
> 
> [...]
> 
>> Does the abidiff tool handle the case when an exported symbol is moved
>> between .c files? This is always a mess with genksyms, because the two
>> .c files have different includes and thus the type expansion stops at
>> different points. So typically the move needs to be reverted as a
>> workaround.
> 
> Let's consider the function:
> 
>   'void foo(struct S*);'
> 
> If two ELF binaries contain a definition of that function foo which ELF
> symbol is exported, if the type struct S hasn't changed, and if the only
> difference between the ELF binaries is that foo was defined in the
> translation unit a.c in the first binary and in b.c in the second
> binary, then the comparison engine of libabigail (which is the library
> that abidiff uses) will consider the declarations of the two foo
> functions as being equal -- no matter what include file comes before the
> definition point of foo in a.c and b.c.  If it does not, then it's a bug
> that ought to be fixed.
> 
> If you feel that I haven't understood your question, then I guess a
> minimal standalone example (in the form of C source code) that
> illustrates your use case could be helpful to me.

A minimal example would be

t1.c:
struct s1;
struct s2 {
int i;
}
struct s3 {
struct s1 *ptr1;
struct s2 *ptr2;
}
void foo(struct s3*);
EXPORT_SYMBOL(foo);

t2.c:
struct s1 {
int j;
}
struct s2;
struct s3 {
struct s1 *ptr1;
struct s2 *ptr2;
}
void foo(struct s3*);
EXPORT_SYMBOL(foo);

genksyms expands this to
void foo ( struct s3 { struct s1 { UNKNOWN } * ptr1 ; struct s2 { int i ; } * 
ptr2 ; } * )

or

void foo ( struct s3 { struct s1 { int j ; } * ptr1 ; struct s2 { UNKNOWN } * 
ptr2 ; } * )

respectively. The types are the same, but their visibility in the
different compilation units differs.

Michal

Re: [PATCH v3 1/1] x86/kbuild: enable modversions for symbols exported from asm

[Michal Marek]

Dne 11.12.2016 v 02:09 Adam Borowski napsal(a):
> Commit 4efca4ed ("kbuild: modversions for EXPORT_SYMBOL() for asm") adds
> modversion support for symbols exported from asm files. Architectures
> must include C-style declarations for those symbols in asm/asm-prototypes.h
> in order for them to be versioned.
> 
> Add these declarations for x86, and an architecture-independent file that
> can be used for common symbols.
> 
> With f27c2f6 reverting 8ab2ae6 ("default exported asm symbols to zero") we
> produce a scary warning on x86, this commit fixes that.
> 
> Signed-off-by: Adam Borowski 
> Tested-by: Kalle Valo 
> Acked-by: Nicholas Piggin 
> Tested-by: Peter Wu 
> Tested-by: Oliver Hartkopp 

Applied to kbuild.git#kbuild now.

Michal

Re: [PATCH v3 1/1] x86/kbuild: enable modversions for symbols exported from asm

[Michal Marek]

Dne 11.12.2016 v 02:09 Adam Borowski napsal(a):
> Commit 4efca4ed ("kbuild: modversions for EXPORT_SYMBOL() for asm") adds
> modversion support for symbols exported from asm files. Architectures
> must include C-style declarations for those symbols in asm/asm-prototypes.h
> in order for them to be versioned.
> 
> Add these declarations for x86, and an architecture-independent file that
> can be used for common symbols.
> 
> With f27c2f6 reverting 8ab2ae6 ("default exported asm symbols to zero") we
> produce a scary warning on x86, this commit fixes that.
> 
> Signed-off-by: Adam Borowski 
> Tested-by: Kalle Valo 
> Acked-by: Nicholas Piggin 
> Tested-by: Peter Wu 
> Tested-by: Oliver Hartkopp 

Applied to kbuild.git#kbuild now.

Michal

Re: [PATCH 2/2] kbuild: make modversion for exported asm symbols more convivial

[Michal Marek]

Dne 8.12.2016 v 05:24 Nicolas Pitre napsal(a):
> Rather than having an asm-prototypes.h file where C prototypes for exported
> asm symbols are centralized, let's have some macros that can be used
> directly in the code where those symbols are exported for genksyms
> consumption.  Either the prototype is provided directly if no include
> files has it, or the include file containing it is specified.
> 
> Signed-off-by: Nicolas Pitre 
> ---
>  include/asm-generic/export.h | 15 +++
>  scripts/Makefile.build   | 22 +++---
>  2 files changed, 30 insertions(+), 7 deletions(-)
> 
> diff --git a/include/asm-generic/export.h b/include/asm-generic/export.h
> index 39a19dc366..83dda5f840 100644
> --- a/include/asm-generic/export.h
> +++ b/include/asm-generic/export.h
> @@ -84,11 +84,26 @@ KSYM(__kcrctab_\name):
>  #define __EXPORT_SYMBOL(sym, val, sec) ___EXPORT_SYMBOL sym, val, sec
>  #endif
>  
> +/* in the non genksyms case those are no-ops */
> +#define SYMBOL_CPROTO(expr)
> +#define SYMBOL_CPROTO_INCLUDE(file)

Do we really _need_ the SYMBOL_CPROTO macro? The exported functions are
called from C files presumably, so there ought to be headers with the
declarations. If these headers can't be included easily, we should fix
them, but having copies of the declarations in the asm files is no big
improvement over the asm-prototypes.h approach, IMO.

Michal

Re: [PATCH 2/2] kbuild: make modversion for exported asm symbols more convivial

[Michal Marek]

Dne 8.12.2016 v 05:24 Nicolas Pitre napsal(a):
> Rather than having an asm-prototypes.h file where C prototypes for exported
> asm symbols are centralized, let's have some macros that can be used
> directly in the code where those symbols are exported for genksyms
> consumption.  Either the prototype is provided directly if no include
> files has it, or the include file containing it is specified.
> 
> Signed-off-by: Nicolas Pitre 
> ---
>  include/asm-generic/export.h | 15 +++
>  scripts/Makefile.build   | 22 +++---
>  2 files changed, 30 insertions(+), 7 deletions(-)
> 
> diff --git a/include/asm-generic/export.h b/include/asm-generic/export.h
> index 39a19dc366..83dda5f840 100644
> --- a/include/asm-generic/export.h
> +++ b/include/asm-generic/export.h
> @@ -84,11 +84,26 @@ KSYM(__kcrctab_\name):
>  #define __EXPORT_SYMBOL(sym, val, sec) ___EXPORT_SYMBOL sym, val, sec
>  #endif
>  
> +/* in the non genksyms case those are no-ops */
> +#define SYMBOL_CPROTO(expr)
> +#define SYMBOL_CPROTO_INCLUDE(file)

Do we really _need_ the SYMBOL_CPROTO macro? The exported functions are
called from C files presumably, so there ought to be headers with the
declarations. If these headers can't be included easily, we should fix
them, but having copies of the declarations in the asm files is no big
improvement over the asm-prototypes.h approach, IMO.

Michal

Re: [PATCH] x86/kbuild: enable modversions for symbols exported from asm

[Michal Marek]

Dne 9.12.2016 v 23:46 Dodji Seketeli napsal(a):
> Hello,
> 
> Nicholas Piggin  a écrit:
> 
> [...]
> 
>> That said, a dwarf based checker tool should be able to do as good a job
>> (maybe a bit better because report is very informative and it may pick up
>> compiler alignments or padding options).
> 
> So, Nicholas was kind enough to send me the two Linux Kernel binaries
> that he built with the tiny little interface change that we were
> discussing earlier.  Here is what the abidiff[1] tools says about that
> interface change:
> 
> $ time ~/git/libabigail/kabidiff/build/tools/abidiff vmlinux.abi1.abi 
> vmlinux.abi2.abi
> Functions changes summary: 0 Removed, 1 Changed, 0 Added function
> Variables changes summary: 0 Removed, 0 Changed, 0 Added variable
> 
> 1 function with some indirect sub-type change:
> 
>   [C]'function int foo(blah*)' at memory.c:82:1 has some indirect 
> sub-type changes:
> parameter 1 of type 'blah*' has sub-type changes:
>   in pointed to type 'struct blah' at memory.c:78:1:
> type size changed from 32 to 64 bits
> 1 data member insertion:
>   'int blah::y', at offset 0 (in bits) at memory.c:79:1
> 1 data member change:
>  'int blah::x' offset changed from 0 to 32 (in bits) (by +32 bits)

For completeness, with a foo.symref file in the tree, genksyms would print

foo.c: warning: foo: modversion changed because of changes in struct blah

So there is some sort of diagnostics already. Does the abidiff tool
handle the case when an exported symbol is moved between .c files? This
is always a mess with genksyms, because the two .c files have different
includes and thus the type expansion stops at different points. So
typically the move needs to be reverted as a workaround.

Michal

Re: [PATCH] x86/kbuild: enable modversions for symbols exported from asm

[Michal Marek]

Dne 9.12.2016 v 23:46 Dodji Seketeli napsal(a):
> Hello,
> 
> Nicholas Piggin  a écrit:
> 
> [...]
> 
>> That said, a dwarf based checker tool should be able to do as good a job
>> (maybe a bit better because report is very informative and it may pick up
>> compiler alignments or padding options).
> 
> So, Nicholas was kind enough to send me the two Linux Kernel binaries
> that he built with the tiny little interface change that we were
> discussing earlier.  Here is what the abidiff[1] tools says about that
> interface change:
> 
> $ time ~/git/libabigail/kabidiff/build/tools/abidiff vmlinux.abi1.abi 
> vmlinux.abi2.abi
> Functions changes summary: 0 Removed, 1 Changed, 0 Added function
> Variables changes summary: 0 Removed, 0 Changed, 0 Added variable
> 
> 1 function with some indirect sub-type change:
> 
>   [C]'function int foo(blah*)' at memory.c:82:1 has some indirect 
> sub-type changes:
> parameter 1 of type 'blah*' has sub-type changes:
>   in pointed to type 'struct blah' at memory.c:78:1:
> type size changed from 32 to 64 bits
> 1 data member insertion:
>   'int blah::y', at offset 0 (in bits) at memory.c:79:1
> 1 data member change:
>  'int blah::x' offset changed from 0 to 32 (in bits) (by +32 bits)

For completeness, with a foo.symref file in the tree, genksyms would print

foo.c: warning: foo: modversion changed because of changes in struct blah

So there is some sort of diagnostics already. Does the abidiff tool
handle the case when an exported symbol is moved between .c files? This
is always a mess with genksyms, because the two .c files have different
includes and thus the type expansion stops at different points. So
typically the move needs to be reverted as a workaround.

Michal

Re: [PATCH] tags: honor COMPILED_SOURCE with apart output directory

[Michal Marek]

Dne 12.12.2016 v 18:13 Robert Jarzmik napsal(a):
> Michal Marek <mma...@suse.com> writes:
>> Please use the KBUILD_OUTPUT variable and not O.
> Well, I can't as far as I know. I tried, see below ...
> 
> Explanation :
>  - if I add just below the line "j=${i/\.[cS]/\.o}" the following :
>   k="$O/${j#$tree}"
> echo "tree=$tree; O=$O; KBUILD_OUTPUT=$KBUILD_OUTPUT; i=$i, j=$j, 
> k=$k"
>  - and I launch:
>   make O=out cscope COMPILED_SOURCE=1
>  
> I get these kind of lines :
> tree=../; O=/home/rj/mio_linux/kernel/out; KBUILD_OUTPUT=;
> i=../kernel/sched/core.c, j=../kernel/sched/core.o, k=../kernel/sched/core.o
> 
> From here I understand that :
>  - $KBUILD_OUTPUT is not usable

Hm, you are right, we never export $KBUILD_OUTPUT anywhere. But, we
actualy chdir into the build tree before doing anything. So you can
assume that if $KBUILD_SRC != "", the build tree is ".".


>> should only match at the beginning of the filename (so use something
>> like ${i#$tree}).
> Ok, I can use then : k="$O/${j#$tree}". The subtle part is when O is empty, in
> which case this returns /, which doesn't look nice, while the former
> expression returned either a substituted path or the source path.

... so you can simply strip the leading $tree here and obtain the path
in the build directory.

Michal

Re: [PATCH] tags: honor COMPILED_SOURCE with apart output directory

[Michal Marek]

Dne 12.12.2016 v 18:13 Robert Jarzmik napsal(a):
> Michal Marek  writes:
>> Please use the KBUILD_OUTPUT variable and not O.
> Well, I can't as far as I know. I tried, see below ...
> 
> Explanation :
>  - if I add just below the line "j=${i/\.[cS]/\.o}" the following :
>   k="$O/${j#$tree}"
> echo "tree=$tree; O=$O; KBUILD_OUTPUT=$KBUILD_OUTPUT; i=$i, j=$j, 
> k=$k"
>  - and I launch:
>   make O=out cscope COMPILED_SOURCE=1
>  
> I get these kind of lines :
> tree=../; O=/home/rj/mio_linux/kernel/out; KBUILD_OUTPUT=;
> i=../kernel/sched/core.c, j=../kernel/sched/core.o, k=../kernel/sched/core.o
> 
> From here I understand that :
>  - $KBUILD_OUTPUT is not usable

Hm, you are right, we never export $KBUILD_OUTPUT anywhere. But, we
actualy chdir into the build tree before doing anything. So you can
assume that if $KBUILD_SRC != "", the build tree is ".".


>> should only match at the beginning of the filename (so use something
>> like ${i#$tree}).
> Ok, I can use then : k="$O/${j#$tree}". The subtle part is when O is empty, in
> which case this returns /, which doesn't look nice, while the former
> expression returned either a substituted path or the source path.

... so you can simply strip the leading $tree here and obtain the path
in the build directory.

Michal

Re: [PATCH] tags: honor COMPILED_SOURCE with apart output directory

[Michal Marek]

Dne 6.12.2016 v 12:54 Robert Jarzmik napsal(a):
> Robert Jarzmik  writes:
> 
>> When the kernel is compiled with an "O=" argument, the object files are
>> not necessarily in the source tree, and more probably in another tree.
>>
>> In this situation, the current used check doesn't work, and
>> COMPILED_SOURCE tags is broken with O= builds.
>>
>> This patch fixes it by looking for object files both in source tree and
>> potential destination tree.
>>
>> Signed-off-by: Robert Jarzmik 
> 
> Hi Marek, ping about this patch ?

Sorry, I missed the patch.

>> --- a/scripts/tags.sh
>> +++ b/scripts/tags.sh
>> @@ -106,7 +106,9 @@ all_compiled_sources()
>>  case "$i" in
>>  *.[cS])
>>  j=${i/\.[cS]/\.o}
>> -if [ -e $j ]; then
>> +k=${i/"$tree"/"$O"\/}
>> +k=${k/\.[cS]/\.o}
>> +if [ -e $j -o -e "$k" ]; then

Please use the KBUILD_OUTPUT variable and not O. Also, the $tree prefix
should only match at the beginning of the filename (so use something
like ${i#$tree}). Last, but not least, the .[cS] -> .o substitution only
needs to be done once. The k variable can use the value of j instead of i.

Thanks,
Michal

Re: [PATCH] tags: honor COMPILED_SOURCE with apart output directory

[Michal Marek]

Dne 6.12.2016 v 12:54 Robert Jarzmik napsal(a):
> Robert Jarzmik  writes:
> 
>> When the kernel is compiled with an "O=" argument, the object files are
>> not necessarily in the source tree, and more probably in another tree.
>>
>> In this situation, the current used check doesn't work, and
>> COMPILED_SOURCE tags is broken with O= builds.
>>
>> This patch fixes it by looking for object files both in source tree and
>> potential destination tree.
>>
>> Signed-off-by: Robert Jarzmik 
> 
> Hi Marek, ping about this patch ?

Sorry, I missed the patch.

>> --- a/scripts/tags.sh
>> +++ b/scripts/tags.sh
>> @@ -106,7 +106,9 @@ all_compiled_sources()
>>  case "$i" in
>>  *.[cS])
>>  j=${i/\.[cS]/\.o}
>> -if [ -e $j ]; then
>> +k=${i/"$tree"/"$O"\/}
>> +k=${k/\.[cS]/\.o}
>> +if [ -e $j -o -e "$k" ]; then

Please use the KBUILD_OUTPUT variable and not O. Also, the $tree prefix
should only match at the beginning of the filename (so use something
like ${i#$tree}). Last, but not least, the .[cS] -> .o substitution only
needs to be done once. The k variable can use the value of j instead of i.

Thanks,
Michal

Re: [PATCH] scripts/kallsyms: remove last remnants of --page-offset option

[Michal Marek]

On Fri, Nov 25, 2016 at 08:43:55AM +, Ard Biesheuvel wrote:
> On 28 October 2016 at 18:09, Ard Biesheuvel  wrote:
> > The implementation of the --page-offset kallsyms command line option has
> > been removed, so remove it from the usage string as well.
> >
> > Signed-off-by: Ard Biesheuvel 
> > ---
> >  scripts/kallsyms.c | 1 -
> >  1 file changed, 1 deletion(-)
> >
> > diff --git a/scripts/kallsyms.c b/scripts/kallsyms.c
> > index 1f22a186c18c..299b92ca1ae0 100644
> > --- a/scripts/kallsyms.c
> > +++ b/scripts/kallsyms.c
> > @@ -76,7 +76,6 @@ static void usage(void)
> >  {
> > fprintf(stderr, "Usage: kallsyms [--all-symbols] "
> > "[--symbol-prefix=] "
> > -   "[--page-offset=] "
> > "[--base-relative] < in.map > out.S\n");
> > exit(1);
> >  }
> > --
> > 2.7.4
> >
> 
> Ping?

Pong, sorry. Applied to kbuild.git#kbuild now.

Michal

Re: [PATCH] scripts/kallsyms: remove last remnants of --page-offset option

[Michal Marek]

On Fri, Nov 25, 2016 at 08:43:55AM +, Ard Biesheuvel wrote:
> On 28 October 2016 at 18:09, Ard Biesheuvel  wrote:
> > The implementation of the --page-offset kallsyms command line option has
> > been removed, so remove it from the usage string as well.
> >
> > Signed-off-by: Ard Biesheuvel 
> > ---
> >  scripts/kallsyms.c | 1 -
> >  1 file changed, 1 deletion(-)
> >
> > diff --git a/scripts/kallsyms.c b/scripts/kallsyms.c
> > index 1f22a186c18c..299b92ca1ae0 100644
> > --- a/scripts/kallsyms.c
> > +++ b/scripts/kallsyms.c
> > @@ -76,7 +76,6 @@ static void usage(void)
> >  {
> > fprintf(stderr, "Usage: kallsyms [--all-symbols] "
> > "[--symbol-prefix=] "
> > -   "[--page-offset=] "
> > "[--base-relative] < in.map > out.S\n");
> > exit(1);
> >  }
> > --
> > 2.7.4
> >
> 
> Ping?

Pong, sorry. Applied to kbuild.git#kbuild now.

Michal

Re: [PATCH v4 0/3] Coccinelle: misc: Improve the script for more accurate results

[Michal Marek]

On Wed, Nov 23, 2016 at 02:12:58PM +0530, Vaishali Thakkar wrote:
> Few changes to improve the results given by the irqf_oneshot.cocci:
> 
> - Change in the matching rules to eliminate false postives in the
>   patch mode
> - Change in the context mode to eliminate false postives in the
>   context mode
> - Support for the missing devm_request_threaded_irq in context, report
>   and org mode
> 
> Changes since v3:
>   - Few more changes in the script to avoid some
>   possible false positives
> - Moved initialization of all expressions in a single line 
> Changes since v2:
>   - Add missing initialization of metavariables
> Changes since v1:
>   - Split patch in to the patchset
> 
> Vaishali Thakkar (3):
>   Coccinelle: misc: Improve the matching of rules
>   Coccinelle: misc: Improve the result given by context mode
>   Coccinelle: misc: Add support for devm variant in all modes

Applied to kbuild.git#misc.

Michal

Re: [PATCH v4 0/3] Coccinelle: misc: Improve the script for more accurate results

[Michal Marek]

On Wed, Nov 23, 2016 at 02:12:58PM +0530, Vaishali Thakkar wrote:
> Few changes to improve the results given by the irqf_oneshot.cocci:
> 
> - Change in the matching rules to eliminate false postives in the
>   patch mode
> - Change in the context mode to eliminate false postives in the
>   context mode
> - Support for the missing devm_request_threaded_irq in context, report
>   and org mode
> 
> Changes since v3:
>   - Few more changes in the script to avoid some
>   possible false positives
> - Moved initialization of all expressions in a single line 
> Changes since v2:
>   - Add missing initialization of metavariables
> Changes since v1:
>   - Split patch in to the patchset
> 
> Vaishali Thakkar (3):
>   Coccinelle: misc: Improve the matching of rules
>   Coccinelle: misc: Improve the result given by context mode
>   Coccinelle: misc: Add support for devm variant in all modes

Applied to kbuild.git#misc.

Michal

Re: [PATCH] kbuild: cmd_export_list: tighten the sed script

[Michal Marek]

On Wed, Nov 16, 2016 at 07:28:39PM -0500, Nicolas Pitre wrote:
> 
> When LTO is used, some ___ksymtab_string sections are seen by this sed
> script, creating lines containing a single ) such as:
> 
> EXPORT(foo)
> )
> )
> EXPORT(bar)
> 
> Let's make it so the + character is also required for any line to be
> printed.
> 
> Signed-off-by: Nicolas Pitre 

Applied to kbuild.git#kbuild.

Michal

Re: [PATCH] kbuild: cmd_export_list: tighten the sed script

[Michal Marek]

On Wed, Nov 16, 2016 at 07:28:39PM -0500, Nicolas Pitre wrote:
> 
> When LTO is used, some ___ksymtab_string sections are seen by this sed
> script, creating lines containing a single ) such as:
> 
> EXPORT(foo)
> )
> )
> EXPORT(bar)
> 
> Let's make it so the + character is also required for any line to be
> printed.
> 
> Signed-off-by: Nicolas Pitre 

Applied to kbuild.git#kbuild.

Michal

Re: [PATCH] kbuild/mkspec: avoid using brace expansion

[Michal Marek]

On Tue, Oct 25, 2016 at 06:00:44PM +0900, Anton Tikhomirov wrote:
> Brace expansion might not work properly if _buildshell RPM macro
> points to a shell other than bash. Particularly, with _bulidshell
> defined to /bin/dash it leads to broken build and source symlinks.
> 
> Signed-off-by: Anton Tikhomirov 
> ---
>  scripts/package/mkspec | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)

Applied to kbuild.git#misc, thanks.

Michal

Re: [PATCH] kbuild/mkspec: avoid using brace expansion

[Michal Marek]

On Tue, Oct 25, 2016 at 06:00:44PM +0900, Anton Tikhomirov wrote:
> Brace expansion might not work properly if _buildshell RPM macro
> points to a shell other than bash. Particularly, with _bulidshell
> defined to /bin/dash it leads to broken build and source symlinks.
> 
> Signed-off-by: Anton Tikhomirov 
> ---
>  scripts/package/mkspec | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)

Applied to kbuild.git#misc, thanks.

Michal

Re: [PATCH v2] Coccinelle: Add misc/boolconv.cocci

[Michal Marek]

On Mon, Oct 17, 2016 at 11:52:24AM -0500, Andrew F. Davis wrote:
> Add a script to check for unneeded conversions to bool.
> 
> Signed-off-by: Andrew F. Davis 
> Acked-by: Julia Lawall 

Applied to kbuild.git#misc, thanks.

Michal

Re: [PATCH v2] Coccinelle: Add misc/boolconv.cocci

[Michal Marek]

On Mon, Oct 17, 2016 at 11:52:24AM -0500, Andrew F. Davis wrote:
> Add a script to check for unneeded conversions to bool.
> 
> Signed-off-by: Andrew F. Davis 
> Acked-by: Julia Lawall 

Applied to kbuild.git#misc, thanks.

Michal

Re: [PATCH v4] Fixes for compiling with clang

[Michal Marek]

On 2016-12-01 19:13, Peter Foley wrote:
> On Tue, Nov 29, 2016 at 6:22 AM, Michal Marek <mma...@suse.com> wrote:
>> Dne 28.11.2016 v 07:44 Peter Foley napsal(a):
>> This adds new -Wno-* options also for the gcc case, is there a reason
>> for this? Also, the -Wno-missing-field-initializers option is not
>> available in some old gccs, so we would need a HOSTCC equivalent of
>> cc-disable-warning.
>>
>> Michal
> 
> It appeared that the conditional was simply reversed, as
> -fno-delete-null-pointer-checks is only supported by gcc, and
> explicitly not supported by clang.
> (see 
> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/Makefile?id=61163efae02040f66a95c8ed17f4407951ba58fa)
> It could be that the fno-delete-null-pointer-checks option was simply
> misplaced, and the Wno-options should still be guarded by if(clang),
> would that be a better approach?

I think so. The -Wno-* options had not been present before clang support
was added. It really looks like the -fno-delete-null-pointer-checks
should not be there. Added Behan to CC.

Michal

Re: [PATCH v4] Fixes for compiling with clang

[Michal Marek]

On 2016-12-01 19:13, Peter Foley wrote:
> On Tue, Nov 29, 2016 at 6:22 AM, Michal Marek  wrote:
>> Dne 28.11.2016 v 07:44 Peter Foley napsal(a):
>> This adds new -Wno-* options also for the gcc case, is there a reason
>> for this? Also, the -Wno-missing-field-initializers option is not
>> available in some old gccs, so we would need a HOSTCC equivalent of
>> cc-disable-warning.
>>
>> Michal
> 
> It appeared that the conditional was simply reversed, as
> -fno-delete-null-pointer-checks is only supported by gcc, and
> explicitly not supported by clang.
> (see 
> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/Makefile?id=61163efae02040f66a95c8ed17f4407951ba58fa)
> It could be that the fno-delete-null-pointer-checks option was simply
> misplaced, and the Wno-options should still be guarded by if(clang),
> would that be a better approach?

I think so. The -Wno-* options had not been present before clang support
was added. It really looks like the -fno-delete-null-pointer-checks
should not be there. Added Behan to CC.

Michal

Re: [PATCH] x86/kbuild: enable modversions for symbols exported from asm

[Michal Marek]

On 2016-12-01 14:58, Arnd Bergmann wrote:
> On Tuesday, November 29, 2016 9:14:46 AM CET Linus Torvalds wrote:
>> On Tue, Nov 29, 2016 at 9:10 AM, Linus Torvalds
>>  wrote:
>>>
>>> So quite frankly, I don't want to make our kernel sources worse due to
>>> broken shit tools getting something wrong that we shouldn't even care
>>> about.
>>
>> And yes, I'm on binutils 2.26 (with no issues), so it could be that
>> it's 2.27 that triggers this.
>>
>> We could make the pr_warn_once() mention "broken binutils?" so that
>> people know why the warning happens.
> 
> I've tried to get to the bottom of this, but couldn't find anything
> related to the toolchain version. I've tried binutils 2.23, 2.24, 2.26
> and 2.27, and also gcc-7.0, gcc-5.4.1 and gcc-4.9.3, but with today's
> linux-next, I always get
> 
> WARNING: EXPORT symbol "mcount" [arch/x86/entry/built-in.ko] version 
> generation failed, symbol will not be versioned.
> WARNING: EXPORT symbol "mcount" [arch/x86/built-in.ko] version generation 
> failed, symbol will not be versioned.
> WARNING: EXPORT symbol "cmpxchg8b_emu" [vmlinux] version generation failed, 
> symbol will not be versioned.
> WARNING: EXPORT symbol "empty_zero_page" [vmlinux] version generation failed, 
> symbol will not be versioned.
> WARNING: EXPORT symbol "mcount" [vmlinux] version generation failed, symbol 
> will not be versioned.
> WARNING: EXPORT symbol "cmpxchg8b_emu" [vmlinux] version generation failed, 
> symbol will not be versioned.
> WARNING: EXPORT symbol "empty_zero_page" [vmlinux] version generation failed, 
> symbol will not be versioned.
> WARNING: EXPORT symbol "mcount" [vmlinux] version generation failed, symbol 
> will not be versioned.
> 
> Out of 12 randconfig builds that had CONFIG_MODVERSIONS enabled, all 12
> had this problem, though not always with all the symbols.

That we are not generating modversions for the asm exports is a known
problem, independent of the toolchain. The problem with some toolchains
(presumably, because that's the next thing to blame if the source and
the .config is identical) is that the CRCs do not appear as 0 in the
___kcrctab/___kcrctab_gpl section.

Michal

Re: [PATCH] x86/kbuild: enable modversions for symbols exported from asm

[Michal Marek]

On 2016-12-01 14:58, Arnd Bergmann wrote:
> On Tuesday, November 29, 2016 9:14:46 AM CET Linus Torvalds wrote:
>> On Tue, Nov 29, 2016 at 9:10 AM, Linus Torvalds
>>  wrote:
>>>
>>> So quite frankly, I don't want to make our kernel sources worse due to
>>> broken shit tools getting something wrong that we shouldn't even care
>>> about.
>>
>> And yes, I'm on binutils 2.26 (with no issues), so it could be that
>> it's 2.27 that triggers this.
>>
>> We could make the pr_warn_once() mention "broken binutils?" so that
>> people know why the warning happens.
> 
> I've tried to get to the bottom of this, but couldn't find anything
> related to the toolchain version. I've tried binutils 2.23, 2.24, 2.26
> and 2.27, and also gcc-7.0, gcc-5.4.1 and gcc-4.9.3, but with today's
> linux-next, I always get
> 
> WARNING: EXPORT symbol "mcount" [arch/x86/entry/built-in.ko] version 
> generation failed, symbol will not be versioned.
> WARNING: EXPORT symbol "mcount" [arch/x86/built-in.ko] version generation 
> failed, symbol will not be versioned.
> WARNING: EXPORT symbol "cmpxchg8b_emu" [vmlinux] version generation failed, 
> symbol will not be versioned.
> WARNING: EXPORT symbol "empty_zero_page" [vmlinux] version generation failed, 
> symbol will not be versioned.
> WARNING: EXPORT symbol "mcount" [vmlinux] version generation failed, symbol 
> will not be versioned.
> WARNING: EXPORT symbol "cmpxchg8b_emu" [vmlinux] version generation failed, 
> symbol will not be versioned.
> WARNING: EXPORT symbol "empty_zero_page" [vmlinux] version generation failed, 
> symbol will not be versioned.
> WARNING: EXPORT symbol "mcount" [vmlinux] version generation failed, symbol 
> will not be versioned.
> 
> Out of 12 randconfig builds that had CONFIG_MODVERSIONS enabled, all 12
> had this problem, though not always with all the symbols.

That we are not generating modversions for the asm exports is a known
problem, independent of the toolchain. The problem with some toolchains
(presumably, because that's the next thing to blame if the source and
the .config is identical) is that the CRCs do not appear as 0 in the
___kcrctab/___kcrctab_gpl section.

Michal

Re: [PATCH] x86/kbuild: enable modversions for symbols exported from asm

[Michal Marek]

On 2016-12-01 05:13, Don Zickus wrote:
> Sorry for chiming in late, but yes RHEL is a big user of MODVERSIONS for our
> kabi protection work.  Despite our best intentions we still have lots of
> partners and customers that provide value-add out-of-tree drivers to their
> customers.  These module builders requested we have a mechanism to allow
> rolling modules forward for each of our minor RHEL updates without breaking
> their drivers.

FWIW, in SLES we use CONFIG_MODVERSION for pretty much the same reasons
you listed. We also enable it in openSUSE, but there it's not as crucial.

Michal

Re: [PATCH] x86/kbuild: enable modversions for symbols exported from asm

[Michal Marek]

On 2016-12-01 05:13, Don Zickus wrote:
> Sorry for chiming in late, but yes RHEL is a big user of MODVERSIONS for our
> kabi protection work.  Despite our best intentions we still have lots of
> partners and customers that provide value-add out-of-tree drivers to their
> customers.  These module builders requested we have a mechanism to allow
> rolling modules forward for each of our minor RHEL updates without breaking
> their drivers.

FWIW, in SLES we use CONFIG_MODVERSION for pretty much the same reasons
you listed. We also enable it in openSUSE, but there it's not as crucial.

Michal

Re: [PATCH] x86/kbuild: enable modversions for symbols exported from asm

[Michal Marek]

On 2016-12-01 04:39, Nicholas Piggin wrote:
> On Thu, 01 Dec 2016 02:35:54 +
> Ben Hutchings  wrote:
>> As I understand it, genksyms incorporates the definitions of a
>> function's parameter and return types - not just their names - and all
>> the types they refer to, recursively.  So a structure size change
>> should change the version of all functions where the function and its
>> caller pass that structure between them, however indirectly.  It finds
>> such indirect ABI breakage for me fairly regularly, though of course I
>> don't know that it finds everything.
> 
> It is only the type name.
> 
> Not only that but even if you did extend it further to structure type
> arrangement then you still have to deal with other structures followed
> via pointers. Or (rarer but not unheard of):
> 
> - changes to structures without changes of the types of their members
> - changes to arguments without changes of their type

This is already covered by genksyms. Try make V=1 with
CONFIG_MODVERSIONS=y and add the -D option to one of the genksyms
command. I wanted to paste the expanded signature for
register_filesystem() as an example, but vger would probably drop the
mail for being too big :).


> - changes to semantics of functions
> - data structures derived in ways other than exported symbols, e.g.,
>   fixed register for `current` on some archs

Right, this is something that genksyms has no idea about.

Michal

Re: [PATCH] x86/kbuild: enable modversions for symbols exported from asm

[Michal Marek]

On 2016-12-01 04:39, Nicholas Piggin wrote:
> On Thu, 01 Dec 2016 02:35:54 +
> Ben Hutchings  wrote:
>> As I understand it, genksyms incorporates the definitions of a
>> function's parameter and return types - not just their names - and all
>> the types they refer to, recursively.  So a structure size change
>> should change the version of all functions where the function and its
>> caller pass that structure between them, however indirectly.  It finds
>> such indirect ABI breakage for me fairly regularly, though of course I
>> don't know that it finds everything.
> 
> It is only the type name.
> 
> Not only that but even if you did extend it further to structure type
> arrangement then you still have to deal with other structures followed
> via pointers. Or (rarer but not unheard of):
> 
> - changes to structures without changes of the types of their members
> - changes to arguments without changes of their type

This is already covered by genksyms. Try make V=1 with
CONFIG_MODVERSIONS=y and add the -D option to one of the genksyms
command. I wanted to paste the expanded signature for
register_filesystem() as an example, but vger would probably drop the
mail for being too big :).


> - changes to semantics of functions
> - data structures derived in ways other than exported symbols, e.g.,
>   fixed register for `current` on some archs

Right, this is something that genksyms has no idea about.

Michal

Re: [PATCH] x86/kbuild: enable modversions for symbols exported from asm

[Michal Marek]

Dne 29.11.2016 v 18:10 Linus Torvalds napsal(a):
> How about this stupid patch? It weakens modversions, but that may be
> ok for Debian, and a better alternative than just saying "we don't
> support it at all".
[...]
> - pr_warn("%s: no symbol version for %s\n", mod->name, symname);
> - return 0;
> + /* Broken toolchain. Warn once, then let it go.. */
> + pr_warn_once("%s: no symbol version for %s\n", mod->name, symname);
> + return 1;

Fine with me, if it goes with the revert of the "depends on BROKEN."

Michal

Re: [PATCH] x86/kbuild: enable modversions for symbols exported from asm

[Michal Marek]

Dne 29.11.2016 v 18:10 Linus Torvalds napsal(a):
> How about this stupid patch? It weakens modversions, but that may be
> ok for Debian, and a better alternative than just saying "we don't
> support it at all".
[...]
> - pr_warn("%s: no symbol version for %s\n", mod->name, symname);
> - return 0;
> + /* Broken toolchain. Warn once, then let it go.. */
> + pr_warn_once("%s: no symbol version for %s\n", mod->name, symname);
> + return 1;

Fine with me, if it goes with the revert of the "depends on BROKEN."

Michal

Re: [PATCH] x86/kbuild: enable modversions for symbols exported from asm

[Michal Marek]

Dne 29.11.2016 v 16:27 Linus Torvalds napsal(a):
> On Nov 29, 2016 5:51 AM, "Adam Borowski"  > wrote:
>>
> 
>> >
>> >  (a) tested
>>
>> By many people.
> 
> No.
> 
> I've tested the build *without* this, and it works fine.
> 
>> >  (b) explains it
>>
>> The actual logic is in 4efca4ed0.  It wants C prototypes defined in
>> asm/asm-prototypes.h that lists symbols defined in assembly -- genksyms
>> knows only how to read C code.
> 
> See above. I'm not taking more random patches that "fix" this when it's
> not broken for me. Not without very explicit explanations of why that
> patch is still needed for others.

The original and easily observable bug is that were are not generating
symbol checksums for the asm-exported symbols, so they default to 0.
This can be seen e.g. in the Module.symvers file. This seemed like a
minor issue, because with the functions written in asm, the type
checking is rather weak (this has been the case even before Al's
patches). However, there is another bug that with _some_ toolchains /
architectures, the checksums do not default to 0, but they are simply
missing in the ___kcrctab* sections and the module loader complains. We
can of course research into the details of the second bug, but we
already know that we are not generating the checksums while we should be.

Michal

Re: [PATCH] x86/kbuild: enable modversions for symbols exported from asm

[Michal Marek]

Dne 29.11.2016 v 16:27 Linus Torvalds napsal(a):
> On Nov 29, 2016 5:51 AM, "Adam Borowski"  > wrote:
>>
> 
>> >
>> >  (a) tested
>>
>> By many people.
> 
> No.
> 
> I've tested the build *without* this, and it works fine.
> 
>> >  (b) explains it
>>
>> The actual logic is in 4efca4ed0.  It wants C prototypes defined in
>> asm/asm-prototypes.h that lists symbols defined in assembly -- genksyms
>> knows only how to read C code.
> 
> See above. I'm not taking more random patches that "fix" this when it's
> not broken for me. Not without very explicit explanations of why that
> patch is still needed for others.

The original and easily observable bug is that were are not generating
symbol checksums for the asm-exported symbols, so they default to 0.
This can be seen e.g. in the Module.symvers file. This seemed like a
minor issue, because with the functions written in asm, the type
checking is rather weak (this has been the case even before Al's
patches). However, there is another bug that with _some_ toolchains /
architectures, the checksums do not default to 0, but they are simply
missing in the ___kcrctab* sections and the module loader complains. We
can of course research into the details of the second bug, but we
already know that we are not generating the checksums while we should be.

Michal

Re: [PATCH v4] Fixes for compiling with clang

[Michal Marek]

Dne 28.11.2016 v 07:44 Peter Foley napsal(a):
> Fix reversed conditional checking if HOSTCC is clang.
> Suppress warnings about unsupported optimization options.
> Suppress warnings about unused functions, as they are generated for
> every module and are therefore far too spammy.
> Disable clang's integrated assembler which is incompatible with kernel
> asm constructs.
> 
> Changes from v1:
> * Move the $(cc-name) conditional below the definition of $(CC)
> Changes from v2:
> * Actually commit the changes in v2.
> Changes from v3:
> * HOSTCC is not guaranteed to be the same as CC, adjust accordingly.
> 
> Signed-off-by: Peter Foley 
> ---
>  Makefile | 12 
>  1 file changed, 8 insertions(+), 4 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index 694111b43cf8..9c4803a74117 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -301,12 +301,12 @@ CONFIG_SHELL := $(shell if [ -x "$$BASH" ]; then echo 
> $$BASH; \
>  
>  HOSTCC   = gcc
>  HOSTCXX  = g++
> -HOSTCFLAGS   = -Wall -Wmissing-prototypes -Wstrict-prototypes -O2 
> -fomit-frame-pointer -std=gnu89
> +HOSTCFLAGS   = -Wall -Wmissing-prototypes -Wstrict-prototypes -O2 
> -fomit-frame-pointer -std=gnu89 \
> +   -Wno-unused-value -Wno-unused-parameter 
> -Wno-missing-field-initializers
>  HOSTCXXFLAGS = -O2

This adds new -Wno-* options also for the gcc case, is there a reason
for this? Also, the -Wno-missing-field-initializers option is not
available in some old gccs, so we would need a HOSTCC equivalent of
cc-disable-warning.

Michal

Re: [PATCH v4] Fixes for compiling with clang

[Michal Marek]

Dne 28.11.2016 v 07:44 Peter Foley napsal(a):
> Fix reversed conditional checking if HOSTCC is clang.
> Suppress warnings about unsupported optimization options.
> Suppress warnings about unused functions, as they are generated for
> every module and are therefore far too spammy.
> Disable clang's integrated assembler which is incompatible with kernel
> asm constructs.
> 
> Changes from v1:
> * Move the $(cc-name) conditional below the definition of $(CC)
> Changes from v2:
> * Actually commit the changes in v2.
> Changes from v3:
> * HOSTCC is not guaranteed to be the same as CC, adjust accordingly.
> 
> Signed-off-by: Peter Foley 
> ---
>  Makefile | 12 
>  1 file changed, 8 insertions(+), 4 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index 694111b43cf8..9c4803a74117 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -301,12 +301,12 @@ CONFIG_SHELL := $(shell if [ -x "$$BASH" ]; then echo 
> $$BASH; \
>  
>  HOSTCC   = gcc
>  HOSTCXX  = g++
> -HOSTCFLAGS   = -Wall -Wmissing-prototypes -Wstrict-prototypes -O2 
> -fomit-frame-pointer -std=gnu89
> +HOSTCFLAGS   = -Wall -Wmissing-prototypes -Wstrict-prototypes -O2 
> -fomit-frame-pointer -std=gnu89 \
> +   -Wno-unused-value -Wno-unused-parameter 
> -Wno-missing-field-initializers
>  HOSTCXXFLAGS = -O2

This adds new -Wno-* options also for the gcc case, is there a reason
for this? Also, the -Wno-missing-field-initializers option is not
available in some old gccs, so we would need a HOSTCC equivalent of
cc-disable-warning.

Michal

Re: [PATCH] Scripts: kconfig: nconf: fix _GNU_SOURCE redefined warning

[Michal Marek]

On Mon, Nov 28, 2016 at 09:29:21PM +0100, Paul Bolle wrote:
> [Dropped Yann.]
> 
> On Tue, 2016-11-29 at 02:46 +0800, Cheah Kok Cheong wrote:
> > On Mon, Nov 28, 2016 at 11:39:01AM +0100, Michal Marek wrote:
> > > The Makefile does not specify -D_GNU_SOURCE. Are you adding it manually?
> 
> On Fedora 24 I'm able to trigger this too:
> $ [...]
> make -f ./scripts/Makefile.build obj=scripts/kconfig nconfig
>   gcc -Wp,-MD,scripts/kconfig/.nconf.o.d -Wall -Wmissing-prototypes 
> -Wstrict-prototypes -O2 -fomit-frame-pointer -std=gnu89   -D_GNU_SOURCE 
> -DCURSES_LOC="" -DNCURSES_WIDECHAR=1 -DLOCALE   -c -o 
> scripts/kconfig/nconf.o scripts/kconfig/nconf.c
> scripts/kconfig/nconf.c:8:0: warning: "_GNU_SOURCE" redefined
>  #define _GNU_SOURCE
>  
> :0:0: note: this is the location of the previous definition
> 
> It's probably added by pkg-config:
> $ pkg-config --cflags ncursesw
> -D_GNU_SOURCE

I see. That's not nice of the pkconfig file to set such macro, but oh
well. I applied the patch to kbuild.git#kconfig now, thanks Cheah.

Michal

Re: [PATCH] Scripts: kconfig: nconf: fix _GNU_SOURCE redefined warning

[Michal Marek]

On Mon, Nov 28, 2016 at 09:29:21PM +0100, Paul Bolle wrote:
> [Dropped Yann.]
> 
> On Tue, 2016-11-29 at 02:46 +0800, Cheah Kok Cheong wrote:
> > On Mon, Nov 28, 2016 at 11:39:01AM +0100, Michal Marek wrote:
> > > The Makefile does not specify -D_GNU_SOURCE. Are you adding it manually?
> 
> On Fedora 24 I'm able to trigger this too:
> $ [...]
> make -f ./scripts/Makefile.build obj=scripts/kconfig nconfig
>   gcc -Wp,-MD,scripts/kconfig/.nconf.o.d -Wall -Wmissing-prototypes 
> -Wstrict-prototypes -O2 -fomit-frame-pointer -std=gnu89   -D_GNU_SOURCE 
> -DCURSES_LOC="" -DNCURSES_WIDECHAR=1 -DLOCALE   -c -o 
> scripts/kconfig/nconf.o scripts/kconfig/nconf.c
> scripts/kconfig/nconf.c:8:0: warning: "_GNU_SOURCE" redefined
>  #define _GNU_SOURCE
>  
> :0:0: note: this is the location of the previous definition
> 
> It's probably added by pkg-config:
> $ pkg-config --cflags ncursesw
> -D_GNU_SOURCE

I see. That's not nice of the pkconfig file to set such macro, but oh
well. I applied the patch to kbuild.git#kconfig now, thanks Cheah.

Michal

Re: [PATCH] x86/kbuild: enable modversions for symbols exported from asm

[Michal Marek]

Dne 29.11.2016 v 03:31 Nicholas Piggin napsal(a):
> On Tue, 29 Nov 2016 01:15:48 +
> Ben Hutchings  wrote:
> 
>> [I've had to guess at the cc list for this, because we no longer have
>> mail archives that preserve them.]
> 
> You got it about right.
> 
>> On Fri, 2016-11-25 at 10:01 -0800, Linus Torvalds wrote:
>>> On Thu, Nov 24, 2016 at 4:40 PM, Nicholas Piggin  wrote: 
>>>  
>
> Yes, manual "marking" is never going to be a viable solution.  

 I guess it really depends on how exactly you want to use it. For distros
 that do stable ABI but rarely may have to break something for security
 reasons, it should work and give exact control.  
>>
>> This is roughly how Debian handles the kernel module ABI during a
>> stable release.
>>
>>> No. Because nobody else will care, so unless it's like a single symbol
>>> or something, it will just be a maintenance nightmare.  
>>
>> I agree with this.  We can explicitly "version" individual symbols
>> anyway by doing something like:
>>
>> -int foo(void);
>> +#define foo foo_2
>> +int foo_2(int);
> 
> Yeah... Benefit being it's very simple and everybody can see exactly
> what it does and knows how it will work.
> 
>>
 What else do people *actually* use it for? Preventing mismatched modules
 when .git version is not attached and release version of the kernel has
 not been bumped. Is that it?  
>>>
>>> It used to be very useful for avoiding loading stale modules and then
>>> wasting days on debugging something that wasn't the case when you had
>>> forgotten to do "make modules_install". Change some subtle internal
>>> ABI issue (add/remove a parameter, whatever) and it would really help.
>>>
>>> These days, for me, LOCALVERSION_AUTO and module signing are what I
>>> personally tend to use.
>>>
>>> The modversions stuff may just be too painful to bother with. Very few
>>> people probably use it, and the ones that do likely don't have any
>>> overriding reason why.  
>> [...]
>>
>> Debian has some strong reasons:

I guess many distros have similar reasons.


>> 1. Changing the release string requires any out-of-tree modules to be
>> upgraded (at least rebuilt) on end-user systems.  So we try to avoid
>> doing that during the lifetime of a stable release, i.e. we don't let
>> the release string change.  Also, the release string is reflected in
>> package names (e.g. linux-image-4.8.0-1-amd64), and introducing new
>> package names requires manual approval by the Debian archive team.
> 
> This is something I've noticed. Would it be better if the module loader
> ignores the kernel version and instead used some internal ABI version
> string to check against? Otherwise (AFAICT) you always have 4.8.0 versions
> despite being 4.8.7 kernel, and you can't upgrade a point release without
> overwriting your old kernel and modules.

The thing is - to maintain an ABI version string, you need some level of
certainty that two given ABIs are really interchangeable. Which means
you need to check whether the symbols _and_ types exposed are unchanged.
Which is a thing that genksyms, the tool behind CONFIG_MODVERSIONS, does
quite well. So yes, you could do a testbuild with CONFIG_MODVERSIONS=y
and a production build with some global ABI string, but what's the point
then.


> It would be nice to get upstream to the point where 4.9 modversions
> works if you just patch out depends BROKEN. That would require reverting
> a few more of Al's arch patches.
> 
> Then in 4.10 we can re-add all those arch patches (which are less
> controversial without the asm-prototypes.h workaround), and implement a
> simple stable ABI version string check, and then in 4.11 we can remove
> modversions.

I'd rather change the kconfig to

depends on BROKEN || 

and eventuallly remove the dependency again. PPC has the header already,
so it can be added right away. I do not know why the x86 patch has not
been merged yet.

Michal

Re: [PATCH] x86/kbuild: enable modversions for symbols exported from asm

[Michal Marek]

Dne 29.11.2016 v 03:31 Nicholas Piggin napsal(a):
> On Tue, 29 Nov 2016 01:15:48 +
> Ben Hutchings  wrote:
> 
>> [I've had to guess at the cc list for this, because we no longer have
>> mail archives that preserve them.]
> 
> You got it about right.
> 
>> On Fri, 2016-11-25 at 10:01 -0800, Linus Torvalds wrote:
>>> On Thu, Nov 24, 2016 at 4:40 PM, Nicholas Piggin  wrote: 
>>>  
>
> Yes, manual "marking" is never going to be a viable solution.  

 I guess it really depends on how exactly you want to use it. For distros
 that do stable ABI but rarely may have to break something for security
 reasons, it should work and give exact control.  
>>
>> This is roughly how Debian handles the kernel module ABI during a
>> stable release.
>>
>>> No. Because nobody else will care, so unless it's like a single symbol
>>> or something, it will just be a maintenance nightmare.  
>>
>> I agree with this.  We can explicitly "version" individual symbols
>> anyway by doing something like:
>>
>> -int foo(void);
>> +#define foo foo_2
>> +int foo_2(int);
> 
> Yeah... Benefit being it's very simple and everybody can see exactly
> what it does and knows how it will work.
> 
>>
 What else do people *actually* use it for? Preventing mismatched modules
 when .git version is not attached and release version of the kernel has
 not been bumped. Is that it?  
>>>
>>> It used to be very useful for avoiding loading stale modules and then
>>> wasting days on debugging something that wasn't the case when you had
>>> forgotten to do "make modules_install". Change some subtle internal
>>> ABI issue (add/remove a parameter, whatever) and it would really help.
>>>
>>> These days, for me, LOCALVERSION_AUTO and module signing are what I
>>> personally tend to use.
>>>
>>> The modversions stuff may just be too painful to bother with. Very few
>>> people probably use it, and the ones that do likely don't have any
>>> overriding reason why.  
>> [...]
>>
>> Debian has some strong reasons:

I guess many distros have similar reasons.


>> 1. Changing the release string requires any out-of-tree modules to be
>> upgraded (at least rebuilt) on end-user systems.  So we try to avoid
>> doing that during the lifetime of a stable release, i.e. we don't let
>> the release string change.  Also, the release string is reflected in
>> package names (e.g. linux-image-4.8.0-1-amd64), and introducing new
>> package names requires manual approval by the Debian archive team.
> 
> This is something I've noticed. Would it be better if the module loader
> ignores the kernel version and instead used some internal ABI version
> string to check against? Otherwise (AFAICT) you always have 4.8.0 versions
> despite being 4.8.7 kernel, and you can't upgrade a point release without
> overwriting your old kernel and modules.

The thing is - to maintain an ABI version string, you need some level of
certainty that two given ABIs are really interchangeable. Which means
you need to check whether the symbols _and_ types exposed are unchanged.
Which is a thing that genksyms, the tool behind CONFIG_MODVERSIONS, does
quite well. So yes, you could do a testbuild with CONFIG_MODVERSIONS=y
and a production build with some global ABI string, but what's the point
then.


> It would be nice to get upstream to the point where 4.9 modversions
> works if you just patch out depends BROKEN. That would require reverting
> a few more of Al's arch patches.
> 
> Then in 4.10 we can re-add all those arch patches (which are less
> controversial without the asm-prototypes.h workaround), and implement a
> simple stable ABI version string check, and then in 4.11 we can remove
> modversions.

I'd rather change the kconfig to

depends on BROKEN || 

and eventuallly remove the dependency again. PPC has the header already,
so it can be added right away. I do not know why the x86 patch has not
been merged yet.

Michal

Re: [PATCH] Scripts: kconfig: nconf: fix _GNU_SOURCE redefined warning

[Michal Marek]

Dne 27.11.2016 v 17:28 Cheah Kok Cheong napsal(a):
> Fix below warning when make nconfig is run initially
> or after make clean.
> 
>   HOSTCC  scripts/kconfig/nconf.o
> scripts/kconfig/nconf.c:8:0: warning: "_GNU_SOURCE" redefined
>  #define _GNU_SOURCE
>  ^
> :0:0: note: this is the location of the previous definition

The Makefile does not specify -D_GNU_SOURCE. Are you adding it manually?

Thanks,
Michal

Re: [PATCH] Scripts: kconfig: nconf: fix _GNU_SOURCE redefined warning

[Michal Marek]

Dne 27.11.2016 v 17:28 Cheah Kok Cheong napsal(a):
> Fix below warning when make nconfig is run initially
> or after make clean.
> 
>   HOSTCC  scripts/kconfig/nconf.o
> scripts/kconfig/nconf.c:8:0: warning: "_GNU_SOURCE" redefined
>  #define _GNU_SOURCE
>  ^
> :0:0: note: this is the location of the previous definition

The Makefile does not specify -D_GNU_SOURCE. Are you adding it manually?

Thanks,
Michal

Re: [PATCH 6/6] deb-pkg: Remove the KBUILD_IMAGE workaround

[Michal Marek]

Dne 25.11.2016 v 13:12 Riku Voipio napsal(a):
> On 22 November 2016 at 23:34, Michal Marek <mma...@suse.com> wrote:
>> The arch Makefile are fixed to set KBUILD_IMAGE to the full patch, so
>> the workaround is no longer needed.
>>
>> Signed-off-by: Michal Marek <mma...@suse.com>
>> ---
> Thanks for cleaning this up.
> 
> Reviewed-by: Riku Voipio <riku.voi...@linaro.org>

Note that this will only be merged once the individual arch patches have
been picked up.

Michal

Re: [PATCH 6/6] deb-pkg: Remove the KBUILD_IMAGE workaround

[Michal Marek]

Dne 25.11.2016 v 13:12 Riku Voipio napsal(a):
> On 22 November 2016 at 23:34, Michal Marek  wrote:
>> The arch Makefile are fixed to set KBUILD_IMAGE to the full patch, so
>> the workaround is no longer needed.
>>
>> Signed-off-by: Michal Marek 
>> ---
> Thanks for cleaning this up.
> 
> Reviewed-by: Riku Voipio 

Note that this will only be merged once the individual arch patches have
been picked up.

Michal