Re: [PATCH 0/7] build updates (and RFC on one-pass kallsyms generation)

2016-10-20 Thread Nicholas Piggin
On Wed, 19 Oct 2016 18:44:22 +1100
Nicholas Piggin  wrote:

> On Wed, 19 Oct 2016 14:15:53 +1100
> Nicholas Piggin  wrote:
> 
> > [*] Building allyesconfig still requires KALLSYMS_EXTRA_PASS=1, which
> > I'm yet to look into.  
> 
> Oh, it's because the kallsyms payload increases kernel image size and that
> causes more linker stubs to be generated, which have symbols, which go into
> kallsyms... What a nightmare.
> 
> We can use --no-emit-stub-syms, but it's kind of nice to have names for
> things.

This is a real quick hack at a way to improve it.

Even if we don't go with the kbuild change (which causes a slight
regression in a handful of symbols), I'll pursue the .kallsyms section
and powerpc change to move it toward the end: that should be enough to
avoid the build failures, and it moves a huge blob of cold data from the
middle of the image.

kbuild: put kallsyms into its own section

With example linker script implementation for powerpc. This allows
us to put the kallsyms section at the end of the image rather than
in the middle, which doesn't jumble all our offsets when linking it
in, or causing liker to add more stubs. Which means we can do kallsyms
generation in one pass. A few values (_end, and some kallsyms_ variables)
do move, so they don't get correct symbols.

diff --git a/arch/powerpc/kernel/vmlinux.lds.S 
b/arch/powerpc/kernel/vmlinux.lds.S
index 115a44e..9ebe1df 100644
--- a/arch/powerpc/kernel/vmlinux.lds.S
+++ b/arch/powerpc/kernel/vmlinux.lds.S
@@ -294,6 +294,10 @@ SECTIONS
 
BSS_SECTION(0, 0, 0)
 
+   .kallsyms : AT(ADDR(.kallsyms) - LOAD_OFFSET) {
+   KEEP(*(.kallsyms))
+   }
+
. = ALIGN(PAGE_SIZE);
_end = . ;
PROVIDE32 (end = .);
diff --git a/kernel/kallsyms.c b/kernel/kallsyms.c
index fafd1a3..311fe07 100644
--- a/kernel/kallsyms.c
+++ b/kernel/kallsyms.c
@@ -37,24 +37,30 @@
  * These will be re-linked against their real values
  * during the second link stage.
  */
-extern const unsigned long kallsyms_addresses[] __weak;
-extern const int kallsyms_offsets[] __weak;
-extern const u8 kallsyms_names[] __weak;
+extern const unsigned long kallsyms_addresses[]
+__attribute__((weak, section(".kallsyms")));
+extern const int kallsyms_offsets[]
+__attribute__((weak, section(".kallsyms")));
+extern const u8 kallsyms_names[]
+__attribute__((weak, section(".kallsyms")));
 
 /*
  * Tell the compiler that the count isn't in the small data section if the arch
  * has one (eg: FRV).
  */
 extern const unsigned long kallsyms_num_syms
-__attribute__((weak, section(".rodata")));
+__attribute__((weak, section(".kallsyms")));
 
 extern const unsigned long kallsyms_relative_base
-__attribute__((weak, section(".rodata")));
+__attribute__((weak, section(".kallsyms")));
 
-extern const u8 kallsyms_token_table[] __weak;
-extern const u16 kallsyms_token_index[] __weak;
+extern const u8 kallsyms_token_table[]
+__attribute__((weak, section(".kallsyms")));
+extern const u16 kallsyms_token_index[]
+__attribute__((weak, section(".kallsyms")));
 
-extern const unsigned long kallsyms_markers[] __weak;
+extern const unsigned long kallsyms_markers[]
+__attribute__((weak, section(".kallsyms")));
 
 static inline int is_kernel_inittext(unsigned long addr)
 {
diff --git a/scripts/kallsyms.c b/scripts/kallsyms.c
index 1f22a18..4327d80 100644
--- a/scripts/kallsyms.c
+++ b/scripts/kallsyms.c
@@ -349,7 +349,7 @@ static void write_src(void)
printf("#define ALGN .align 4\n");
printf("#endif\n");
 
-   printf("\t.section .rodata, \"a\"\n");
+   printf("\t.section .kallsyms, \"a\"\n");
 
/* Provide proper symbols relocatability by their relativeness
 * to a fixed anchor point in the runtime image, either '_text'
commit 61502b241e9b597a60d6032a5f5a4c1dfbcafece
Author: Nicholas Piggin 
Date:   Wed Oct 19 19:56:26 2016 +1100

kbuild: one-pass kallsyms generation

With kallsyms at the end of the image, linking kallsyms doesn't
change around all offsets or cause linker stubs to be added. This
means kallsyms generation can be done in just one pass, and it
won't fail due to linker stubs.

diff --git a/arch/powerpc/kernel/vmlinux.lds.S 
b/arch/powerpc/kernel/vmlinux.lds.S
index 9ebe1df..534c8df 100644
--- a/arch/powerpc/kernel/vmlinux.lds.S
+++ b/arch/powerpc/kernel/vmlinux.lds.S
@@ -190,29 +190,6 @@ SECTIONS
*(.machine.desc)
__machine_desc_end = . ;
}
-#ifdef CONFIG_RELOCATABLE
-   . = ALIGN(8);
-   .dynsym : AT(ADDR(.dynsym) - LOAD_OFFSET)
-   {
-#ifdef CONFIG_PPC32
-   __dynamic_symtab = .;
-#endif
-   *(.dynsym)
-   }
-   .dynstr : AT(ADDR(.dynstr) - LOAD_OFFSET) { *(.dynstr) }
-   .dynamic : AT(ADDR(.dynamic) - LOAD_OFFSET)
-   {
-   __dynamic_start = .;
-   *(.dynamic)
-   }
-   .hash : AT(ADDR(.hash) - LOAD_OFFSET) { *(.hash) }
-   .interp : AT(ADDR(.in

Re: [PATCH 0/7] build updates

2016-10-19 Thread Nicholas Piggin
On Wed, 19 Oct 2016 14:15:53 +1100
Nicholas Piggin  wrote:

> [*] Building allyesconfig still requires KALLSYMS_EXTRA_PASS=1, which
> I'm yet to look into.

Oh, it's because the kallsyms payload increases kernel image size and that
causes more linker stubs to be generated, which have symbols, which go into
kallsyms... What a nightmare.

We can use --no-emit-stub-syms, but it's kind of nice to have names for
things.

Thanks,
Nick


[PATCH 0/7] build updates

2016-10-18 Thread Nicholas Piggin
Hi,

I was hoping to get these posted earlier, but they had a dependency
on the kbuild tree which took a while to merge. Hopefully we can get
at least the first 4 patches in, which are just sanity checks on the
final binary.

With this series applied, I'm able to build[*] and boot a 64-bit
book3s allyesconfig kernel, a 200MB vmlinux (although today's tree
breaks in some random driver initcall in late boot). That's largely
a culmination of Stephen Rothwell's work (including the kbuild changes
that were merged earlier in 4.9), I just picked up some of his patches
so attributions may not be 100% reflective of reality.

[*] Building allyesconfig still requires KALLSYMS_EXTRA_PASS=1, which
I'm yet to look into.

Thanks,
Nick

Nicholas Piggin (7):
  powerpc: use the new post-link pass to check relocations
  powerpc: add arch/powerpc/tools directory
  powerpc/64s: tool to flag direct branches from unrelocated interrupt
vectors
  powerpc/64: tool to check head sections location sanity
  powerpc/64: handle linker stubs in low .text code
  powerpc: switch to using thin archives if COMPILE_TEST is set
  powerpc/64: allow CONFIG_RELOCATABLE if COMPILE_TEST

 arch/powerpc/Kconfig   |  5 +-
 arch/powerpc/Makefile  | 17 +
 arch/powerpc/Makefile.postlink | 48 ++
 arch/powerpc/include/asm/head-64.h | 20 --
 arch/powerpc/kernel/vmlinux.lds.S  | 28 ++--
 .../gcc-check-mprofile-kernel.sh   |  0
 arch/powerpc/tools/head_check.sh   | 74 ++
 arch/powerpc/{ => tools}/relocs_check.sh   |  0
 arch/powerpc/tools/unrel_branch_check.sh   | 56 
 9 files changed, 203 insertions(+), 45 deletions(-)
 create mode 100644 arch/powerpc/Makefile.postlink
 rename arch/powerpc/{scripts => tools}/gcc-check-mprofile-kernel.sh (100%)
 create mode 100755 arch/powerpc/tools/head_check.sh
 rename arch/powerpc/{ => tools}/relocs_check.sh (100%)
 create mode 100755 arch/powerpc/tools/unrel_branch_check.sh

-- 
2.9.3