[The backtrace confirms what I reported to Alexander Richardson and Shawn Webb earlier.]
On 2018-Nov-1, at 6:40 PM, Mark Millard <marklmi26-fbsd at yahoo.com> wrote: > On 2018-Nov-1, at 5:41 PM, Konstantin Belousov <kostikbel at gmail.com> wrote: > >> On Tue, Oct 30, 2018 at 12:45:13PM -0700, Mark Millard wrote: >>> Konstantin Belousov kostikbel at gmail.com wrote on >>> Tue Oct 30 18:04:04 UTC 2018 : >>> >>>> On Tue, Oct 30, 2018 at 03:32:40PM +0000, Alexander Richardson wrote: >>>>> On Tue, 30 Oct 2018 at 10:17, Michael Tuexen >>>>> <Michael.Tuexen at macmic.franken.de> wrote: >>>>>> >>>>>>> On 29. Oct 2018, at 22:08, Alex Richardson <arichardson at FreeBSD.org> >>>>>>> wrote: >>>>>>> >>>>>>> Author: arichardson >>>>>>> Date: Mon Oct 29 21:08:02 2018 >>>>>>> New Revision: 339876 >>>>>>> URL: https://svnweb.freebsd.org/changeset/base/339876 >>>>>>> >>>>>>> Log: >>>>>>> rtld: set obj->textsize correctly >>>>>>> >>>>>>> With lld-generated binaries the first PT_LOAD will usually be a >>>>>>> read-only >>>>>>> segment unless you pass --no-rosegment. For those binaries the textsize >>>>>>> is >>>>>>> determined by the next PT_LOAD. To allow both LLD and bfd 2.17 binaries >>>>>>> to >>>>>>> be parsed correctly use the end of the last PT_LOAD that is marked as >>>>>>> executable instead. >>>>>>> >>>>>>> I noticed that the value was wrong while adding some debug prints for >>>>>>> some rtld >>>>>>> changes for CHERI binaries. `obj->textsize` only seems to be used by >>>>>>> PPC so the >>>>>>> effect is untested. However, the value before was definitely wrong and >>>>>>> the new >>>>>>> result matches the phdrs. >>>>>> I build kernel and world with a revision later than this on a PPC. >>>>>> Buildword >>>>>> ends up with a world where almost all binaries are segfaulting.... >>>>>> Especially gdb >>>>>> (but svn, ls or so all segfault). >>>>>> >>>>>> Best regards >>>>>> Michael >>>>> >>>>> This is rather surprising since if anything the range of the icache >>>>> flush should increase rather than decrease after this change. >>>>> >>>>> I can only see this causing a behaviour change if we actually need to >>>>> flush more than just the executable segments. >>>>> Is it possible that some binary/library contains a non-executable >>>>> segment as the first PT_LOAD? >>>>> Or is there some linker script that adds custom PHDRS? >>>>> >>>> Could it be that there is a hole between start of the object mapping and >>>> the last PT_LOADable segment eligible for execution ? >>> >>> [This note may be easier to deal with than the first >>> note that I sent out.] >>> >>> [My examples are from devel/powerpc64-xtoolchain-gcc used >>> to buildworld buildkernel targeting a head -r339076 based >>> powerpc64 environment. I do that on powerpc64 as well.] >>> >>> powerpc64 loads the readonly data and the readonly code in one PT_LOAD, >>> the first. The 2nd PT_LOAD is for sections without the readonly status, >>> that includes .got and .plt being spanned. See below from >>> objdump -ph for /bin/ls : >>> >>> Program Header: >>> PHDR off 0x0000000000000040 vaddr 0x0000000010000040 paddr >>> 0x0000000010000040 align 2**3 >>> filesz 0x0000000000000188 memsz 0x0000000000000188 flags r-- >>> INTERP off 0x00000000000001c8 vaddr 0x00000000100001c8 paddr >>> 0x00000000100001c8 align 2**0 >>> filesz 0x0000000000000015 memsz 0x0000000000000015 flags r-- >>> LOAD off 0x0000000000000000 vaddr 0x0000000010000000 paddr >>> 0x0000000010000000 align 2**16 >>> filesz 0x000000000000910c memsz 0x000000000000910c flags r-x >>> LOAD off 0x0000000000009110 vaddr 0x0000000010019110 paddr >>> 0x0000000010019110 align 2**16 >>> filesz 0x0000000000000ee0 memsz 0x00000000000010e8 flags rw- >>> DYNAMIC off 0x0000000000009138 vaddr 0x0000000010019138 paddr >>> 0x0000000010019138 align 2**3 >>> filesz 0x00000000000001c0 memsz 0x00000000000001c0 flags rw- >>> NOTE off 0x00000000000001e0 vaddr 0x00000000100001e0 paddr >>> 0x00000000100001e0 align 2**2 >>> filesz 0x0000000000000030 memsz 0x0000000000000030 flags r-- >>> STACK off 0x0000000000000000 vaddr 0x0000000000000000 paddr >>> 0x0000000000000000 align 2**4 >>> filesz 0x0000000000000000 memsz 0x0000000000000000 flags rw- >> >> We only need program headers, and we only need them from the object >> which load causes the fault. It might be not the binary but some library >> that triggers the fault. So the backtrace and some information from the >> state of the image is needed. >> >> You can build only rtld and use it as the standalone program to initiate >> the image activation: >> <path>/ld-elf.so.1 /bin/ls >> or similar. > > . . . It stops when the dcbst in __syncicache runs into an address in the p_align 65536 caused hole between the two PT_LOAD's with PF_X. /bin/ls itself has such a hole, as do the .so libraries involved. Details follow. (gdb) run /bin/ls Starting program: /usr/obj/powerpcvtsc_gcc421/powerpc.powerpc/usr/src/powerpc.powerpc/libexec/rtld-elf/ld-elf.so.1.full /bin/ls Program received signal SIGSEGV, Segmentation fault. __syncicache (from=0x1800000, len=102400) at /usr/src/lib/libc/powerpc/gen/syncicache.c:94 94 __asm __volatile ("dcbst 0,%0" :: "r"(p)); (gdb) bt #0 __syncicache (from=0x1800000, len=102400) at /usr/src/lib/libc/powerpc/gen/syncicache.c:94 #1 0x01012b58 in reloc_non_plt (obj=0x41041000, obj_rtld=<optimized out>, flags=4, lockstate=0x0) at /usr/src/libexec/rtld-elf/powerpc/reloc.c:330 #2 0x010175f4 in relocate_object (obj=0x41041000, bind_now=0 '\000', rtldobj=0x103ea88 <obj_rtld>, flags=4, lockstate=0x0) at /usr/src/libexec/rtld-elf/rtld.c:2849 #3 0x0101776c in relocate_objects (first=<optimized out>, bind_now=0 '\000', rtldobj=0x103ea88 <obj_rtld>, flags=4, lockstate=0x0) at /usr/src/libexec/rtld-elf/rtld.c:2908 #4 0x0101beb0 in _rtld (sp=<optimized out>, exit_proc=0xffffdc08, objp=0xffffdc0c) at /usr/src/libexec/rtld-elf/rtld.c:677 #5 0x0101217c in .rtld_start () at /usr/src/libexec/rtld-elf/powerpc/rtld_start.S:98 Backtrace stopped: frame did not save the PC This is because there are 2 PT_LOAD's that have PF_X but for which there is an address range between that spans may pages for which there is no header entry for, in part a consequence of p_align being 65536 so forcing lots of space between entries 2 and 3 ( /bin/ls example): entry: 2 p_type: PT_LOAD p_offset: 0 p_vaddr: 0x1800000 p_paddr: 0x1800000 p_filesz: 34112 p_memsz: 34112 p_flags: PF_X|PF_R p_align: 65536 entry: 3 p_type: PT_LOAD p_offset: 34112 p_vaddr: 0x1818540 p_paddr: 0x1818540 p_filesz: 316 p_memsz: 1752 p_flags: PF_X|PF_W|PF_R p_align: 65536 ( check 2's p_paddr+ p_memsz vs. 3's p_vaddr Later below the failing address will be shown as between those. ) [powerpc64 only gets one PT_LOAD with PF_X and so can not have space between such PT_LOAD's as things currently are.] The source code related details are as follows and shows that it was processing /bin/ls itself ( obj_main->path = 0x41042000 "/bin/ls" and the failing address ) : [The presentation order is not necessarily the same as my exploration order.] (gdb) list 89 off = (u_int)from & (cacheline_size - 1); 90 l = len += off; 91 p = (char *)from - off; 92 93 do { 94 __asm __volatile ("dcbst 0,%0" :: "r"(p)); 95 p += cacheline_size; 96 } while ((l -= cacheline_size) > 0); 97 __asm __volatile ("sync"); 98 p = (char *)from - off; [Reminder for entry 2: p_vaddr: 0x1800000 p_memsz: 34112 0x1800000+34112 = 0x1808540 ] (gdb) print p $4 = 0x1809000 <error: Cannot access memory at address 0x1809000> (gdb) up #1 0x01012b58 in reloc_non_plt (obj=0x41041000, obj_rtld=<optimized out>, flags=4, lockstate=0x0) at /usr/src/libexec/rtld-elf/powerpc/reloc.c:330 330 __syncicache(obj->mapbase, obj->textsize); (gdb) list 325 done: 326 if (cache != NULL) 327 free(cache); 328 329 /* Synchronize icache for text seg in case we made any changes */ 330 __syncicache(obj->mapbase, obj->textsize); 331 332 return (r); 333 } 334 (gdb) up #2 0x010175f4 in relocate_object (obj=0x41041000, bind_now=0 '\000', rtldobj=0x103ea88 <obj_rtld>, flags=4, lockstate=0x0) at /usr/src/libexec/rtld-elf/rtld.c:2849 2849 if (reloc_non_plt(obj, rtldobj, flags, lockstate)) (gdb) list 2844 /* There are relocations to the write-protected text segment. */ 2845 if (obj->textrel && reloc_textrel_prot(obj, true) != 0) 2846 return (-1); 2847 2848 /* Process the non-PLT non-IFUNC relocations. */ 2849 if (reloc_non_plt(obj, rtldobj, flags, lockstate)) 2850 return (-1); 2851 2852 /* Re-protected the text segment. */ 2853 if (obj->textrel && reloc_textrel_prot(obj, false) != 0) (gdb) up #3 0x0101776c in relocate_objects (first=<optimized out>, bind_now=0 '\000', rtldobj=0x103ea88 <obj_rtld>, flags=4, lockstate=0x0) at /usr/src/libexec/rtld-elf/rtld.c:2908 2908 error = relocate_object(obj, bind_now, rtldobj, flags, (gdb) list 2903 2904 for (error = 0, obj = first; obj != NULL; 2905 obj = TAILQ_NEXT(obj, next)) { 2906 if (obj->marker) 2907 continue; 2908 error = relocate_object(obj, bind_now, rtldobj, flags, 2909 lockstate); 2910 if (error == -1) 2911 break; 2912 } (gdb) up #4 0x0101beb0 in _rtld (sp=<optimized out>, exit_proc=0xffffdc08, objp=0xffffdc0c) at /usr/src/libexec/rtld-elf/rtld.c:677 677 if (relocate_objects(obj_main, (gdb) list 672 * block even if they didn't ask for it. 673 */ 674 allocate_tls_offset(entry->obj); 675 } 676 677 if (relocate_objects(obj_main, 678 ld_bind_now != NULL && *ld_bind_now != '\0', 679 &obj_rtld, SYMLOOK_EARLY, NULL) == -1) 680 rtld_die(); 681 (gdb) up #5 0x0101217c in .rtld_start () at /usr/src/libexec/rtld-elf/powerpc/rtld_start.S:98 98 bl _rtld@plt /* &_start = _rtld(sp, &exit_proc, &obj_main)*/ (gdb) list 93 addi %r3,%r4,-4 /* locate argc ptr, &argv[-1] */ 94 95 addi %r4,%r1,8 /* &exit_proc on stack */ 96 addi %r5,%r1,12 /* &obj_main on stack */ 97 98 bl _rtld@plt /* &_start = _rtld(sp, &exit_proc, &obj_main)*/ 99 mtlr %r3 100 101 /* 102 * Restore args, with new obj/exit proc (gdb) down #4 0x0101beb0 in _rtld (sp=<optimized out>, exit_proc=0xffffdc08, objp=0xffffdc0c) at /usr/src/libexec/rtld-elf/rtld.c:677 677 if (relocate_objects(obj_main, . . . (gdb) print *obj_main $3 = {magic = 0, version = 0, next = {tqe_next = 0x41041200, tqe_prev = 0x103ea68 <obj_list>}, path = 0x41042000 "/bin/ls", origin_path = 0x0, refcount = 1, holdcount = 0, dl_refcount = 0, mapbase = 0x1800000 "\177ELF\001\002\001\t", mapsize = 102400, textsize = 102400, vaddrbase = 25165824, relocbase = 0x0, dynamic = 0x1818554, entry = 0x18022e0 "|\b\002\246\224!\377\340= \001\202\223\241", phdr = 0x1800034, phsize = 224, interp = 0x1800114 "/libexec/ld-elf.so.1", stack_flags = 6, tlsindex = 0, tlsinit = 0x0, tlsinitsize = 0, tlssize = 0, tlsoffset = 0, tlsalign = 0, relro_page = 0x0, relro_size = 0, pltgot = 0x18186f4, rel = 0x0, relsize = 0, rela = 0x1801ebc, relasize = 1008, pltrel = 0x0, pltrelsize = 0, pltrela = 0x1801f28, pltrelasize = 900, symtab = 0x1800798, strtab = 0x1801418 "", strsize = 2241, verneed = 0x1801e6c, verneednum = 1, verdef = 0x0, verdefnum = 0, versyms = 0x1801cda, buckets = 0x1800164, nbuckets = 197, chains = 0x1800478, nchains = 200, nbuckets_gnu = 0, symndx_gnu = 0, maskwords_bm_gnu = 0, shift2_gnu = 0, dynsymcount = 200, bloom_gnu = 0x0, buckets_gnu = 0x0, chain_zero_gnu = 0x0, rpath = 0x0, runpath = 0x0, needed = 0x41042010, needed_filtees = 0x0, needed_aux_filtees = 0x0, names = {stqh_first = 0x0, stqh_last = 0x410410f8}, vertab = 0x41043080, vernum = 6, init = 25174700, fini = 25197908, preinit_array = 0, init_array = 0, fini_array = 0, preinit_array_num = 0, init_array_num = 0, fini_array_num = 0, osrel = 1200084, mainprog = 1 '\001', rtld = 0 '\000', relocated = 1 '\001', ver_checked = 1 '\001', textrel = 0 '\000', symbolic = 0 '\000', bind_now = 0 '\000', traced = 0 '\000', jmpslots_done = 0 '\000', init_done = 0 '\000', tls_done = 1 '\001', phdr_alloc = 0 '\000', z_origin = 0 '\000', z_nodelete = 0 '\000', z_noopen = 0 '\000', z_loadfltr = 0 '\000', z_interpose = 0 '\000', z_nodeflib = 0 '\000', z_global = 0 '\000', ref_nodel = 0 '\000', init_scanned = 0 '\000', on_fini_list = 0 '\000', dag_inited = 0 '\000', filtees_loaded = 0 '\000', irelative = 0 '\000', gnu_ifunc = 0 '\000', non_plt_gnu_ifunc = 0 '\000', crt_no_init = 1 '\001', valid_hash_sysv = 1 '\001', valid_hash_gnu = 0 '\000', dlopened = 0 '\000', marker = 0 '\000', unholdfree = 0 '\000', doomed = 0 '\000', linkmap = {l_addr = 0x1800000 "\177ELF\001\002\001\t", l_name = 0x41042000 "/bin/ls", l_ld = 0x1818554, l_next = 0x41041334, l_prev = 0x0}, dldags = {stqh_first = 0x0, stqh_last = 0x41041148}, dagmembers = {stqh_first = 0x0, stqh_last = 0x41041150}, dev = 0, ino = 0, priv = 0x0} === Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) _______________________________________________ svn-src-head@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to "svn-src-head-unsubscr...@freebsd.org"
