Re: Bug#649038: elfutils FTBFS on kfreebsd
On Mon, 2013-11-11 at 23:31 +0100, Robert Millan wrote: > On 11/11/2013 15:32, Mark Wielaard wrote: > > On Sun, 2013-11-10 at 00:45 +0100, Robert Millan wrote: > >> Nothing as far as ELF compliance is concerned. This tag is ment to be > >> consumed by the kernel ELF loader only. > > > > For elfutils elflint it also matters for things like STB_GNU_UNIQUE and > > STT_GNU_IFUNC. Does ELFOSABI_FREEBSD indicate the binaray can or cannot > > contain such symbol types or bindings? > > No, it just gives information about the kernel<->user ABI. > > AFAIK STB_GNU_UNIQUE and STT_GNU_IFUNC are userland facilities. Is this > correct? It tells the shared library loader, ld.so, about the features of the ELF file, so it can resolve symbols with those bindings and/or types. > > elflint assumes those only occur > > with ELFOSABI_LINUX currently. > > Well I suppose you could extend this assumption to cover > ELFOSABI_FREEBSD as well? > > Though presence of ELFOSABI_FREEBSD is not enough to determine that they > are available. Indeed. Normally for the GNU toolchain, ELFOSABI_NONE describes "generic" ELF files, which don't use GNU extensions like STB_GNU_UNIQUE and STT_GNU_IFUNC. And ELFOSABI_LINUX describe ELF files which do use such GNU extensions. What is the convention on Debian/kfreebsd? > > /proc/PID/maps, /proc/PID/exe, /proc/PID/mem, > > Mostly OK I think. > > > /proc/TID/status > > It seems we don't have this. At least not in 9.0. :-( This one is mostly just used as a workaround to see whether ptrace correctly propagates a SIGSTOP on attach/detach. It can probably be ignored if ptrace "behaves". > > and /proc/PID/auvx > > Neither... That was a type BTW. Should have been auxv. It is sometimes possible to get at the AUXV information in a different way. If not available only some functionality is degraded. > > And /proc/kallsyms and /proc/modules are used to inspect kernel modules > > with libdwfl. > > Nope. For module listing we have kldstat() / kldfind() / kldnext(). > > > See libdwfl/linux-proc-maps.c (backend for dwfl_linux_proc_report) > > and libdwfl/linux-kernel-modules.c (backend for > > dwfl_linux_kernel_report_kernel and dwfl_linux_kernel_report_modules). > > Someone might want to provide > > backends for kfreebsd if the corresponding libdwfl dwfl_linux_* > functionality is wanted there. The kernel parts probably won't easily > work, the user > > space parts probably will assuming the /proc interface is linprocfs style > > and sufficiently compatible. > > Is it possible to disable this functionality on kFreeBSD? A quick > linprocfs solution seems like dead-end, and we really don't have the > manpower to write new backends from scratch. Well, it doesn't do much harm. It just means you cannot easily introspect the kernel and kernel modules with elfutils/libdwfl. But that is not functionality many programs want/need anyway. If they do, then there are probably other issues they have to deal with that make them depend on the linux kernel anyway. Cheers, Mark -- To UNSUBSCRIBE, email to debian-bsd-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1384336137.3813.116.ca...@bordewijk.wildebeest.org
Re: Bug#649038: elfutils FTBFS on kfreebsd
On Sun, 2013-11-10 at 00:45 +0100, Robert Millan wrote: > ELFOSABI_FREEBSD indicates this > binary has been built to run on kFreeBSD and uses its kernel ABI. > > If a binary is set to ELFOSABI_LINUX, then the kernel will enable Linux > emulation mode, i.e. Linux syscall interface. Aha. Interesting. Just curious. On Debain GNU/FreeBSD wouldn't the emulation mode be more natural? > Kernel modules are also built as ELF files, but AFAIK their e_ident is > not checked for. > > >> - If it is the correct OS ABI then what would an elflint program > >> need to know about it to make sure all its requirements are met? > > Nothing as far as ELF compliance is concerned. This tag is ment to be > consumed by the kernel ELF loader only. For elfutils elflint it also matters for things like STB_GNU_UNIQUE and STT_GNU_IFUNC. Does ELFOSABI_FREEBSD indicate the binaray can or cannot contain such symbol types or bindings? elflint assumes those only occur with ELFOSABI_LINUX currently. > >> - The other failures look like issues with the /proc interface > >> on the install. Does the /proc interface follow the Linux kernel > >> /proc interface that some of the tests rely on? > > Yes. But as there's no standard covering Linux-style /proc, it can't > ever be 100% complete. FreeBSD developers provide an emulated > "linprocfs" for compatibility purposes and try to keep up, but depending > on what you do it might not work. > > Also, this is only provided on GNU/kFreeBSD. FreeBSD systems either use > the native FreeBSD-style /proc or none at all. If you want to support > FreeBSD as well, it's better if you use sysctls or whatever you need for > what you're testing. This is mainly for the libdwl dwfl_linux_* group of functions. /proc/PID/maps, /proc/PID/exe, /proc/PID/mem, /proc/TID/status and /proc/PID/auvx are used to inspect user space binaries with libdwfl. And /proc/kallsyms and /proc/modules are used to inspect kernel modules with libdwfl. See libdwfl/linux-proc-maps.c (backend for dwfl_linux_proc_report) and libdwfl/linux-kernel-modules.c (backend for dwfl_linux_kernel_report_kernel and dwfl_linux_kernel_report_modules). Someone might want to provide backends for kfreebsd if the corresponding libdwfl dwfl_linux_* functionality is wanted there. The kernel parts probably won't easily work, the user space parts probably will assuming the /proc interface is linprocfs style and sufficiently compatible. Cheers, Mark -- To UNSUBSCRIBE, email to debian-bsd-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1384180329.4008.495.ca...@bordewijk.wildebeest.org
Re: Bug#726248: sdt.h conflict with kfreebsd-kernel-headers and systemtap-sdt-dev
On Sun, Oct 20, 2013 at 01:07:33AM +0200, Robert Millan wrote: > > But then programs that expect the header to be in the default place > > wouldn't build. The whole idea is that programs that use sys/sdt.h > > (and optionally the dtrace script) to use DTRACE_PROBE macros to > > define SDT probe points get them without having to change anything > > to their build system. They just detect in configure the sys/sdt.h > > is available (possibly checking for the dtrace script and whether > > the compiler is capable of building with DTRACE_PROBE macros). > > That is how for example qemu, java hotspot or libreoffice do it. > > So the probes you're referring to were added for DTrace, and SystemTap > takes advantage of them by providing a DTrace-compatible API. Is this > correct? It is a two way street. Or multi-way, it isn't just systemtap, gdb and perf for example also take advantage of them. For java the SDT markers were already there and all it took was build them with the right sys/sdt.h installed and now you can use them with any tool (stap, gdb, perf) that can read the right ELF section magic. For qemu it was the other way around, they were specifically added for stap, but you can also use them from gdb or perf. And when rebuild on Solaris with their sys/sdt.h they are also usable by dtrace. See e.g. https://www.berrange.com/posts/2011/11/30/watching-the-libvirt-rpc-protocol-using-systemtap/ > > The default sys/sdt.h header should match the toolchain and user space > > you are using. > > Toolchain and user space are very generic terms. Which components do you > have in mind? sys/sdt.h and compiler as producer of the SDT ELF section, gdb/binutils, perf and stap as consumers at a minimum. > If I understand correctly, we can't support both SystemTap and DTrace at > the same time, because SystemTap only aims for API compatibility, not > ABI compatibility, so we have to choose one to link everything against? > > SystemTap is a Linux tool. What is the advantage of using it on > kFreeBSD, in comparison with DTrace? It really isn't just systemtap, that is just one of the programs that can consume the SDT markers produced in the special ELF notes. gdb and perf at least can also use them for debugging and profiling. I don't know about dtrace, but on a system with a GNU toolchain I assume it can be made to work with the SDT ELF section too. It is all documented https://sourceware.org/systemtap/wiki/UserSpaceProbeImplementation For systemtap that is just one of the probe sources you can use. There are others inside the kernel, timers, syscalls, etc. I assume dtrace also has various sources from which it can generate traces? > As DTrace is fully integrated in FreeBSD upstream, theoretically we > could use it to debug both kernel and userland. Is it possible to do the > same with SystemTap? Yes, systemtap can also use both kernel and userland probes. perf also can profile both kernel and user space. gdb only does user space, but there is a special linux kernel kgdb. > > But if you think that on kfreebsd programs should be build against > > a different sys/sdt.h that is fine too (but then programs like gdb > > will not work with the SDT probes in programs and libraries or will > > be less efficient when handling things in glibc and libgcc). > > So you're saying that GDB has features which work with SystemTap, but > not with DTrace? I don't know what your sys/sdt.h produces when it sees a DTRACE_PROBE macro. I don't believe gdb knows how to interpret those. But it does know how to interpret the SDT markers produced by the sys/sdt.h header from std-dev. You can set (conditional) breakpoints on them, etc. And it even knows about some special ones in glibc and libgcc that help gdb with exceptions or shared library loading issues. You can think of DTRACE_PROBE macros as special places to set a breakpoint on. gdb uses those in one specific way, systemtap uses it for tracing, perf uses them for profiling, etc. > This would only work if SystemTap could be adjusted to use the same ABI > as DTrace. That would be the best solution IMHO. But it might not be > feasible. Systemtap doesn't work on any system/kernel that has a "dtrace ABI" as far as I know. Is there a discription of this ABI? I guess if there is a specification of how SDT probes are embedded in could be made to also interpret them (given we find some volunteers). BTW. This isn't a new thing on GNU/Linux. This has been integrated in various distributions for years. That is why the request came to make sdt-dev an integral part (arch:all) for Debian too. Then tools like gdb can rely on it and do several things currently not possible or less efficiently. > > Does any user space program Debian actually use the kfreebsd > > sys/sdt.h variant? > > Well, I guess that anything that has support for DTRACE_PROBE is already > using it. Haven't checked if this actually works though. It would be good to know if it actually works and has any programs/users. I was only
Re: Bug#726248: sdt.h conflict with kfreebsd-kernel-headers and systemtap-sdt-dev
On Sat, Oct 19, 2013 at 05:00:48PM +0200, Robert Millan wrote: > If you want to avoid modifying programs that #include , why > not just install it in /usr/include/systemtap/sys/sdt.h ? Then you can > build them with -I/usr/include/systemtap so that your version takes > preference. But then programs that expect the header to be in the default place wouldn't build. The whole idea is that programs that use sys/sdt.h (and optionally the dtrace script) to use DTRACE_PROBE macros to define SDT probe points get them without having to change anything to their build system. They just detect in configure the sys/sdt.h is available (possibly checking for the dtrace script and whether the compiler is capable of building with DTRACE_PROBE macros). That is how for example qemu, java hotspot or libreoffice do it. The default sys/sdt.h header should match the toolchain and user space you are using. It looks to me that for Debian that should be the one from sdt-dev, not the fbsd specific kernel header. That way all the default tools can get and use the SDT probes. But if you think that on kfreebsd programs should be build against a different sys/sdt.h that is fine too (but then programs like gdb will not work with the SDT probes in programs and libraries or will be less efficient when handling things in glibc and libgcc). All that is really needed is that the sdt-dev package can be made arch:all. Whether or not it is installed by default and/or whether it provides the default sys/sdt.h alternative header is secondary. I don't know enough about Debian packaging to suggest the right course of action of making the package arch:all. But that is what Bug#726248 is really about as I understood it. systemtap-sdt-dev: should be "Arch: all" so gcc and libc can B-D on it. How did you resolve the conflicts between kfreebsd-kernel-headers and glibc-headers? Maybe the same solution can be used here? Does any user space program Debian actually use the kfreebsd sys/sdt.h variant? Are there any programs that can use the SDT probes it generates? If not, then maybe you can just not install it for now and get the conflict between the packages resolved? Cheers, Mark -- To UNSUBSCRIBE, email to debian-bsd-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20131019160639.ga17...@toonder.wildebeest.org
Re: Bug#726248: sdt.h conflict with kfreebsd-kernel-headers and systemtap-sdt-dev
On Sat, 2013-10-19 at 02:21 +0200, Robert Millan wrote: > On 18/10/2013 22:18, Timo Juhani Lindfors wrote: > > It is certainly meant to be usable for software that wants to use SDT > > probes (like glibc in this example) and software that wants to > > read/inspect the SDT probes embedded in other software (like gdb in this > > example). > > So the SDT probes provided by systemtap-sdt-dev are not kernel-specific? No, they are not. Just like I suspect the variant in the kfreebsd-kernel-headers isn't. They are for programs that want to define Staticly Defined Tracepoints for user applications. > It seems the namespace collision is not by chance. The in > systemtap is probably inspired by a DTrace version of , which > is what FreeBSD kernel headers seem to be providing. Indeed. They were created so that for programs that include sys/sdt.h and define DTRACE_PROBE macros everything just works. Same include, same macros, just a recompile against the gnu toolchain. Then all the SDT tracepoints in the program are available to all the tools that can read them. > How compatible are the different implementations? They are meant to be source compatible so that if programs use DTRACE_PROBE macros they get build with SDT probes that systemtap, perf, gdb, etc. can use to introspect the program. High-level overview: http://tromey.com/blog/?p=687 Implementation bits that describe the ELF sections created: https://sourceware.org/systemtap/wiki/UserSpaceProbeImplementation > If they're meant to be > bit-by-bit compatible, then it should be possible for systemtap-sdt-dev > to just skip this file on kfreebsd-* so that applications will use the > FreeBSD version. Do you think this could work? I suspect, but haven't checked, that the sys/sdt.h variant in kfreebsd-kernel-headers is also source compatible, but might produce different ELF section bits, so that tools like gdb won't be able to read the SDT probes when that version is used. So, it should be possible to install either of the variants to build depending on the user space tools one wants/needs. The case given in the bug is glibc and gdb which I believe are both used on the Debian kfreebsd. But the real goal is just to have the sdt-dev package be arch:any. They don't have to be used on the Debian kfreebsd arch if really not needed, as long as they don't conflict in such a way that that would prevent the package being arch:any. Which is why I suggested extracting your sys/sdt.h variant also in a separate package (not being build essential), so that either can be installed (even if they would conflict with each other). But note that I am not a Debian packager, so I might miss some other more obvious solution. -- To UNSUBSCRIBE, email to debian-bsd-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1382144237.4472.9240.ca...@bordewijk.wildebeest.org
