Re: failure of pructl (atexit/_Block_copy/--no-allow-shlib-undefined)
David Chisnall wrote this message on Thu, Dec 02, 2021 at 10:34 +: > On 02/12/2021 09:51, Dimitry Andric wrote: > > Apparently the "block runtime" is supposed to provide the actual object, > > so I guess you have to explicitly link to that runtime? > > The block runtime provides this symbol. You use this libc API, you must > be compiling with a toolchain that supports blocks and must be providing > the blocks symbols. If you don't use `atexit_b` or any of the other > `_b` APIs then you don't need to link the blocks runtime. > > I am not sure why this is causing linker failures - if it's a weak > symbol and it's not defined then that's entirely expected: the point of > a weak symbol is that it might not be defined. This avoids the need to > link libc to the blocks runtime for code that doesn't use blocks (i.e. > most code that doesn't come from macOS). > > This code is not using `atexit_b`, but because it is using `atexit` the > linker is complaining that the compilation unit containing `atexit` is > referring to a symbol that isn't defined. I assume that this failure was due to a recent llvm change, because I haven't received any failures about pructl until Nov 16th, 2021, despite the port and code being untouched since 2020-09-22. Digging in a bit more, it looks like libpru is compiled w/ -fblocks, and so depending upon the _Block_copy symbol, the atexit is just the "closest" symbol that's defined". pructl is not, but even compiling pructl w/ -fblocks, doesn't fix the link error, as it looks like the block runtime isn't linked. If I manually include /usr/lib/libBlocksRuntime.so, then pructl is able to link. I can't seem to find any docs on clang about how to properly compile code that uses blocks, so, unless someone points me to docs on how to compile blocks enable programs, I'll just patch libpru to not use blocks since it seems like blocks is well supported. I don't want to fix this code every few years when things change. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."
Re: [REVIEW] Hide BIT_* macros from userland code
Hey Stefan, On Thu, Dec 02, 2021 at 05:26:55PM +0100, Stefan Esser wrote: > I have created > > https://reviews.freebsd.org/D33235 > > to remove the BIT_* macros used in the kernel from the userland API. > > They conflict with differing definitions in some 3rd party code and > lead to compile issues in a number of ports (via CPU_* macros based > on the BIT_* macros). > > See PR259787 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=259787 > for an example of such a problem. I recently was in a position to evaluate BIT_* macros for userland use. It was around the time when the conversation regarding hiding BIT_* from userland, which conversation caused me to find another solution. I think such an API is incredibly useful, so I wonder if there's a way to satisfy both. For example, maybe prefix the userland side with a USERLAND_ or something similar? Kernel would use BIT_* and userland would use USERLAND_BIT_* (just spitballing, not actually advocating for "USERLAND_BIT_*" but rather just the idea of it.) Thanks, -- Shawn Webb Cofounder / Security Engineer HardenedBSD https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc signature.asc Description: PGP signature
[REVIEW] Hide BIT_* macros from userland code
I have created https://reviews.freebsd.org/D33235 to remove the BIT_* macros used in the kernel from the userland API. They conflict with differing definitions in some 3rd party code and lead to compile issues in a number of ports (via CPU_* macros based on the BIT_* macros). See PR259787 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=259787 for an example of such a problem. OpenPGP_signature Description: OpenPGP digital signature
Re: failure of pructl (atexit/_Block_copy/--no-allow-shlib-undefined)
On 02/12/2021 09:51, Dimitry Andric wrote: Apparently the "block runtime" is supposed to provide the actual object, so I guess you have to explicitly link to that runtime? The block runtime provides this symbol. You use this libc API, you must be compiling with a toolchain that supports blocks and must be providing the blocks symbols. If you don't use `atexit_b` or any of the other `_b` APIs then you don't need to link the blocks runtime. I am not sure why this is causing linker failures - if it's a weak symbol and it's not defined then that's entirely expected: the point of a weak symbol is that it might not be defined. This avoids the need to link libc to the blocks runtime for code that doesn't use blocks (i.e. most code that doesn't come from macOS). This code is not using `atexit_b`, but because it is using `atexit` the linker is complaining that the compilation unit containing `atexit` is referring to a symbol that isn't defined. David
Re: Problem compiling ports
Confirmed. Thanks for your quick reaction! On Tue, 30 Nov 2021 22:45:37 + Brooks Davis wrote: > Yeah, I've got this in progress. > > -- Brooks > > On Wed, Dec 01, 2021 at 06:57:56AM +0900, Tomoaki AOKI wrote: > > It would be better making new special handling like BE_AMDGPU for it. > > Building with BE_STANDARD just for this would be a pain for some users. > > > > (CC'ing freebsd-ports ML.) > > > > > > On Tue, 30 Nov 2021 17:45:30 + > > Brooks Davis wrote: > > > > > In the config for devel/llvm11, is BE_STANDARD enabled? If not, you > > > won't have the web assembly backend. > > > > > > -- Brooks > > > > > > On Tue, Nov 30, 2021 at 05:24:45PM +, Filippo Moretti via current > > > wrote: > > > > error: unable to create target: 'No available targets are compatible > > > > with triple "wasm32-unknown-wasi"' > > > > 1 error generated. > > > > gmake[3]: *** [Makefile:380: > > > > /usr/ports/devel/wasi-libc/work/wasi-libc-ad5133410f66b93a2381db5b542aad5e0964db96/build/dlmalloc/src/dlmalloc.o] > > > > Error 1 > > > > error: unable to create target: 'No available targets are compatible > > > > with triple "wasm32-unknown-wasi"' > > > > 1 error generated. > > > > gmake[3]: *** [Makefile:440: startup_files] Error 1 > > > > gmake[3]: Leaving directory > > > > '/usr/ports/devel/wasi-libc/work/wasi-libc-ad5133410f66b93a2381db5b542aad5e0964db96' > > > > ===> Compilation failed unexpectedly. > > > > Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the > > > > failure to > > > > the maintainer. > > > > *** Error code 1 > > > > > > > > Stop. > > > > make[2]: stopped in /usr/ports/devel/wasi-libc > > > > *** Error code 1 > > > > > > > > Stop. > > > > make[1]: stopped in /usr/ports/devel/wasi-libcxx*** Error code 1 > > > > > > > > Stop. > > > > make: stopped in /usr/ports/www/firefox > > > > > > > > ===>>> make build failed for www/firefox > > > > ===>>> Aborting update > > > > > > > > > > > > ??File "/usr/local/lib/python3.8/site-packages/setuptools/dist.py", > > > > line 799, in > > > > ?? eps = map(lambda e: e.load(), > > > > pkg_resources.iter_entry_points(group)) > > > > ?? File > > > > "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", > > > > line 2449, in load > > > > ?? self.require(*args, **kwargs) > > > > ?? File > > > > "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", > > > > line 2472, in require > > > > ?? items = working_set.resolve(reqs, env, installer, > > > > extras=self.extras) > > > > ?? File > > > > "/usr/local/lib/python3.8/site-packages/pkg_resources/__init__.py", > > > > line 772, in resolve > > > > ?? raise DistributionNotFound(req, requirers) > > > > pkg_resources.DistributionNotFound: The 'packaging>=20.0' distribution > > > > was not found and is required by the application > > > > *** Error code 1 > > > > > > > > Stop. > > > > make: stopped in /usr/ports/devel/py-pycparser > > > > > > > > ===>>> make build failed for devel/py-pycparser@py38 > > > > ===>>> Aborting update > > > > > > > > FreeBSD sting 14.0-CURRENT FreeBSD 14.0-CURRENT #62 > > > > heads/main-n251146-d109559ddbf: Mon Nov 29 12:18:48 CET 2021 > > > > root@sting:/usr/obj/usr/src/amd64.amd64/sys/STING?? amd64 > > > > > > > > > > > > > > -- > > Tomoaki AOKI > > -- 青木 知明 [Tomoaki AOKI]
Re: failure of pructl (atexit/_Block_copy/--no-allow-shlib-undefined)
On 2 Dec 2021, at 06:42, Kyle Evans wrote:
> On Wed, Dec 1, 2021 at 8:05 PM John-Mark Gurney wrote:
>>
>> Hello,
>>
>> It seems like the recent changes to make --no-allow-shlib-undefined
>> broke pructl.
>>
>> lib/libc/stdlib/atexit.c uses a weak _Block_copy symbol, but
>> pructl does not use atexit_b, and yet gets the following error:
>> : && /usr/bin/cc -Werror -O2 -pipe -fstack-protector-strong -isystem
>> /usr/local/include -fno-strict-aliasing -std=c99 -fstack-protector-strong
>> CMakeFiles/pructl.dir/pructl.c.o -o pructl -Wl,-rpath,/usr/local/lib:
>> /usr/local/lib/libpru.so && :
>> ld: error: /lib/libc.so.7: undefined reference to _Block_copy
>> [--no-allow-shlib-undefined]
>> cc: error: linker command failed with exit code 1 (use -v to see invocation)
>>
>> What is the correct fix? It seems like atexit.c or the linker should
>> be fixed, as pructl doesn't use atexit_b at all.
>>
>
> CC dim@ and jrtc27@... this seems like a toolchain regression? We're
> relying on the address of weak _Block_copy to simply evaluate to NULL
> if it's undefined here, which seems legit and pretty well-defined at
> this point from my recollection.
Naive patch, which appears to work:
diff --git a/devel/pructl/files/patch-CMakeLists.txt
b/devel/pructl/files/patch-CMakeLists.txt
index f378dd44e6f6..8a41cc91aba8 100644
--- a/devel/pructl/files/patch-CMakeLists.txt
+++ b/devel/pructl/files/patch-CMakeLists.txt
@@ -1,9 +1,16 @@
--- CMakeLists.txt.orig2018-12-24 20:28:37 UTC
+++ CMakeLists.txt
-@@ -8,5 +8,5 @@ find_library(libedit NAMES edit)
+@@ -4,9 +4,10 @@ add_executable(pructl pructl.c)
+ add_executable(prudbg prudbg.c)
+ include_directories(/usr/local/include ${CMAKE_SOURCE_DIR}/../libpru)
+ find_library(libpru NAMES pru PATHS /usr/local/lib
${CMAKE_SOURCE_DIR}/../libpru/build)
++find_library(libBlocksRuntime NAMES BlocksRuntime)
+ find_library(libedit NAMES edit)
find_library(libutil NAMES util)
- target_link_libraries(pructl ${libpru})
- target_link_libraries(prudbg ${libpru} ${libedit} ${libutil})
+-target_link_libraries(pructl ${libpru})
+-target_link_libraries(prudbg ${libpru} ${libedit} ${libutil})
-set(CMAKE_C_FLAGS "-Weverything -Werror")
++target_link_libraries(pructl ${libpru} ${libBlocksRuntime})
++target_link_libraries(prudbg ${libpru} ${libBlocksRuntime} ${libedit}
${libutil})
+set(CMAKE_C_FLAGS "-Werror")
install(TARGETS pructl prudbg DESTINATION sbin)
-Dimitry
signature.asc
Description: Message signed with OpenPGP
Re: failure of pructl (atexit/_Block_copy/--no-allow-shlib-undefined)
On 2 Dec 2021, at 06:42, Kyle Evans wrote: > > On Wed, Dec 1, 2021 at 8:05 PM John-Mark Gurney wrote: >> >> Hello, >> >> It seems like the recent changes to make --no-allow-shlib-undefined >> broke pructl. >> >> lib/libc/stdlib/atexit.c uses a weak _Block_copy symbol, but >> pructl does not use atexit_b, and yet gets the following error: >> : && /usr/bin/cc -Werror -O2 -pipe -fstack-protector-strong -isystem >> /usr/local/include -fno-strict-aliasing -std=c99 -fstack-protector-strong >> CMakeFiles/pructl.dir/pructl.c.o -o pructl -Wl,-rpath,/usr/local/lib: >> /usr/local/lib/libpru.so && : >> ld: error: /lib/libc.so.7: undefined reference to _Block_copy >> [--no-allow-shlib-undefined] >> cc: error: linker command failed with exit code 1 (use -v to see invocation) >> >> What is the correct fix? It seems like atexit.c or the linker should >> be fixed, as pructl doesn't use atexit_b at all. >> > > CC dim@ and jrtc27@... this seems like a toolchain regression? We're > relying on the address of weak _Block_copy to simply evaluate to NULL > if it's undefined here, which seems legit and pretty well-defined at > this point from my recollection. What do you mean exactly with "here"? In libc? In atexit? In libpru? I can't make it out from the context, sorry. :) As far as I can see, _Block_copy has always been a weak undefined symbol in libc.so: 4: 0 NOTYPE WEAK DEFAULT UND _Block_copy Apparently the "block runtime" is supposed to provide the actual object, so I guess you have to explicitly link to that runtime? I know next to nothing about the blocks stuff, so it's all pretty much unknown territory to me... :) -Dimitry signature.asc Description: Message signed with OpenPGP
sys/conf: Simplify configuration files
Hi fellow kernel developers, I have two patches for config(8) which will allow us to group filenames in sys/conf/files* among others. Any strong opinions about the following change? https://reviews.freebsd.org/D33224 --HPS
