Re: MacPorts vs. Apple compiler issues, Handle
Hi, Sergio Had wrote: Could you refer me to the beginning of this discussion please? it started on the Mac Users mailing list, you can find in the archives! I am also involved in AF project:) Oh, I am curious, how? I am essentially the only active developer currently, except Roy who does its windows fork. Riccardo
Re: MacPorts vs. Apple compiler issues, Handle
Hi, Joshua Root wrote: (Moving to macports-dev as it is a better fit for this topic.) indeed, it is a development issue, although well, not for a MacPorts package (yet?) but use of MP development tools. Issues that only appear at higher optimisation levels also often involve undefined behaviour. Right.. I reduced optimization to O1 with no change, I have strange issues compiling with O0! Thread 0 Crashed:: Dispatch queue: com.apple.main-thread 0 XUL 0x00010f5468e1 nsWindowWatcher::OpenWindowInternal(mozIDOMWindowProxy*, char const*, char const*, char const*, bool, bool, bool, nsITabParent*, nsIArray*, nsIDocShellLoadInfo*, mozIDOMWindowProxy**) + 273 And this is where it happened. Since this is not a full debug build, there is no line number information, but you at least know which method is doing the bad memory access. But it should be a debug build. Well a build with debug symbols (not a firefox-style debug which adds also a lot of debug code). I add: ac_add_options --disable-strip and this helps on Linux usually. Still, the nsWindowWatcher class gave me a clue and I found a couple of Firefox patches to import which initialized parameters, checked them, etc and now the error changed to> 0 XUL 0x0001035f5c44 JS::Rooted::registerWithRootLists(js::RootLists&) + 20 1 ??? 0x7ffeecb477f0 0 + 140732869670896 this is bad, since it is inside the JS engine. Also the JS engine works on other system when compiled with modern clang and gcc! Also here I don't have a class name which maps directly to a file which I can easily inspect. I will try clang 7 & 8, just so. Riccardo
Re: MacPorts vs. Apple compiler issues, Handle
(Moving to macports-dev as it is a better fit for this topic.) On 18/3/2024 22:50, Riccardo Mottola wrote: I will do another compilation reducing the optimization level. GCC has an issue where beyond gcc6 certain optimizations need to be disabled, or AF crashes. Issues that only appear at higher optimisation levels also often involve undefined behaviour. Exception Type: EXC_BAD_ACCESS (SIGSEGV) Exception Codes: KERN_INVALID_ADDRESS at 0x7ffe2007 So here is what happened: SIGSEGV means the program tried to access memory that it should not have. The page was not mapped or had the wrong permissions for what it was trying to do. The memory address that it attempted to access is also shown. Thread 0 Crashed:: Dispatch queue: com.apple.main-thread 0 XUL 0x00010f5468e1 nsWindowWatcher::OpenWindowInternal(mozIDOMWindowProxy*, char const*, char const*, char const*, bool, bool, bool, nsITabParent*, nsIArray*, nsIDocShellLoadInfo*, mozIDOMWindowProxy**) + 273 And this is where it happened. Since this is not a full debug build, there is no line number information, but you at least know which method is doing the bad memory access. - Josh
Re: MacPorts vs. Apple compiler issues, Handle
Hi Joshua, Joshua Root wrote: Yes, any number of programs have had bugs that only became apparent when building with a newer compiler. But that's a very broad category, so without knowing the nature of the crash you're seeing, it's impossible to say what else might have had a similar problem. I imagined that, but perhaps somebody experienced like a recurring issue! On FreeBSD I compile with clang 12 and it works, so they issue could be Apple specific, either in the Apple code part or in dependent SDK and libraries. Bugs in clang are not unknown, but far more common is programs relying on certain behaviour for operations that have undefined behaviour according to the language standard. Or sometimes the layout of memory just shifts slightly, causing a previously benign buffer overflow to do damage. Sometimes a newer compiler will also default to a newer version of the language, so if that's the cause of the differing behaviour, you can use -std to change it back. I did try clang 10 and it compiles, but crashes. So probably it is not a compiler bug (although sometimes the remain undetected for years...). I will do another compilation reducing the optimization level. GCC has an issue where beyond gcc6 certain optimizations need to be disabled, or AF crashes. Otherwise, you have to do the hard work of debugging: Look at the crash report to see where and how the code is crashing, pay attention to compiler warnings, and systematically eliminate possibilities of what could be going wrong. Building with clang's sanitizers may be helpful, particularly -fsanitize=address and -fsanitize=undefined, though in a code base this size I wouldn't be surprised if they also detected hundreds of other potential issues. Indeed, it is a big code base, with hundreds of warnings... I don't get very smart on the crash report either. Where is the issue? I don't see a trace and don't know how to read it. Sorry to admit that! Process: arcticfox [93365] Path: /Users/USER/Documents/*/ArcticFox.app/Contents/MacOS/arcticfox Identifier: org.mozilla.arctic fox Version: 44.0 (4424.3.18) Code Type: X86-64 (Native) Parent Process: Python [93301] Responsible: arcticfox [93365] User ID: 501 Date/Time: 2024-03-18 11:52:05.968 +0100 OS Version: Mac OS X 10.13.6 (17G14042) Report Version: 12 Anonymous UUID: 195FD101-66EF-B75D-42D8-012488A629B7 Time Awake Since Boot: 49000 seconds System Integrity Protection: disabled Crashed Thread: 0 Dispatch queue: com.apple.main-thread Exception Type: EXC_BAD_ACCESS (SIGSEGV) Exception Codes: KERN_INVALID_ADDRESS at 0x7ffe2007 Exception Note: EXC_CORPSE_NOTIFY Termination Signal: Segmentation fault: 11 Termination Reason: Namespace SIGNAL, Code 0xb Terminating Process: exc handler [0] VM Regions Near 0x7ffe2007: Stack 7531-75392000 [ 520K] rw-/rwx SM=COW thread 29 --> MALLOC_TINY 7ffe25c0-7ffe2600 [ 4096K] rw-/rwx SM=PRV Thread 0 Crashed:: Dispatch queue: com.apple.main-thread 0 XUL 0x00010f5468e1 nsWindowWatcher::OpenWindowInternal(mozIDOMWindowProxy*, char const*, char const*, char const*, bool, bool, bool, nsITabParent*, nsIArray*, nsIDocShellLoadInfo*, mozIDOMWindowProxy**) + 273 Thread 1: 0 libsystem_kernel.dylib 0x7fff50c8728a __workq_kernreturn + 10 1 libsystem_pthread.dylib 0x7fff50e4e20e _pthread_wqthread + 1552 2 libsystem_pthread.dylib 0x7fff50e4dbe9 start_wqthread + 13 Thread 2: 0 libsystem_kernel.dylib 0x7fff50c8728a __workq_kernreturn + 10 1 libsystem_pthread.dylib 0x7fff50e4e009 _pthread_wqthread + 1035 2 libsystem_pthread.dylib 0x7fff50e4dbe9 start_wqthread + 13 Thread 3: 0 libsystem_kernel.dylib 0x7fff50c8728a __workq_kernreturn + 10 1 libsystem_pthread.dylib 0x7fff50e4e20e _pthread_wqthread + 1552 2 libsystem_pthread.dylib 0x7fff50e4dbe9 start_wqthread + 13 Thread 4: 0 libsystem_kernel.dylib 0x7fff50c8728a __workq_kernreturn + 10 1 libsystem_pthread.dylib 0x7fff50e4e20e _pthread_wqthread + 1552 2 libsystem_pthread.dylib 0x7fff50e4dbe9 start_wqthread + 13 Thread 5: 0 libsystem_kernel.dylib 0x7fff50c8728a __workq_kernreturn + 10 1 libsystem_pthread.dylib 0x7fff50e4e009 _pthread_wqthread + 1035 2 libsystem_pthread.dylib 0x7fff50e4dbe9 start_wqthread + 13 Thread 6:: Gecko_IOThread 0 libsystem_kernel.dylib 0x7fff50c87bea kevent + 10 1 XUL 0x00010da95bff kq_dispatch + 687 Thread 7:: Socket Thread 0 libsystem_kernel.dylib 0x7fff50c86cf2 __select
Re: MacPorts vs. Apple compiler issues, Handle
On 13/3/2024 20:52, Riccardo Mottola wrote: I found a minor patch by Firefox to solve this namespace collision. So for whatever reason Apple's did differently, it now compiles on all compilers. However clang9 generates a crashing executable. I tried on both 10.11 and 10.13. - Apple clang: works fine - MacPorts clang 7: compiles & works - MacPorts clang 9 : compiles but fails anybody has seen this behaviour in other software, perhaps smaller and easier to test than ArcticFox? Yes, any number of programs have had bugs that only became apparent when building with a newer compiler. But that's a very broad category, so without knowing the nature of the crash you're seeing, it's impossible to say what else might have had a similar problem. Bugs in clang are not unknown, but far more common is programs relying on certain behaviour for operations that have undefined behaviour according to the language standard. Or sometimes the layout of memory just shifts slightly, causing a previously benign buffer overflow to do damage. Sometimes a newer compiler will also default to a newer version of the language, so if that's the cause of the differing behaviour, you can use -std to change it back. Otherwise, you have to do the hard work of debugging: Look at the crash report to see where and how the code is crashing, pay attention to compiler warnings, and systematically eliminate possibilities of what could be going wrong. Building with clang's sanitizers may be helpful, particularly -fsanitize=address and -fsanitize=undefined, though in a code base this size I wouldn't be surprised if they also detected hundreds of other potential issues. - Josh
Re: MacPorts vs. Apple compiler issues, Handle
Hi, Joshua Root wrote: The version difference is less slight than you might think. Apple clang version numbers are unrelated to the version number of the llvm.org release they are based on. Apple clang 9.1.0 is closest to the clang-5.0 port. The name collision looks genuine however. I can only assume that MacTypes.h is not included or is preprocessed differently based on different defines in the other compiler. I found a minor patch by Firefox to solve this namespace collision. So for whatever reason Apple's did differently, it now compiles on all compilers. However clang9 generates a crashing executable. I tried on both 10.11 and 10.13. - Apple clang: works fine - MacPorts clang 7: compiles & works - MacPorts clang 9 : compiles but fails anybody has seen this behaviour in other software, perhaps smaller and easier to test than ArcticFox? Riccardo
Re: MacPorts vs. Apple compiler issues, Handle
Hi Joshua, Joshua Root wrote: MacPort installs: $ clang-mp-9.0 --version clang version 9.0.1 Apple: $ clang --version Apple LLVM version 9.1.0 (clang-902.0.39.1) Is there some Apple "trick" or is it the slight compiler difference in version? The version difference is less slight than you might think. Apple clang version numbers are unrelated to the version number of the llvm.org release they are based on. Apple clang 9.1.0 is closest to the clang-5.0 port. Aha, I supposed that the number in parenthesis would match, but not. Do you think it make sense for me to try clang5 or clang6? The name collision looks genuine however. I can only assume that MacTypes.h is not included or is preprocessed differently based on different defines in the other compiler. Do you think an issue in MacPorts compiler or a problem with the code? On Linux and FreeBSD ArcticFox compiles with gcc13 and clang10, but this is an issue specific to the Mac code path, so not directly comparable. I checked XPCShellEnvironment.cpp, there is no direct inclusion of MacTypes.h Riccardo
Re: MacPorts vs. Apple compiler issues, Handle
MacPort installs: $ clang-mp-9.0 --version clang version 9.0.1 Apple: $ clang --version Apple LLVM version 9.1.0 (clang-902.0.39.1) Is there some Apple "trick" or is it the slight compiler difference in version? The version difference is less slight than you might think. Apple clang version numbers are unrelated to the version number of the llvm.org release they are based on. Apple clang 9.1.0 is closest to the clang-5.0 port. The name collision looks genuine however. I can only assume that MacTypes.h is not included or is preprocessed differently based on different defines in the other compiler. - Josh
