Re: Is --with-zlib=bundled broken on MacOS aarch64 12.2.1?
Hello Magnus, On 28/03/22 5:21 pm, Magnus Ihse Bursie wrote: On 2022-03-28 09:03, David Holmes wrote: On 28/03/2022 4:56 pm, Alan Bateman wrote: On 28/03/2022 07:46, David Holmes wrote: Hi Jai, It isn't obvious to me that the bundled sources are actually intended to build on macOS. There's no include of unistd.h to get the lseek definition. I think the context here is that Jai is chasing an issue that may be bug in the libz on macOS. Building the bundled version and comparing results would lead to useful information. AFAIK, the system zlib has always been used since 7u4 when the macOS was added. I think the Apple JDK did the same. So there may be a small bit of "porting" to do, adds include files, etc. Okay, I suggest adding the include of unistd.h as a starter then. But Jai should note that this is not a build issue per-se - making those sources buildable on all desired platforms is the job of the owners of that src in the OpenJDK. I agree fully, but I'd also like to add that it is not clear that this is not "supposed" to work. If bundled zlib is not buildable on macOS (that was news to me), then the correct build system behavior should have been to block it in configure. So, Jaikiran, if you intend to get this to work on macOS, great! If you're not taking that route, please let me know so I can open a bug on prohibiting bundled zlib on maccOS from configure. I used David's suggestion to add that include header: diff --git a/src/java.base/share/native/libzip/zlib/gzlib.c b/src/java.base/share/native/libzip/zlib/gzlib.c index a814dd8c7b6..9df629d4205 100644 --- a/src/java.base/share/native/libzip/zlib/gzlib.c +++ b/src/java.base/share/native/libzip/zlib/gzlib.c @@ -35,6 +35,7 @@ #if defined(_LARGEFILE64_SOURCE) && _LFS64_LARGEFILE-0 # define LSEEK lseek64 #else +# include # define LSEEK lseek #endif #endif That did help me get past the lseek error, but there are some more errors (as noted in that log) which need to be addressed too. I don't have the necessary knowledge of C ecosystem to decide what set of includes and whether those includes need to be conditional for macOS, are required to get this building. The other thing to consider with this code is that it is bundled code and the real code lies in a separate upstream zlib project[1]. So I think whatever changes we do here might have to be co-ordinated back to that project. Please go ahead with your plan to create a JBS issue to prohibit this combination in the meantime. What surprises me a bit is that looking at the commit history within this source area, there haven't been any commits in this code for around 4 years now. So I suspect this hasn't been working at least that long or maybe the compiler rules got more stricter during that period. [1] https://github.com/madler/zlib/tree/v1.2.11 -Jaikiran
Re: RFR: 8242181: [Linux] Show source information when printing native stack traces in hs_err files [v5]
On Mon, 28 Mar 2022 11:18:36 GMT, Christian Hagedorn wrote: >> As I said before, I think it would be nice to share DWARF parser between SA >> and HotSpot. Can you expose these mechanisms? It may be another RFE, and may >> need to think other platforms. > >> As I said before, I think it would be nice to share DWARF parser between SA >> and HotSpot. Can you expose these mechanisms? It may be another RFE, and may >> need to think other platforms. > > That would be good to have. However, I'm not familiar with the SA code and > how it works to share code with HotSpot. And I'm also not sure how much > overlap the two parsers actually have. I quickly skimmed through the DWARF > parsing code of the SA and it seems that its main usage is for parsing call > frame information (as described in section 6.4 of the DWARF 4 spec) which is > not supported/needed in this patch. There is still some code that could be > shared though like opening a DWARF file with its checks or reading an LEB 128 > etc. Might be worth to investigate further if the two implementations can be > merged/reused to some extent. But I propose to file a separate RFE for that. > What do you think? @chhagedorn > There is still some code that could be shared though like opening a DWARF > file with its checks or reading an LEB 128 etc. Might be worth to investigate > further if the two implementations can be merged/reused to some extent. But I > propose to file a separate RFE for that. What do you think? Yeah, let's investigate about it in another RFE. IMHO we can share some codes about DWARF between HotSpot and SA, and also we might need DWARF-based call frame parser in HotSpot because some 3rd-party native libraries don't use base pointer (RBP) to store SP due to optimization. In SA side, it would be useful if we can check native source file and line number in mixed jstack with your change. So I want to unify DWARF parser (processor) between HotSpot and SA, but it might be long journey... thus I agree with you to file it as another RFE. - PR: https://git.openjdk.java.net/jdk/pull/7126
Re: RFR: 8242181: [Linux] Show source information when printing native stack traces in hs_err files [v5]
On Mon, 28 Mar 2022 12:58:20 GMT, Thomas Stuefe wrote:
>> Christian Hagedorn has updated the pull request with a new target base due
>> to a merge or a rebase. The pull request now contains 54 commits:
>>
>> - Updating some comments
>> - Cleanup loading dwarf file and add summary
>> - Review comments of first pass by Thomas except dwarf file loading
>> - Merge branch 'master' into JDK-8242181
>> - Make dwarf tag NOT_PRODUCT
>> - Change log_* to log_develop_* and log_warning to log_develop_info
>> - Update test/hotspot/jtreg/runtime/ErrorHandling/TestDwarf.java
>>
>>Co-authored-by: Erik Joelsson <37597443+erik...@users.noreply.github.com>
>> - Update test/hotspot/jtreg/runtime/ErrorHandling/TestDwarf.java
>>
>>Co-authored-by: Erik Joelsson <37597443+erik...@users.noreply.github.com>
>> - Better formatting of trace output
>> - some code move and more cleanups
>> - ... and 44 more:
>> https://git.openjdk.java.net/jdk/compare/efd3967b...5bea4841
>
> src/hotspot/share/utilities/elfFile.cpp line 450:
>
>> 448: if (buf == nullptr) {
>> 449: return false;
>> 450: }
>
> I'd move this close to and local to where it is used.
>
> Also, you seem to repeat the same pattern a lot "NEW_RESOURCE_ARRAY(n), if
> error return something". I'd factor this out to an utility function or
> utility macro, maybe one where you pass the error return value as macro
> parameter.
Thomas's comment caught my attention in the email. NEW_RESOURCE_ARRAY aborts
the VM on OOM. Use NEW_RESOURCE_ARRAY_RETURN_NULL if you want to continue.
-
PR: https://git.openjdk.java.net/jdk/pull/7126
Re: RFR: 8283723: Update Visual Studio 2022 to version 17.1.0 for Oracle builds on Windows [v3]
On Mon, 28 Mar 2022 16:05:03 GMT, Mikael Vidstedt wrote: >> make/devkit/createWindowsDevkit.sh line 74: >> >>> 72: # Work around the insanely named ProgramFiles(x86) env variable >>> 73: PROGRAMFILES_X86="$($WINDOWS_PATH_TO_UNIX_PATH "$(cmd.exe /c set | sed >>> -n 's/^ProgramFiles(x86)=//p' | tr -d '\r')")" >>> 74: PROGRAMFILES="$($WINDOWS_PATH_TO_UNIX_PATH "$(cmd.exe /c set | sed -n >>> 's/^PROGRAMFILES=//p' | tr -d '\r')")" >> >> This complex dance isn't necessary; it's a special operation for just the >> "ProgramFiles(x86)" variable, since cygwin can't handle environment >> variables with `()` in the name. (See the comment above as well) >> >> I think you will be able to access it with just a simple `$PROGRAMFILES`. > > Good point, thank you. I removed that whole line and verified that creating > the devkit still works. While it did seem to work I realized that `PROGRAMFILES` wasn't converted to a unix path anymore, so I added back that conversion but without the special handling needed for the x86 variable. - PR: https://git.openjdk.java.net/jdk/pull/7974
Re: RFR: 8283723: Update Visual Studio 2022 to version 17.1.0 for Oracle builds on Windows [v3]
On Mon, 28 Mar 2022 16:40:31 GMT, Mikael Vidstedt wrote: >> Oracle is updating the version of Visual Studio for building the JDK on >> Windows to Visual Studio 2022 17.1.0. >> >> This change adds support for building devkits based on VS 2022. Instead of >> creating a new script file for that I decided to combine it with the logic >> in createWindowsDevkit2019.sh and rename the resulting file. I decided to >> dropped support for VS 2017 since that version is pretty old now, let me >> know if you'd prefer to see it kept around. >> >> Testing: tier1-5 + additional testing. > > Mikael Vidstedt has updated the pull request incrementally with one > additional commit since the last revision: > > Convert PROGRAMFILES to unix path Marked as reviewed by erikj (Reviewer). - PR: https://git.openjdk.java.net/jdk/pull/7974
Re: RFR: 8283723: Update Visual Studio 2022 to version 17.1.0 for Oracle builds on Windows [v3]
> Oracle is updating the version of Visual Studio for building the JDK on > Windows to Visual Studio 2022 17.1.0. > > This change adds support for building devkits based on VS 2022. Instead of > creating a new script file for that I decided to combine it with the logic in > createWindowsDevkit2019.sh and rename the resulting file. I decided to > dropped support for VS 2017 since that version is pretty old now, let me know > if you'd prefer to see it kept around. > > Testing: tier1-5 + additional testing. Mikael Vidstedt has updated the pull request incrementally with one additional commit since the last revision: Convert PROGRAMFILES to unix path - Changes: - all: https://git.openjdk.java.net/jdk/pull/7974/files - new: https://git.openjdk.java.net/jdk/pull/7974/files/8a1072cb..5a3bd52b Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=7974&range=02 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=7974&range=01-02 Stats: 1 line in 1 file changed: 1 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/7974.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/7974/head:pull/7974 PR: https://git.openjdk.java.net/jdk/pull/7974
Re: RFR: 8283723: Update Visual Studio 2022 to version 17.1.0 for Oracle builds on Windows [v2]
> Oracle is updating the version of Visual Studio for building the JDK on > Windows to Visual Studio 2022 17.1.0. > > This change adds support for building devkits based on VS 2022. Instead of > creating a new script file for that I decided to combine it with the logic in > createWindowsDevkit2019.sh and rename the resulting file. I decided to > dropped support for VS 2017 since that version is pretty old now, let me know > if you'd prefer to see it kept around. > > Testing: tier1-5 + additional testing. Mikael Vidstedt has updated the pull request incrementally with one additional commit since the last revision: Remove PROGRAMFILES workaround - Changes: - all: https://git.openjdk.java.net/jdk/pull/7974/files - new: https://git.openjdk.java.net/jdk/pull/7974/files/6413e7ff..8a1072cb Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=7974&range=01 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=7974&range=00-01 Stats: 1 line in 1 file changed: 0 ins; 1 del; 0 mod Patch: https://git.openjdk.java.net/jdk/pull/7974.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/7974/head:pull/7974 PR: https://git.openjdk.java.net/jdk/pull/7974
Re: RFR: 8283723: Update Visual Studio 2022 to version 17.1.0 for Oracle builds on Windows [v2]
On Mon, 28 Mar 2022 11:54:28 GMT, Magnus Ihse Bursie wrote: >> Mikael Vidstedt has updated the pull request incrementally with one >> additional commit since the last revision: >> >> Remove PROGRAMFILES workaround > > make/devkit/createWindowsDevkit.sh line 74: > >> 72: # Work around the insanely named ProgramFiles(x86) env variable >> 73: PROGRAMFILES_X86="$($WINDOWS_PATH_TO_UNIX_PATH "$(cmd.exe /c set | sed >> -n 's/^ProgramFiles(x86)=//p' | tr -d '\r')")" >> 74: PROGRAMFILES="$($WINDOWS_PATH_TO_UNIX_PATH "$(cmd.exe /c set | sed -n >> 's/^PROGRAMFILES=//p' | tr -d '\r')")" > > This complex dance isn't necessary; it's a special operation for just the > "ProgramFiles(x86)" variable, since cygwin can't handle environment variables > with `()` in the name. (See the comment above as well) > > I think you will be able to access it with just a simple `$PROGRAMFILES`. Good point, thank you. I removed that whole line and verified that creating the devkit still works. - PR: https://git.openjdk.java.net/jdk/pull/7974
Re: RFR: 8242181: [Linux] Show source information when printing native stack traces in hs_err files [v5]
On Mon, 28 Mar 2022 12:20:44 GMT, Thomas Stuefe wrote:
>> Christian Hagedorn has updated the pull request with a new target base due
>> to a merge or a rebase. The pull request now contains 54 commits:
>>
>> - Updating some comments
>> - Cleanup loading dwarf file and add summary
>> - Review comments of first pass by Thomas except dwarf file loading
>> - Merge branch 'master' into JDK-8242181
>> - Make dwarf tag NOT_PRODUCT
>> - Change log_* to log_develop_* and log_warning to log_develop_info
>> - Update test/hotspot/jtreg/runtime/ErrorHandling/TestDwarf.java
>>
>>Co-authored-by: Erik Joelsson <37597443+erik...@users.noreply.github.com>
>> - Update test/hotspot/jtreg/runtime/ErrorHandling/TestDwarf.java
>>
>>Co-authored-by: Erik Joelsson <37597443+erik...@users.noreply.github.com>
>> - Better formatting of trace output
>> - some code move and more cleanups
>> - ... and 44 more:
>> https://git.openjdk.java.net/jdk/compare/efd3967b...5bea4841
>
> src/hotspot/share/utilities/elfFile.hpp line 217:
>
>> 215:public:
>> 216: DwarfFilePath(const char* filename, char* buf, size_t buf_len) :
>> _filename(filename), _path(buf), _path_len(buf_len) {
>> 217: size_t offset = (strlen(filename) + 4) >> 2u;
>
> Why not use align()? Would be more readable.
See also comments in load_dwarf_file(). I'd really be more happy with more
verifications.
-
PR: https://git.openjdk.java.net/jdk/pull/7126
Re: RFR: 8242181: [Linux] Show source information when printing native stack traces in hs_err files [v4]
On Thu, 24 Feb 2022 08:14:34 GMT, Thomas Stuefe wrote:
>> Christian Hagedorn has updated the pull request incrementally with one
>> additional commit since the last revision:
>>
>> Make dwarf tag NOT_PRODUCT
>
> src/hotspot/share/utilities/decoder_elf.cpp line 67:
>
>> 65: if (!os::dll_address_to_library_name(pc, filepath, sizeof(filepath),
>> &offset_in_library) || offset_in_library < 0) {
>> 66: // Method not found. offset_in_library should not overflow.
>> 67: log_develop_info(dwarf)("Did not find library for address "
>> INTPTR_FORMAT, p2i(pc));
>
> I know this has been discussed and decided, but I feel uncomfortable about
> this logging here. Also because it sets a precedent for using UL inside
> signal handling.
Note, if you do log, it would be nice to be precise and distinguish between
dll_address_to_library_name returning false and returning an offset outside the
library bounds.
-
PR: https://git.openjdk.java.net/jdk/pull/7126
Re: RFR: 8242181: [Linux] Show source information when printing native stack traces in hs_err files [v5]
On Mon, 28 Feb 2022 16:22:25 GMT, Christian Hagedorn wrote: >> When printing the native stack trace on Linux (mostly done for hs_err >> files), it only prints the method with its parameters and a relative offset >> in the method: >> >> Stack: [0x7f6e01739000,0x7f6e0183a000], sp=0x7f6e01838110, >> free space=1020k >> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native >> code) >> V [libjvm.so+0x620d86] Compilation::~Compilation()+0x64 >> V [libjvm.so+0x624b92] Compiler::compile_method(ciEnv*, ciMethod*, int, >> bool, DirectiveSet*)+0xec >> V [libjvm.so+0x8303ef] >> CompileBroker::invoke_compiler_on_method(CompileTask*)+0x899 >> V [libjvm.so+0x82f067] CompileBroker::compiler_thread_loop()+0x3df >> V [libjvm.so+0x84f0d1] CompilerThread::thread_entry(JavaThread*, >> JavaThread*)+0x69 >> V [libjvm.so+0x1209329] JavaThread::thread_main_inner()+0x15d >> V [libjvm.so+0x12091c9] JavaThread::run()+0x167 >> V [libjvm.so+0x1206ada] Thread::call_run()+0x180 >> V [libjvm.so+0x1012e55] thread_native_entry(Thread*)+0x18f >> >> This makes it sometimes difficult to see where exactly the methods were >> called from and sometimes almost impossible when there are multiple >> invocations of the same method within one method. >> >> This patch improves this by providing source information (filename + line >> number) to the native stack traces on Linux similar to what's already done >> on Windows (see >> [JDK-8185712](https://bugs.openjdk.java.net/browse/JDK-8185712)): >> >> Stack: [0x7f34fca18000,0x7f34fcb19000], sp=0x7f34fcb17110, >> free space=1020k >> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native >> code) >> V [libjvm.so+0x620d86] Compilation::~Compilation()+0x64 >> (c1_Compilation.cpp:607) >> V [libjvm.so+0x624b92] Compiler::compile_method(ciEnv*, ciMethod*, int, >> bool, DirectiveSet*)+0xec (c1_Compiler.cpp:250) >> V [libjvm.so+0x8303ef] >> CompileBroker::invoke_compiler_on_method(CompileTask*)+0x899 >> (compileBroker.cpp:2291) >> V [libjvm.so+0x82f067] CompileBroker::compiler_thread_loop()+0x3df >> (compileBroker.cpp:1966) >> V [libjvm.so+0x84f0d1] CompilerThread::thread_entry(JavaThread*, >> JavaThread*)+0x69 (compilerThread.cpp:59) >> V [libjvm.so+0x1209329] JavaThread::thread_main_inner()+0x15d >> (thread.cpp:1297) >> V [libjvm.so+0x12091c9] JavaThread::run()+0x167 (thread.cpp:1280) >> V [libjvm.so+0x1206ada] Thread::call_run()+0x180 (thread.cpp:358) >> V [libjvm.so+0x1012e55] thread_native_entry(Thread*)+0x18f >> (os_linux.cpp:705) >> >> For Linux, we need to parse the debug symbols which are generated by GCC in >> DWARF - a standardized debugging format. This patch adds support for DWARF >> 4, the default of GCC 10.x, for 32 and 64 bit architectures (tested with >> x86_32, x86_64 and AArch64). DWARF 5 is not supported as it was still >> experimental and not generated for HotSpot. However, newer GCC version may >> soon generate DWARF 5 by default in which case this parser either needs to >> be extended or the build of HotSpot configured to only emit DWARF 4. >> >> The code follows the parsing steps described in the official DWARF 4 spec: >> https://dwarfstd.org/doc/DWARF4.pdf >> I added references to the corresponding sections throughout the code. >> However, I tried to explain the steps from the DWARF spec directly in the >> code (method names, comments etc.). This allows to follow the code without >> the need to actually deep dive into the spec. >> >> The comments at the `Dwarf` class in the `elf.hpp` file explain in more >> detail how a DWARF file is structured and how the parsing algorithm works to >> get to the filename and line number information. There are more class >> comments throughout the `elf.hpp` file about how different DWARF sections >> are structured and how the parsing algorithm needs to fetch the required >> information. Therefore, I will not repeat the exact workings of the >> algorithm here but refer to the code comments. I've tried to add as much >> information as possible to improve the readability. >> >> Generally, I've tried to stay away from adding any assertions as this code >> is almost always executed when already processing a VM error. Instead, the >> DWARF parser aims to just exit gracefully and possibly omit source >> information for a stack frame instead of risking to stop writing the hs_err >> file when an assertion would have failed. To debug failures, `-Xlog:dwarf` >> can be used with `info`, `debug` or `trace` which provides logging messages >> throughout parsing. >> >> **Testing:** >> Apart from manual testing, I've added two kinds of tests: >> - A JTreg test: Spawns new VMs to let them crash in various ways. The test >> reads the created hs_err files to check if the DWARF parsing could correctly >> find the filename and line number. For normal HotSpot files, I could not >> check against hardcoded fi
Re: RFR: 8242181: [Linux] Show source information when printing native stack traces in hs_err files [v4]
On Tue, 8 Feb 2022 08:17:17 GMT, Christian Hagedorn wrote: >> When printing the native stack trace on Linux (mostly done for hs_err >> files), it only prints the method with its parameters and a relative offset >> in the method: >> >> Stack: [0x7f6e01739000,0x7f6e0183a000], sp=0x7f6e01838110, >> free space=1020k >> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native >> code) >> V [libjvm.so+0x620d86] Compilation::~Compilation()+0x64 >> V [libjvm.so+0x624b92] Compiler::compile_method(ciEnv*, ciMethod*, int, >> bool, DirectiveSet*)+0xec >> V [libjvm.so+0x8303ef] >> CompileBroker::invoke_compiler_on_method(CompileTask*)+0x899 >> V [libjvm.so+0x82f067] CompileBroker::compiler_thread_loop()+0x3df >> V [libjvm.so+0x84f0d1] CompilerThread::thread_entry(JavaThread*, >> JavaThread*)+0x69 >> V [libjvm.so+0x1209329] JavaThread::thread_main_inner()+0x15d >> V [libjvm.so+0x12091c9] JavaThread::run()+0x167 >> V [libjvm.so+0x1206ada] Thread::call_run()+0x180 >> V [libjvm.so+0x1012e55] thread_native_entry(Thread*)+0x18f >> >> This makes it sometimes difficult to see where exactly the methods were >> called from and sometimes almost impossible when there are multiple >> invocations of the same method within one method. >> >> This patch improves this by providing source information (filename + line >> number) to the native stack traces on Linux similar to what's already done >> on Windows (see >> [JDK-8185712](https://bugs.openjdk.java.net/browse/JDK-8185712)): >> >> Stack: [0x7f34fca18000,0x7f34fcb19000], sp=0x7f34fcb17110, >> free space=1020k >> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native >> code) >> V [libjvm.so+0x620d86] Compilation::~Compilation()+0x64 >> (c1_Compilation.cpp:607) >> V [libjvm.so+0x624b92] Compiler::compile_method(ciEnv*, ciMethod*, int, >> bool, DirectiveSet*)+0xec (c1_Compiler.cpp:250) >> V [libjvm.so+0x8303ef] >> CompileBroker::invoke_compiler_on_method(CompileTask*)+0x899 >> (compileBroker.cpp:2291) >> V [libjvm.so+0x82f067] CompileBroker::compiler_thread_loop()+0x3df >> (compileBroker.cpp:1966) >> V [libjvm.so+0x84f0d1] CompilerThread::thread_entry(JavaThread*, >> JavaThread*)+0x69 (compilerThread.cpp:59) >> V [libjvm.so+0x1209329] JavaThread::thread_main_inner()+0x15d >> (thread.cpp:1297) >> V [libjvm.so+0x12091c9] JavaThread::run()+0x167 (thread.cpp:1280) >> V [libjvm.so+0x1206ada] Thread::call_run()+0x180 (thread.cpp:358) >> V [libjvm.so+0x1012e55] thread_native_entry(Thread*)+0x18f >> (os_linux.cpp:705) >> >> For Linux, we need to parse the debug symbols which are generated by GCC in >> DWARF - a standardized debugging format. This patch adds support for DWARF >> 4, the default of GCC 10.x, for 32 and 64 bit architectures (tested with >> x86_32, x86_64 and AArch64). DWARF 5 is not supported as it was still >> experimental and not generated for HotSpot. However, newer GCC version may >> soon generate DWARF 5 by default in which case this parser either needs to >> be extended or the build of HotSpot configured to only emit DWARF 4. >> >> The code follows the parsing steps described in the official DWARF 4 spec: >> https://dwarfstd.org/doc/DWARF4.pdf >> I added references to the corresponding sections throughout the code. >> However, I tried to explain the steps from the DWARF spec directly in the >> code (method names, comments etc.). This allows to follow the code without >> the need to actually deep dive into the spec. >> >> The comments at the `Dwarf` class in the `elf.hpp` file explain in more >> detail how a DWARF file is structured and how the parsing algorithm works to >> get to the filename and line number information. There are more class >> comments throughout the `elf.hpp` file about how different DWARF sections >> are structured and how the parsing algorithm needs to fetch the required >> information. Therefore, I will not repeat the exact workings of the >> algorithm here but refer to the code comments. I've tried to add as much >> information as possible to improve the readability. >> >> Generally, I've tried to stay away from adding any assertions as this code >> is almost always executed when already processing a VM error. Instead, the >> DWARF parser aims to just exit gracefully and possibly omit source >> information for a stack frame instead of risking to stop writing the hs_err >> file when an assertion would have failed. To debug failures, `-Xlog:dwarf` >> can be used with `info`, `debug` or `trace` which provides logging messages >> throughout parsing. >> >> **Testing:** >> Apart from manual testing, I've added two kinds of tests: >> - A JTreg test: Spawns new VMs to let them crash in various ways. The test >> reads the created hs_err files to check if the DWARF parsing could correctly >> find the filename and line number. For normal HotSpot files, I could not >> check against hardcoded fil
Re: RFR: 8283723: Update Visual Studio 2022 to version 17.1.0 for Oracle builds on Windows
On Sun, 27 Mar 2022 05:08:52 GMT, Mikael Vidstedt wrote: > Oracle is updating the version of Visual Studio for building the JDK on > Windows to Visual Studio 2022 17.1.0. > > This change adds support for building devkits based on VS 2022. Instead of > creating a new script file for that I decided to combine it with the logic in > createWindowsDevkit2019.sh and rename the resulting file. I decided to > dropped support for VS 2017 since that version is pretty old now, let me know > if you'd prefer to see it kept around. > > Testing: tier1-5 + additional testing. make/devkit/createWindowsDevkit.sh line 74: > 72: # Work around the insanely named ProgramFiles(x86) env variable > 73: PROGRAMFILES_X86="$($WINDOWS_PATH_TO_UNIX_PATH "$(cmd.exe /c set | sed -n > 's/^ProgramFiles(x86)=//p' | tr -d '\r')")" > 74: PROGRAMFILES="$($WINDOWS_PATH_TO_UNIX_PATH "$(cmd.exe /c set | sed -n > 's/^PROGRAMFILES=//p' | tr -d '\r')")" This complex dance isn't necessary; it's a special operation for just the "ProgramFiles(x86)" variable, since cygwin can't handle environment variables with `()` in the name. (See the comment above as well) I think you will be able to access it with just a simple `$PROGRAMFILES`. - PR: https://git.openjdk.java.net/jdk/pull/7974
Re: Is --with-zlib=bundled broken on MacOS aarch64 12.2.1?
On 2022-03-28 09:03, David Holmes wrote: On 28/03/2022 4:56 pm, Alan Bateman wrote: On 28/03/2022 07:46, David Holmes wrote: Hi Jai, It isn't obvious to me that the bundled sources are actually intended to build on macOS. There's no include of unistd.h to get the lseek definition. I think the context here is that Jai is chasing an issue that may be bug in the libz on macOS. Building the bundled version and comparing results would lead to useful information. AFAIK, the system zlib has always been used since 7u4 when the macOS was added. I think the Apple JDK did the same. So there may be a small bit of "porting" to do, adds include files, etc. Okay, I suggest adding the include of unistd.h as a starter then. But Jai should note that this is not a build issue per-se - making those sources buildable on all desired platforms is the job of the owners of that src in the OpenJDK. I agree fully, but I'd also like to add that it is not clear that this is not "supposed" to work. If bundled zlib is not buildable on macOS (that was news to me), then the correct build system behavior should have been to block it in configure. So, Jaikiran, if you intend to get this to work on macOS, great! If you're not taking that route, please let me know so I can open a bug on prohibiting bundled zlib on maccOS from configure. And when getting stuff to compile on new platforms might still require some assistance and/or interaction with the build group, to get it right. But, the grunt work lies with the component team, yes. /Magnus Cheers, David -Alan
Re: RFR: 8242181: [Linux] Show source information when printing native stack traces in hs_err files [v5]
On Mon, 28 Mar 2022 09:13:39 GMT, Yasumasa Suenaga wrote: > As I said before, I think it would be nice to share DWARF parser between SA > and HotSpot. Can you expose these mechanisms? It may be another RFE, and may > need to think other platforms. That would be good to have. However, I'm not familiar with the SA code and how it works to share code with HotSpot. And I'm also not sure how much overlap the two parsers actually have. I quickly skimmed through the DWARF parsing code of the SA and it seems that it's main usage is for parsing call frame information (as described in section 6.4 of the DWARF 4 spec) which is not supported/needed in this patch. There is still some code that could be shared though like opening a DWARF file with its checks or reading an LEB 128 etc. Might be worth to investigate further if the two implementations can be merged/reused to some extent. But I propose to file a separate RFE for that. What do you think? - PR: https://git.openjdk.java.net/jdk/pull/7126
Re: RFR: 8242181: [Linux] Show source information when printing native stack traces in hs_err files [v5]
On Mon, 28 Feb 2022 16:22:25 GMT, Christian Hagedorn wrote: >> When printing the native stack trace on Linux (mostly done for hs_err >> files), it only prints the method with its parameters and a relative offset >> in the method: >> >> Stack: [0x7f6e01739000,0x7f6e0183a000], sp=0x7f6e01838110, >> free space=1020k >> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native >> code) >> V [libjvm.so+0x620d86] Compilation::~Compilation()+0x64 >> V [libjvm.so+0x624b92] Compiler::compile_method(ciEnv*, ciMethod*, int, >> bool, DirectiveSet*)+0xec >> V [libjvm.so+0x8303ef] >> CompileBroker::invoke_compiler_on_method(CompileTask*)+0x899 >> V [libjvm.so+0x82f067] CompileBroker::compiler_thread_loop()+0x3df >> V [libjvm.so+0x84f0d1] CompilerThread::thread_entry(JavaThread*, >> JavaThread*)+0x69 >> V [libjvm.so+0x1209329] JavaThread::thread_main_inner()+0x15d >> V [libjvm.so+0x12091c9] JavaThread::run()+0x167 >> V [libjvm.so+0x1206ada] Thread::call_run()+0x180 >> V [libjvm.so+0x1012e55] thread_native_entry(Thread*)+0x18f >> >> This makes it sometimes difficult to see where exactly the methods were >> called from and sometimes almost impossible when there are multiple >> invocations of the same method within one method. >> >> This patch improves this by providing source information (filename + line >> number) to the native stack traces on Linux similar to what's already done >> on Windows (see >> [JDK-8185712](https://bugs.openjdk.java.net/browse/JDK-8185712)): >> >> Stack: [0x7f34fca18000,0x7f34fcb19000], sp=0x7f34fcb17110, >> free space=1020k >> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native >> code) >> V [libjvm.so+0x620d86] Compilation::~Compilation()+0x64 >> (c1_Compilation.cpp:607) >> V [libjvm.so+0x624b92] Compiler::compile_method(ciEnv*, ciMethod*, int, >> bool, DirectiveSet*)+0xec (c1_Compiler.cpp:250) >> V [libjvm.so+0x8303ef] >> CompileBroker::invoke_compiler_on_method(CompileTask*)+0x899 >> (compileBroker.cpp:2291) >> V [libjvm.so+0x82f067] CompileBroker::compiler_thread_loop()+0x3df >> (compileBroker.cpp:1966) >> V [libjvm.so+0x84f0d1] CompilerThread::thread_entry(JavaThread*, >> JavaThread*)+0x69 (compilerThread.cpp:59) >> V [libjvm.so+0x1209329] JavaThread::thread_main_inner()+0x15d >> (thread.cpp:1297) >> V [libjvm.so+0x12091c9] JavaThread::run()+0x167 (thread.cpp:1280) >> V [libjvm.so+0x1206ada] Thread::call_run()+0x180 (thread.cpp:358) >> V [libjvm.so+0x1012e55] thread_native_entry(Thread*)+0x18f >> (os_linux.cpp:705) >> >> For Linux, we need to parse the debug symbols which are generated by GCC in >> DWARF - a standardized debugging format. This patch adds support for DWARF >> 4, the default of GCC 10.x, for 32 and 64 bit architectures (tested with >> x86_32, x86_64 and AArch64). DWARF 5 is not supported as it was still >> experimental and not generated for HotSpot. However, newer GCC version may >> soon generate DWARF 5 by default in which case this parser either needs to >> be extended or the build of HotSpot configured to only emit DWARF 4. >> >> The code follows the parsing steps described in the official DWARF 4 spec: >> https://dwarfstd.org/doc/DWARF4.pdf >> I added references to the corresponding sections throughout the code. >> However, I tried to explain the steps from the DWARF spec directly in the >> code (method names, comments etc.). This allows to follow the code without >> the need to actually deep dive into the spec. >> >> The comments at the `Dwarf` class in the `elf.hpp` file explain in more >> detail how a DWARF file is structured and how the parsing algorithm works to >> get to the filename and line number information. There are more class >> comments throughout the `elf.hpp` file about how different DWARF sections >> are structured and how the parsing algorithm needs to fetch the required >> information. Therefore, I will not repeat the exact workings of the >> algorithm here but refer to the code comments. I've tried to add as much >> information as possible to improve the readability. >> >> Generally, I've tried to stay away from adding any assertions as this code >> is almost always executed when already processing a VM error. Instead, the >> DWARF parser aims to just exit gracefully and possibly omit source >> information for a stack frame instead of risking to stop writing the hs_err >> file when an assertion would have failed. To debug failures, `-Xlog:dwarf` >> can be used with `info`, `debug` or `trace` which provides logging messages >> throughout parsing. >> >> **Testing:** >> Apart from manual testing, I've added two kinds of tests: >> - A JTreg test: Spawns new VMs to let them crash in various ways. The test >> reads the created hs_err files to check if the DWARF parsing could correctly >> find the filename and line number. For normal HotSpot files, I could not >> check against hardcoded fi
Re: RFR: 8242181: [Linux] Show source information when printing native stack traces in hs_err files [v5]
On Mon, 28 Mar 2022 08:44:10 GMT, Thomas Stuefe wrote: > > Ping - may I get another review for this change? > > I'll take a look later today or tomorrow. This is nice stuff, I'm looking > forward to having it upstream. That's great, thanks Thomas! - PR: https://git.openjdk.java.net/jdk/pull/7126
Re: RFR: 8242181: [Linux] Show source information when printing native stack traces in hs_err files [v5]
On Mon, 28 Mar 2022 06:46:30 GMT, Christian Hagedorn wrote: > Ping - may I get another review for this change? I'll take a look later today or tomorrow. This is nice stuff, I'm looking forward to having it upstream. - PR: https://git.openjdk.java.net/jdk/pull/7126
Re: Is --with-zlib=bundled broken on MacOS aarch64 12.2.1?
On 28/03/2022 4:56 pm, Alan Bateman wrote: On 28/03/2022 07:46, David Holmes wrote: Hi Jai, It isn't obvious to me that the bundled sources are actually intended to build on macOS. There's no include of unistd.h to get the lseek definition. I think the context here is that Jai is chasing an issue that may be bug in the libz on macOS. Building the bundled version and comparing results would lead to useful information. AFAIK, the system zlib has always been used since 7u4 when the macOS was added. I think the Apple JDK did the same. So there may be a small bit of "porting" to do, adds include files, etc. Okay, I suggest adding the include of unistd.h as a starter then. But Jai should note that this is not a build issue per-se - making those sources buildable on all desired platforms is the job of the owners of that src in the OpenJDK. Cheers, David -Alan
