Re: RFR: 8203290: [PPC64, s390] Check functionality of JDK-8199712 (Flight Recorder) [v18]
On Wed, 2 Feb 2022 17:17:10 GMT, Martin Doerr wrote: >> Tyler Steele has updated the pull request incrementally with one additional >> commit since the last revision: >> >> Edit thread_aix.cpp to match thread_linux.cpp in >> pd_get_top_fram_for_profiling and ...for_signal_handler > > Thanks for the update. As David had written, the Oracle Copyright lines you > have added are not correct: "Copyright (c) 2022, 2022". I suggest to avoid > adding new lines for this change until the topic is clarified. I received some guidance from our legal team. They suggested that I add the Oracle copyright header to files that only had the SAP header if I have made changes. They also suggested adding an IBM copyright line to files I have modified as well. @TheRealMDoerr, I know you also reached out to Iris. Have you heard back from them? I'm a fan of more testing @tstuefe. Let me know if you discover anything interesting. - PR: https://git.openjdk.java.net/jdk/pull/6885
Re: RFR: 8203290: [PPC64, s390] Check functionality of JDK-8199712 (Flight Recorder) [v20]
> Just in time for the holidays I have completed an implementation of the JFR > functionality for AIX. As a side note, this is my first submission to OpenJDK > 👋 > > ### Implementation notes and alternatives considered > > After modifying the build system to allow the --enable-jvm-feature-jfr to > work on AIX, my task was to implement the interfaces from os_perf.hpp. The > os_perf_aix.cpp implementation that existed was, I believe, a copy of the > Linux implementation. A review of the code in that file showed that > NetworkInterface, CPUPerformanceInterface, and SystemProcessInterface would > require modification to work on AIX. Using the Linux implementation as a > guide, I initially expected to use files from the aix procfs like > /proc//psinfo, and /proc//status in a manner similar to the Linux > implementation. However, I ended up using libperfstat for all functionality > required by the interfaces. > > ### Testing > > Testing for JFR seems to be quite light in the repo both before and after > this change. In addition to performing manual testing, I have added some > basic sanity checks that ensure events can be created and committed (using > jtreg), and performs some basic checks on the results of the interface member > functions (using gtest). > > ### More notes > > I've sent an email to the JFR group about a TOC overflow warning I > encountered while building (for the release server target). I believe the fix > is to pass -qpic=large when using the xlc toolchain, but my modifications to > flags-cflags.m4 have not been successful in removing this warning. Tyler Steele has updated the pull request with a new target base due to a merge or a rebase. The pull request now contains ten commits: - Merge branch 'master' into JDK-8203290 - Adds Oracle & IBM copyrights as per guidance from IBM legal team. - Removes incorrect Oracle copyright line from libperfstat_aix.cpp/hpp, and loadlib_aix.hpp - Edit thread_aix.cpp to match thread_linux.cpp in pd_get_top_fram_for_profiling and ...for_signal_handler - Addresses issues from review and other sm fixes - Adds commenting in regards to memory handling by SystemProcess & NetworkInterface classes - Replaces explicit initialization and copy of structs with memcpy and memset as appropriate - Renames internal struct definitions in os_perf_aix - Other minor fixes - Changes macoss -> macosx in problem list - Refactors loadlib_aix: Removes redundant c++ class - Merge branch 'master' into JDK-8203290 - Implements JFR on AIX - Implements interfaces from os_perf.hpp in os_perf_aix.cpp - Updates libperfstat_aix to contain functionality needed by os_perf_aix - Implements missing functionality in loadlib_aix (Fixes failure noted by TestNativeLibraies.java) - Removes platform checks for --enable-feature-jfr (Now enable-able on all platforms) - Enables TestNetworkUtilizationEvent.java which now passes on AIX - Updates AIX JavaThread::pd_get_top_frame_for_profiling with changes from Linux - Changes: https://git.openjdk.java.net/jdk/pull/6885/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=6885&range=19 Stats: 1235 lines in 10 files changed: 466 ins; 502 del; 267 mod Patch: https://git.openjdk.java.net/jdk/pull/6885.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6885/head:pull/6885 PR: https://git.openjdk.java.net/jdk/pull/6885
RFR: 8176706: Additional Date-Time Formats
Following the prior discussion [1], here is the PR for the subject enhancement. CSR has also been updated according to the suggestion. [1] https://mail.openjdk.java.net/pipermail/core-libs-dev/2022-January/085175.html - Commit messages: - Removed trailing space - Merge branch 'master' into skeleton - Tidying up - Some more tests - Brought back the part that was mistakenly removed - Merge branch 'master' into skeleton - Reverted IllegalArgumentException back - Further cleanups - Removed exceptions from ofLocalizedPattern() method, as they are lazily thrown on format() - Cleanup - ... and 19 more: https://git.openjdk.java.net/jdk/compare/cda9c301...f9311dce Changes: https://git.openjdk.java.net/jdk/pull/7340/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=7340&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8176706 Stats: 777 lines in 12 files changed: 733 ins; 9 del; 35 mod Patch: https://git.openjdk.java.net/jdk/pull/7340.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/7340/head:pull/7340 PR: https://git.openjdk.java.net/jdk/pull/7340
Re: [Ping2?] [8u] RFR: 8210283: Support git as an SCM alternative in the build
On Wed, 2021-12-22 at 11:14 +0100, Severin Gehwolf wrote: > On Fri, 2021-12-10 at 15:11 +0100, Severin Gehwolf wrote: > > Hi, > > > > Please review this adaptation of the corresponding JDK 11 patch. The > > JDK 11u patch didn't apply because the build system is widely different > > between these two releases. > > > > The main difference is make/common/MakeBase.gmk (JDK 8) vs > > make/SourceRevision.gmk (JDK 11). I've basically rewritten those parts > > of the patch. All the SCM handling in JDK 8 is in MakeBase. > > FindAllReposAbs isn't present in JDK 8u. > > > > Bug: https://bugs.openjdk.java.net/browse/JDK-8210283 > > webrev: https://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8210283/01/webrev/ > > > > Testing: "make --trace source-tips" on mercurial tree as well as > > the git mirror. $IMAGE_DIR/release file contains the SHA of > > the sources it was built from with 'git:' or 'hg:' prefixes. > > > > Thoughts? > > Anyone? When building from the git read-only mirror the "release" file > no longer includes the git sha it was built from without this fix. Anyone willing to review this? Thanks, Severin
Re: Segfault when building openjdk13 with openjdk12
On Thu, 2022-02-03 at 12:31 -0500, Abigail G wrote: > On Wed, 2022-02-02 at 12:12 +0300, Aleksey Shipilev wrote: > > On 2/2/22 08:53, Abigail G wrote: > > > Whoops, looks like I made the zip wrong, this one should work: > > > https://0x0.st/oHxy.zip > > > > So it looks like a GC crash: > > > > #Â SIGSEGV (0xb) at pc=0x7fa2ba719208, pid=29539, tid=29557 > > > > siginfo: si_signo: 11 (SIGSEGV), si_code: 1 (SEGV_MAPERR), si_addr: > > 0x0100 > > > > Current thread (0x7fa2b4091800):Â GCTaskThread "GC Thread#0" > > [stack: > > 0x7fa2b8c1,0x7fa2b8d1] [id=29557] > > > > > > If I was stuck with issue like this, I'll try the following: > > > > Â *) Test the memory. Memory problems frequently manifest as GC > > bugs: > > Â Â Â https://shipilev.net/jvm/test-your-memory/ > > Can confirm it's not memory, I was able to reproduce it on another > machine. > > > > > Â *) Use fastdebug JDK as the boot JDK, looking for a reasonable > > assert instead of anonymous crash. > > This probably requires building it first after configuring using -- > > with-debug-level=fastdebug. You > > said the build eventually succeeds, so a few tries might be in > > order. > > Will try this. > > > > > Â *) Bootstrap JDK 13 build with JDK 13. AFAICS, the actual > > requirement for boot JDK is 12..13. > > The problem with this is jdk13 doesn't exist yet in void, so we have > to > do a bootstrap chain up from 11 (the latest version packaged). > > > Regardless, even if you find a bug in JDK 12, it is unlikely to get > > fixed, as no one maintains JDK > > 12. Azul maintains JDK 13, so depending on your luck of identifying > > the issue with JDK 13, I'd say > > fixing JDK 13 might be possible. > > > > Thanks for the tips! > > Abigail G Ok, I build jdk12 with fastdebug and built jdk13 with that, still segfaults but maybe the core dump will have more info now: https://0x0.st/oHkf.zip Abigail G
Re: Segfault when building openjdk13 with openjdk12
On Wed, 2022-02-02 at 12:12 +0300, Aleksey Shipilev wrote: > On 2/2/22 08:53, Abigail G wrote: > > Whoops, looks like I made the zip wrong, this one should work: > > https://0x0.st/oHxy.zip > > So it looks like a GC crash: > > #Â SIGSEGV (0xb) at pc=0x7fa2ba719208, pid=29539, tid=29557 > > siginfo: si_signo: 11 (SIGSEGV), si_code: 1 (SEGV_MAPERR), si_addr: > 0x0100 > > Current thread (0x7fa2b4091800):Â GCTaskThread "GC Thread#0" > [stack: > 0x7fa2b8c1,0x7fa2b8d1] [id=29557] > > > If I was stuck with issue like this, I'll try the following: > > Â *) Test the memory. Memory problems frequently manifest as GC bugs: > Â Â Â https://shipilev.net/jvm/test-your-memory/ Can confirm it's not memory, I was able to reproduce it on another machine. > > Â *) Use fastdebug JDK as the boot JDK, looking for a reasonable > assert instead of anonymous crash. > This probably requires building it first after configuring using -- > with-debug-level=fastdebug. You > said the build eventually succeeds, so a few tries might be in order. Will try this. > > Â *) Bootstrap JDK 13 build with JDK 13. AFAICS, the actual > requirement for boot JDK is 12..13. The problem with this is jdk13 doesn't exist yet in void, so we have to do a bootstrap chain up from 11 (the latest version packaged). > Regardless, even if you find a bug in JDK 12, it is unlikely to get > fixed, as no one maintains JDK > 12. Azul maintains JDK 13, so depending on your luck of identifying > the issue with JDK 13, I'd say > fixing JDK 13 might be possible. > Thanks for the tips! Abigail G
Re: RFR: 8277204: Implementation of JEP 8264130: PAC-RET protection for Linux/AArch64 [v18]
> PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One > of its uses is to protect against ROP based attacks. This is done by > signing the Link Register whenever it is stored on the stack, and > authenticating the value when it is loaded back from the stack. If an > attacker were to try to change control flow by editing the stack then > the authentication check of the Link Register will fail, causing a > segfault when the function returns. > > On a system with PAC enabled, it is expected that all applications will > be compiled with ROP protection. Fedora 33 and upwards already provide > this. By compiling for ARMv8.0, GCC and LLVM will only use the set of > PAC instructions that exist in the NOP space - on hardware without PAC, > these instructions act as NOPs, allowing backward compatibility for > negligible performance cost (2 NOPs per non-leaf function). > > Hardware is currently limited to the Apple M1 MacBooks. All testing has > been done within a Fedora Docker image. A run of SpecJVM showed no > difference to that of noise - which was surprising. > > The most important part of this patch is simply compiling using branch > protection provided by GCC/LLVM. This protects all C++ code from being > used in ROP attacks, removing all static ROP gadgets from use. > > The remainder of the patch adds ROP protection to runtime generated > code, in both stubs and compiled Java code. Attacks here are much harder > as ROP gadgets must be found dynamically at runtime. If/when AOT > compilation is added to JDK, then all stubs and compiled Java will be > susceptible ROP gadgets being found by static analysis and therefore > potentially as vulnerable as C++ code. > > There are a number of places where the VM changes control flow by > rewriting the stack or otherwise. I’ve done some analysis as to how > these could also be used for attacks (which I didn’t want to post here). > These areas can be protected ensuring the pointers to various stubs and > entry points are stored in memory as signed pointers. These changes are > simple to make (they can be reduced to a type change in common code and > a few addition sign/auth calls in the backend), but there a lot of them > and the total code change is fairly large. I’m happy to provide a few > work in progress patches. > > In order to match the security benefits of the Apple Arm64e ABI across > the whole of JDK, then all the changes mentioned above would be > required. Alan Hayward has updated the pull request incrementally with one additional commit since the last revision: Documentation updates - Changes: - all: https://git.openjdk.java.net/jdk/pull/6334/files - new: https://git.openjdk.java.net/jdk/pull/6334/files/6255d4c8..d97883b5 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6334&range=17 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6334&range=16-17 Stats: 34 lines in 2 files changed: 33 ins; 0 del; 1 mod Patch: https://git.openjdk.java.net/jdk/pull/6334.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6334/head:pull/6334 PR: https://git.openjdk.java.net/jdk/pull/6334
Re: RFR: 8277204: Implementation of JEP 8264130: PAC-RET protection for Linux/AArch64 [v17]
On Wed, 2 Feb 2022 16:03:48 GMT, Alan Hayward wrote: >> PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One >> of its uses is to protect against ROP based attacks. This is done by >> signing the Link Register whenever it is stored on the stack, and >> authenticating the value when it is loaded back from the stack. If an >> attacker were to try to change control flow by editing the stack then >> the authentication check of the Link Register will fail, causing a >> segfault when the function returns. >> >> On a system with PAC enabled, it is expected that all applications will >> be compiled with ROP protection. Fedora 33 and upwards already provide >> this. By compiling for ARMv8.0, GCC and LLVM will only use the set of >> PAC instructions that exist in the NOP space - on hardware without PAC, >> these instructions act as NOPs, allowing backward compatibility for >> negligible performance cost (2 NOPs per non-leaf function). >> >> Hardware is currently limited to the Apple M1 MacBooks. All testing has >> been done within a Fedora Docker image. A run of SpecJVM showed no >> difference to that of noise - which was surprising. >> >> The most important part of this patch is simply compiling using branch >> protection provided by GCC/LLVM. This protects all C++ code from being >> used in ROP attacks, removing all static ROP gadgets from use. >> >> The remainder of the patch adds ROP protection to runtime generated >> code, in both stubs and compiled Java code. Attacks here are much harder >> as ROP gadgets must be found dynamically at runtime. If/when AOT >> compilation is added to JDK, then all stubs and compiled Java will be >> susceptible ROP gadgets being found by static analysis and therefore >> potentially as vulnerable as C++ code. >> >> There are a number of places where the VM changes control flow by >> rewriting the stack or otherwise. I’ve done some analysis as to how >> these could also be used for attacks (which I didn’t want to post here). >> These areas can be protected ensuring the pointers to various stubs and >> entry points are stored in memory as signed pointers. These changes are >> simple to make (they can be reduced to a type change in common code and >> a few addition sign/auth calls in the backend), but there a lot of them >> and the total code change is fairly large. I’m happy to provide a few >> work in progress patches. >> >> In order to match the security benefits of the Apple Arm64e ABI across >> the whole of JDK, then all the changes mentioned above would be >> required. > > Alan Hayward has updated the pull request incrementally with one additional > commit since the last revision: > > Update copyrights to 2022 As requested in the RFE, added the new flag to the man page. Also updated the building.md instructions. However, I'm not sure how to add to the release notes - I can't find any files or a process. - PR: https://git.openjdk.java.net/jdk/pull/6334
Re: RFR: 8277204: Implementation of JEP 8264130: PAC-RET protection for Linux/AArch64 [v17]
On Thu, 3 Feb 2022 12:11:16 GMT, Alan Hayward wrote: > As mentioned on the CSR, the JEP is being dropped - unless anyone has any > objections. JDK-8277204 will become a normal RFE. Good decision. - PR: https://git.openjdk.java.net/jdk/pull/6334
Re: RFR: 8274980: Improve adhoc build version strings [v4]
> Current adhoc version build strings are not ideal. Some of the problems: > * A build number of "0" is inserted, which make the version string look like > it's an official build, at least when not reading carefully > * The version string gives little indication on what source code the build > was based > > Also, an error was discovered in how the build system generates version > strings without a build number (since this basically never happened before). > A version without a build number, and without a PRE value, but with an OPT > value, should have a "+-" separator, according to JEP 223. While this was > correctly handled, the "+-" separator was also applied to versions without a > build, but with both PRE and OPT. In this case, the "+" should be omitted, > according to JEP 223. This bug is fixed by this commit. > > Ideally, the adhoc version string should include something along the lines of > the output of `git describe`. However, since the version string is set at > configure time, not build time, this will almost immediately become > misleading. :-( A substitute is proposed in this patch, where the branch name > is included in the OPT string (if it's not `master`). > > Finally, this patch fixes hotspot so it can properly build without a build > number. This was the last blocker for why the build system always required > the "0" as build number before. To facilitate interoperability with external > tools (like jib) that still sets build number to "0" to really signal "no > build number", a build number exactly matching "0" will be interpreted as > having no build number. Magnus Ihse Bursie has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains six additional commits since the last revision: - Merge branch 'master' into improve-adhoc-version-string - IS_EMPTY define did not work on msvc - Break long lines - Skip setting git branch in adhoc version opt - Fix for abtract_vm_version so it really handles empty VERSION_BUILD - 8274980: Improve adhoc build version strings - Changes: - all: https://git.openjdk.java.net/jdk/pull/6152/files - new: https://git.openjdk.java.net/jdk/pull/6152/files/3c93c740..0954dd55 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jdk&pr=6152&range=03 - incr: https://webrevs.openjdk.java.net/?repo=jdk&pr=6152&range=02-03 Stats: 999707 lines in 3982 files changed: 531148 ins; 444662 del; 23897 mod Patch: https://git.openjdk.java.net/jdk/pull/6152.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/6152/head:pull/6152 PR: https://git.openjdk.java.net/jdk/pull/6152
Re: RFR: 8277204: Implementation of JEP 8264130: PAC-RET protection for Linux/AArch64 [v17]
On Wed, 2 Feb 2022 16:03:48 GMT, Alan Hayward wrote: >> PAC is an optional feature in AArch64 8.3 and is compulsory in v9. One >> of its uses is to protect against ROP based attacks. This is done by >> signing the Link Register whenever it is stored on the stack, and >> authenticating the value when it is loaded back from the stack. If an >> attacker were to try to change control flow by editing the stack then >> the authentication check of the Link Register will fail, causing a >> segfault when the function returns. >> >> On a system with PAC enabled, it is expected that all applications will >> be compiled with ROP protection. Fedora 33 and upwards already provide >> this. By compiling for ARMv8.0, GCC and LLVM will only use the set of >> PAC instructions that exist in the NOP space - on hardware without PAC, >> these instructions act as NOPs, allowing backward compatibility for >> negligible performance cost (2 NOPs per non-leaf function). >> >> Hardware is currently limited to the Apple M1 MacBooks. All testing has >> been done within a Fedora Docker image. A run of SpecJVM showed no >> difference to that of noise - which was surprising. >> >> The most important part of this patch is simply compiling using branch >> protection provided by GCC/LLVM. This protects all C++ code from being >> used in ROP attacks, removing all static ROP gadgets from use. >> >> The remainder of the patch adds ROP protection to runtime generated >> code, in both stubs and compiled Java code. Attacks here are much harder >> as ROP gadgets must be found dynamically at runtime. If/when AOT >> compilation is added to JDK, then all stubs and compiled Java will be >> susceptible ROP gadgets being found by static analysis and therefore >> potentially as vulnerable as C++ code. >> >> There are a number of places where the VM changes control flow by >> rewriting the stack or otherwise. I’ve done some analysis as to how >> these could also be used for attacks (which I didn’t want to post here). >> These areas can be protected ensuring the pointers to various stubs and >> entry points are stored in memory as signed pointers. These changes are >> simple to make (they can be reduced to a type change in common code and >> a few addition sign/auth calls in the backend), but there a lot of them >> and the total code change is fairly large. I’m happy to provide a few >> work in progress patches. >> >> In order to match the security benefits of the Apple Arm64e ABI across >> the whole of JDK, then all the changes mentioned above would be >> required. > > Alan Hayward has updated the pull request incrementally with one additional > commit since the last revision: > > Update copyrights to 2022 As mentioned on the CSR, the JEP is being dropped - unless anyone has any objections. JDK-8277204 will become a normal RFE. - PR: https://git.openjdk.java.net/jdk/pull/6334
Re: RFR: 8203290: [PPC64, s390] Check functionality of JDK-8199712 (Flight Recorder) [v19]
On Thu, 3 Feb 2022 11:14:19 GMT, Martin Doerr wrote: > Looks good to me, too. I think it is ready for integration assuming all > change requests were taken care of and tests have passed. @TheRealMDoerr we should test the latest version of this patch in our nightlies, just in case - PR: https://git.openjdk.java.net/jdk/pull/6885
Re: RFR: 8203290: [PPC64, s390] Check functionality of JDK-8199712 (Flight Recorder) [v19]
On Wed, 2 Feb 2022 22:42:47 GMT, Tyler Steele wrote: >> Just in time for the holidays I have completed an implementation of the JFR >> functionality for AIX. As a side note, this is my first submission to >> OpenJDK 👋 >> >> ### Implementation notes and alternatives considered >> >> After modifying the build system to allow the --enable-jvm-feature-jfr to >> work on AIX, my task was to implement the interfaces from os_perf.hpp. The >> os_perf_aix.cpp implementation that existed was, I believe, a copy of the >> Linux implementation. A review of the code in that file showed that >> NetworkInterface, CPUPerformanceInterface, and SystemProcessInterface would >> require modification to work on AIX. Using the Linux implementation as a >> guide, I initially expected to use files from the aix procfs like >> /proc//psinfo, and /proc//status in a manner similar to the Linux >> implementation. However, I ended up using libperfstat for all functionality >> required by the interfaces. >> >> ### Testing >> >> Testing for JFR seems to be quite light in the repo both before and after >> this change. In addition to performing manual testing, I have added some >> basic sanity checks that ensure events can be created and committed (using >> jtreg), and performs some basic checks on the results of the interface >> member functions (using gtest). >> >> ### More notes >> >> I've sent an email to the JFR group about a TOC overflow warning I >> encountered while building (for the release server target). I believe the >> fix is to pass -qpic=large when using the xlc toolchain, but my >> modifications to flags-cflags.m4 have not been successful in removing this >> warning. > > Tyler Steele has updated the pull request incrementally with one additional > commit since the last revision: > > Removes incorrect Oracle copyright line from libperfstat_aix.cpp/hpp, and > loadlib_aix.hpp Looks good to me, too. I think it is ready for integration assuming all change requests were taken care of and tests have passed. - Marked as reviewed by mdoerr (Reviewer). PR: https://git.openjdk.java.net/jdk/pull/6885
