Reproducible MacOS Codesign'ed builds?

2022-05-23 Thread Andrew Leonard
Hi, Has anyone looked into reproducible builds for codesign'd MacOS builds? Basically Apple codesigning is non-deterministic, which is not surprisingly I guess, so naturally makes reproducible builds a bit tricky. The general theme for this sort of issue seems to be to remove the signature before

Integrated: 8284539: Configure --with-source-date=version fails on MacOS

2022-04-19 Thread Andrew Leonard
On Thu, 14 Apr 2022 16:13:59 GMT, Andrew Leonard wrote: > JDK-8282769 added support for more ISO-8601 formats, but remove handling of > just a date "-MM-DD" being present, which is the case for a configure > using --with-source-date=version which uses the date str

Re: RFR: 8284539: Configure --with-source-date=version fails on MacOS

2022-04-19 Thread Andrew Leonard
date string from >> version-numbers.conf. >> Also, the first date parse had an invalid format string "%FZ %TZ", with too >> many Zs. >> This PR corrects the first date parse to parse a standard ISO-8601 Zulu >> date: "%FT%TZ" >> Then it ad

Re: RFR: 8284539: Configure --with-source-date=version fails on MacOS

2022-04-19 Thread Andrew Leonard
On Thu, 14 Apr 2022 16:13:59 GMT, Andrew Leonard wrote: > JDK-8282769 added support for more ISO-8601 formats, but remove handling of > just a date "-MM-DD" being present, which is the case for a configure > using --with-source-date=version which uses the date str

Re: RFR: 8284539: Configure --with-source-date=version fails on MacOS

2022-04-14 Thread Andrew Leonard
On Thu, 14 Apr 2022 16:13:59 GMT, Andrew Leonard wrote: > JDK-8282769 added support for more ISO-8601 formats, but remove handling of > just a date "-MM-DD" being present, which is the case for a configure > using --with-source-date=version which uses the date str

Re: RFR: 8284661: Reproducible assembly builds without relative linking [v7]

2022-04-14 Thread Andrew Leonard
Compliation.gmk. > > Signed-off-by: Andrew Leonard Andrew Leonard has updated the pull request incrementally with one additional commit since the last revision: Trigger checks - Changes: - all: https://git.openjdk.java.net/jdk/pull/8177/files - new: https://git.openjdk.j

Re: RFR: 8284539: Configure --with-source-date=version fails on MacOS

2022-04-14 Thread Andrew Leonard
On Thu, 14 Apr 2022 16:13:59 GMT, Andrew Leonard wrote: > JDK-8282769 added support for more ISO-8601 formats, but remove handling of > just a date "-MM-DD" being present, which is the case for a configure > using --with-source-date=version which uses the date str

RFR: 8284539: Configure --with-source-date=version fails on MacOS

2022-04-14 Thread Andrew Leonard
mat string "%FZ %TZ", with too many Zs. This PR corrects the first date parse to parse a standard ISO-8601 Zulu date: "%FT%TZ" Then it adds the final check for no time being specified. Signed-off-by: Andrew Leonard - Commit messages: - 8284539: Configure --

Re: RFR: 8284661: Reproducible assembly builds without relative linking [v5]

2022-04-14 Thread Andrew Leonard
On Thu, 14 Apr 2022 11:13:20 GMT, Andrew Leonard wrote: >> make/data/autoheaders/assemblyprefix.h line 25: >> >>> 23: >>> 24: // ASSEMBLY_SRC_FILE gets replaced by relative or absolute file path >>> 25: // in NativeCompilation.gmk, this ensures

Re: RFR: 8284661: Reproducible assembly builds without relative linking [v5]

2022-04-14 Thread Andrew Leonard
On Wed, 13 Apr 2022 13:55:59 GMT, Andrew Leonard wrote: >> This PR removes the need for relative object file linking introduced by >> JDK-8284437 for linux libraries, by specifying >> .file directives in the linux .S source files. The >> source files specify a .file AS

Re: RFR: 8284661: Reproducible assembly builds without relative linking [v6]

2022-04-14 Thread Andrew Leonard
Compliation.gmk. > > Signed-off-by: Andrew Leonard Andrew Leonard has updated the pull request incrementally with one additional commit since the last revision: 8284661: Reproducible assembly builds without relative linking Signed-off-by: Andrew Leonard - Ch

Re: RFR: 8284661: Reproducible assembly builds without relative linking [v5]

2022-04-14 Thread Andrew Leonard
On Thu, 14 Apr 2022 10:13:32 GMT, Maxim Kartashev wrote: >> Andrew Leonard has updated the pull request incrementally with two >> additional commits since the last revision: >> >> - 8284661: Reproducible assembly builds without relative linking >> >&

Re: RFR: 8284661: Reproducible assembly builds without relative linking [v4]

2022-04-13 Thread Andrew Leonard
On Wed, 13 Apr 2022 11:37:36 GMT, Magnus Ihse Bursie wrote: > The latest version looks much better! Fix the test to check for linux, and > the indentation in the new header file, and I'd say you're good to go! @magicus update ready, with test fix, indentation and remove .file form the linux

Re: RFR: 8284661: Reproducible assembly builds without relative linking [v5]

2022-04-13 Thread Andrew Leonard
Compliation.gmk. > > Signed-off-by: Andrew Leonard Andrew Leonard has updated the pull request incrementally with two additional commits since the last revision: - 8284661: Reproducible assembly builds without relative linking Signed-off-by: Andrew Leonard - 8284661: Reproducible as

Re: RFR: 8284661: Reproducible assembly builds without relative linking [v4]

2022-04-13 Thread Andrew Leonard
On Wed, 13 Apr 2022 11:36:41 GMT, Magnus Ihse Bursie wrote: >> Andrew Leonard has updated the pull request incrementally with one >> additional commit since the last revision: >> >> 8284661: Reproducible assembly builds without relative linking >> >

Re: RFR: 8284661: Reproducible assembly builds without relative linking [v4]

2022-04-13 Thread Andrew Leonard
On Wed, 13 Apr 2022 11:41:14 GMT, Andrew Leonard wrote: >> make/data/autoheaders/assemblyprefix.h line 24: >> >>> 22: # >>> 23: >>> 24: // ASSEMBLY_SRC_FILE gets replaced by relative or absolute file >>> path >> >> While

Re: RFR: 8284661: Reproducible assembly builds without relative linking [v4]

2022-04-13 Thread Andrew Leonard
Compliation.gmk. > > Signed-off-by: Andrew Leonard Andrew Leonard has updated the pull request incrementally with one additional commit since the last revision: 8284661: Reproducible assembly builds without relative linking Signed-off-by: Andrew Leonard - Ch

Re: RFR: 8284661: Reproducible assembly builds without relative linking [v3]

2022-04-13 Thread Andrew Leonard
On Tue, 12 Apr 2022 10:27:30 GMT, Magnus Ihse Bursie wrote: > Actually, a better option might be to let GCC append the `.file` > automatically to all assembly files. I think this can be done by creating a > file like `make/data/autoheaders/assemblyprefix.h` with: > > ``` > //

Re: RFR: 8284661: Reproducible assembly builds without relative linking

2022-04-13 Thread Andrew Leonard
On Tue, 12 Apr 2022 08:39:54 GMT, Andrew Haley wrote: > > I excluded all kinds of debuginfo files because I didn't know if they could > > be made free of absolute paths, and it was out of scope for what I was > > doing at the time. > > GCC, I believe, uses whatever pathname you give to the

Re: RFR: 8284661: Reproducible assembly builds without relative linking [v3]

2022-04-11 Thread Andrew Leonard
On Mon, 11 Apr 2022 15:41:20 GMT, Andrew Leonard wrote: >> This PR removes the need for relative object file linking introduced by >> JDK-8284437 for linux libraries, by specifying >> .file directives in the linux .S source files. The >> source files specify a .file AS

Re: RFR: 8284661: Reproducible assembly builds without relative linking [v3]

2022-04-11 Thread Andrew Leonard
On Mon, 11 Apr 2022 21:03:22 GMT, Magnus Ihse Bursie wrote: >> Andrew Leonard has updated the pull request incrementally with one >> additional commit since the last revision: >> >> 8284661: Reproducible assembly builds without relative linking >> >

Re: RFR: 8284661: Reproducible assembly builds without relative linking [v3]

2022-04-11 Thread Andrew Leonard
On Mon, 11 Apr 2022 17:40:58 GMT, Erik Joelsson wrote: >> @erikj79 As far as I can see only AIX is the exception of not being Linux >> and using .debuginfo suffix, are there others? >> I am currently testing AIX, so I will see if it passes, as you say, I would >> have my doubts it will! > >

Re: RFR: 8284661: Reproducible assembly builds without relative linking

2022-04-11 Thread Andrew Leonard
On Mon, 11 Apr 2022 16:58:38 GMT, Erik Joelsson wrote: > > thanks @erikj79 Not familiar with AbsPathsInImage, could you give some > > background please? wondering why it's not found these paths before, maybe > > it's not running on debugimages? What tier does it get run in? > > I wrote this

Re: RFR: 8284661: Reproducible assembly builds without relative linking [v3]

2022-04-11 Thread Andrew Leonard
On Mon, 11 Apr 2022 16:53:41 GMT, Erik Joelsson wrote: >> Andrew Leonard has updated the pull request incrementally with one >> additional commit since the last revision: >> >> 8284661: Reproducible assembly builds without relative linking >> >> Sig

Re: RFR: 8284661: Reproducible assembly builds without relative linking [v3]

2022-04-11 Thread Andrew Leonard
On Mon, 11 Apr 2022 15:41:20 GMT, Andrew Leonard wrote: >> This PR removes the need for relative object file linking introduced by >> JDK-8284437 for linux libraries, by specifying >> .file directives in the linux .S source files. The >> source files specify a .file AS

Re: RFR: 8284661: Reproducible assembly builds without relative linking [v3]

2022-04-11 Thread Andrew Leonard
Compliation.gmk. > > Signed-off-by: Andrew Leonard Andrew Leonard has updated the pull request incrementally with one additional commit since the last revision: 8284661: Reproducible assembly builds without relative linking Signed-off-by: Andrew Leonard - Ch

Re: RFR: 8284661: Reproducible assembly builds without relative linking [v2]

2022-04-11 Thread Andrew Leonard
Compliation.gmk. > > Signed-off-by: Andrew Leonard Andrew Leonard has updated the pull request incrementally with one additional commit since the last revision: 8284661: Reproducible assembly builds without relative linking Signed-off-by: Andrew Leonard - Ch

Re: RFR: 8284661: Reproducible assembly builds without relative linking

2022-04-11 Thread Andrew Leonard
On Mon, 11 Apr 2022 14:50:37 GMT, Andrew Leonard wrote: >> Non-determinism in .debuginfo straight away makes .so libraries >> non-deterministic due to the embedded debuginfo CRC value. >> @erikj79 i'll try removing .debuginfo from the exceptions and try it... > >> @a

Re: RFR: 8284661: Reproducible assembly builds without relative linking

2022-04-11 Thread Andrew Leonard
On Mon, 11 Apr 2022 14:05:54 GMT, Andrew Leonard wrote: >> This PR removes the need for relative object file linking introduced by >> JDK-8284437 for linux libraries, by specifying >> .file directives in the linux .S source files. The >> source files specify a .file AS

Re: RFR: 8284661: Reproducible assembly builds without relative linking

2022-04-11 Thread Andrew Leonard
SEMBLY_SRC_FILE >> where ASSEMBLY_SRC_FILE is defined by the NativeCompliation.gmk. >> >> Signed-off-by: Andrew Leonard > > Maybe some short explanation should be added as comment in the src files? > > If this makes the debuginfo files free from absolute paths o

Re: RFR: 8284661: Reproducible assembly builds without relative linking

2022-04-11 Thread Andrew Leonard
On Mon, 11 Apr 2022 09:43:37 GMT, Andrew Leonard wrote: > This PR removes the need for relative object file linking introduced by > JDK-8284437 for linux libraries, by specifying > .file directives in the linux .S source files. The > source files specify a .file ASSEMBLY_SRC_

Re: RFR: 8284661: Reproducible assembly builds without relative linking

2022-04-11 Thread Andrew Leonard
On Mon, 11 Apr 2022 09:43:37 GMT, Andrew Leonard wrote: > This PR removes the need for relative object file linking introduced by > JDK-8284437 for linux libraries, by specifying > .file directives in the linux .S source files. The > source files specify a .file ASSEMBLY_SRC_

RFR: 8284661: Reproducible assembly builds without relative linking

2022-04-11 Thread Andrew Leonard
-by: Andrew Leonard - Commit messages: - 8284661: Reproducible assembly builds without relative linking Changes: https://git.openjdk.java.net/jdk/pull/8177/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk=8177=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8284661

Re: RFR: 8284437: Building from different users/workspace is not always deterministic [v2]

2022-04-08 Thread Andrew Leonard
On Fri, 8 Apr 2022 14:42:34 GMT, Andrew Leonard wrote: >> Andrew Leonard has updated the pull request incrementally with one >> additional commit since the last revision: >> >> Trigger checks > > I am not sure why without the explicit .file directive that the

Re: RFR: 8284437: Building from different users/workspace is not always deterministic [v2]

2022-04-08 Thread Andrew Leonard
On Wed, 6 Apr 2022 13:30:28 GMT, Andrew Leonard wrote: >> This PR fixes the non-deterministic behavior when building on linux with >> different userids or within >> different workspace folders. >> It fixes the following issues: >> - MakeZipReproducible.java used t

Re: RFR: 8284437: Building from different users/workspace is not always deterministic [v2]

2022-04-08 Thread Andrew Leonard
On Fri, 8 Apr 2022 14:18:50 GMT, Maxim Kartashev wrote: >> @magicus >>> Is it possible to pass the ".file" directive on the command line? >> >> I don't think so. Your other idea works quite well, though. If you have this >> in the source >> >>.file THIS_FILE >> >> and you specify

Re: RFR: 8284437: Building from different users/workspace is not always deterministic [v2]

2022-04-08 Thread Andrew Leonard
On Fri, 8 Apr 2022 13:11:32 GMT, Maxim Kartashev wrote: >>> While we're at it, let me repeat my question from the alias: >>> >>> I was wondering why were the changes in `make/autoconf/flags-cflags.m4` >>> made under: >>> >>> ``` >>> if test "x$TOOLCHAIN_TYPE" = xgcc; then ... >>> ``` >>> >>>

Re: RFR: 8284437: Building from different users/workspace is not always deterministic [v2]

2022-04-08 Thread Andrew Leonard
On Fri, 8 Apr 2022 12:43:46 GMT, Maxim Kartashev wrote: > While we're at it, let me repeat my question from the alias: > > I was wondering why were the changes in `make/autoconf/flags-cflags.m4` made > under: > > ``` > if test "x$TOOLCHAIN_TYPE" = xgcc; then ... > ``` > > but not also

Re: RFR: 8284437: Building from different users/workspace is not always deterministic [v2]

2022-04-08 Thread Andrew Leonard
On Fri, 8 Apr 2022 12:24:38 GMT, Maxim Kartashev wrote: > FWIW, I (locally) solved the problem of absolute path names in the compiled > assembly by adding the `.file` directive. For example: > > ``` > --- a/src/hotspot/os_cpu/linux_aarch64/atomic_linux_aarch64.S > +++

Re: RFR: 8284437: Building from different users/workspace is not always deterministic [v2]

2022-04-08 Thread Andrew Leonard
On Wed, 6 Apr 2022 13:30:28 GMT, Andrew Leonard wrote: >> This PR fixes the non-deterministic behavior when building on linux with >> different userids or within >> different workspace folders. >> It fixes the following issues: >> - MakeZipReproducible.java used t

Re: RFR: 8284437: Building from different users/workspace is not always deterministic [v2]

2022-04-08 Thread Andrew Leonard
On Wed, 6 Apr 2022 13:30:28 GMT, Andrew Leonard wrote: >> This PR fixes the non-deterministic behavior when building on linux with >> different userids or within >> different workspace folders. >> It fixes the following issues: >> - MakeZipReproducible.java used t

Re: RFR: 8284437: Building from different users/workspace is not always deterministic [v2]

2022-04-08 Thread Andrew Leonard
On Fri, 8 Apr 2022 09:30:51 GMT, Andrew Leonard wrote: >> Actually, I think that the GNU assembler supports debug prefix mapping. See >> e.g. [this bug report](https://github.com/ocaml/ocaml/issues/10770), >> stating: "With GNU tools, to enable debug prefix map on C sou

Re: RFR: 8284437: Building from different users/workspace is not always deterministic [v2]

2022-04-08 Thread Andrew Leonard
On Fri, 8 Apr 2022 08:31:45 GMT, Magnus Ihse Bursie wrote: >> One way would be to check if we have either assembly source files, or >> anything in EXTRA_OBJ, and if so, do the relative linking. That would at >> least bring down collateral damage significantly, since the majority of libs >>

Re: RFR: 8284437: Building from different users/workspace is not always deterministic [v2]

2022-04-07 Thread Andrew Leonard
On Thu, 7 Apr 2022 15:51:30 GMT, Magnus Ihse Bursie wrote: >> Andrew Leonard has updated the pull request incrementally with one >> additional commit since the last revision: >> >> Trigger checks > > make/common/NativeCompilation.gmk line 1159: > >&

Re: RFR: 8284437: Building from different users/workspace is not always deterministic [v2]

2022-04-07 Thread Andrew Leonard
On Thu, 7 Apr 2022 16:33:44 GMT, Erik Joelsson wrote: >> make/common/NativeCompilation.gmk line 1159: >> >>> 1157: ifeq ($(call isTargetOs, linux), true) >>> 1158: ifeq ($$($1_COMPILE_WITH_DEBUG_SYMBOLS), true) >>> 1159: $1_LINK_OBJS_RELATIVE := true >> >> I realize

Integrated: 8284437: Building from different users/workspace is not always deterministic

2022-04-06 Thread Andrew Leonard
On Wed, 6 Apr 2022 10:27:40 GMT, Andrew Leonard wrote: > This PR fixes the non-deterministic behavior when building on linux with > different userids or within > different workspace folders. > It fixes the following issues: > - MakeZipReproducible.java used to produce repro

Re: RFR: 8284437: Building from different users/workspace is not always deterministic [v2]

2022-04-06 Thread Andrew Leonard
d using -frandom-seed. > - For reproducible builds when producing debug symbols use relative object > paths for library linking to remove absolute MASM object paths. > > Signed-off-by: Andrew Leonard Andrew Leonard has updated the pull request incrementally with one additi

RFR: 8284437: Building from different users/workspace is not always deterministic

2022-04-06 Thread Andrew Leonard
cing debug symbols use relative object paths for library linking to remove absolute MASM object paths. Signed-off-by: Andrew Leonard - Commit messages: - 8284437: Building from different users/workspace is not always deterministic - 8284437: Building from different users

Integrated: 8283315: jrt-fs.jar not always deterministically built

2022-03-18 Thread Andrew Leonard
On Thu, 17 Mar 2022 11:09:24 GMT, Andrew Leonard wrote: > JarArchive.gmk uses an un-sorted jar @, thus depending on exactly > how that list gets ordered by relevant OS querys. Such queries can differ in > order on different CPU architectures (Intel vs AMD). > > This

Re: RFR: 8283315: jrt-fs.jar not always deterministically built [v5]

2022-03-18 Thread Andrew Leonard
> JarArchive.gmk uses an un-sorted jar @, thus depending on exactly > how that list gets ordered by relevant OS querys. Such queries can differ in > order on different CPU architectures (Intel vs AMD). > > This PR adds a "sort" to the jar @ contents. > > Signed-

Re: RFR: 8283315: jrt-fs.jar not always deterministically built [v4]

2022-03-17 Thread Andrew Leonard
> JarArchive.gmk uses an un-sorted jar @, thus depending on exactly > how that list gets ordered by relevant OS querys. Such queries can differ in > order on different CPU architectures (Intel vs AMD). > > This PR adds a "sort" to the jar @ contents. > > Signed-

Re: RFR: 8283315: jrt-fs.jar not always deterministically built [v3]

2022-03-17 Thread Andrew Leonard
> JarArchive.gmk uses an un-sorted jar @, thus depending on exactly > how that list gets ordered by relevant OS querys. Such queries can differ in > order on different CPU architectures (Intel vs AMD). > > This PR adds a "sort" to the jar @ contents. > > Signed-

Re: RFR: 8283315: jrt-fs.jar not always deterministically built [v2]

2022-03-17 Thread Andrew Leonard
> JarArchive.gmk uses an un-sorted jar @, thus depending on exactly > how that list gets ordered by relevant OS querys. Such queries can differ in > order on different CPU architectures (Intel vs AMD). > > This PR adds a "sort" to the jar @ contents. > > Signed-

Re: RFR: 8283315: jrt-fs.jar not always deterministically built [v2]

2022-03-17 Thread Andrew Leonard
On Thu, 17 Mar 2022 11:28:55 GMT, Andrew Leonard wrote: >> make/common/JarArchive.gmk line 196: >> >>> 194: if [ "`$(WC) -l $$($1_BIN)/_the.$$($1_JARNAME)_contents | $(AWK) >>> '{ print 1 }'`" -gt "0" ]; then \ >>> 195:

Re: RFR: 8283315: jrt-fs.jar not always deterministically built

2022-03-17 Thread Andrew Leonard
>> This PR adds a "sort" to the jar @ contents. >> >> Signed-off-by: Andrew Leonard > > make/common/JarArchive.gmk line 196: > >> 194: if [ "`$(WC) -l $$($1_BIN)/_the.$$($1_JARNAME)_contents | $(AWK) >> '{ print 1 }'`" -

RFR: 8283315: jrt-fs.jar not always deterministically built

2022-03-17 Thread Andrew Leonard
JarArchive.gmk uses an un-sorted jar @, thus depending on exactly how that list gets ordered by relevant OS querys. Such queries can differ in order on different CPU architectures (Intel vs AMD). This PR adds a "sort" to the jar @ contents. Signed-off-by: Andrew Leonard ---

Integrated: 8279834: Alpine Linux fails to build when --with-source-date enabled

2022-01-11 Thread Andrew Leonard
On Tue, 11 Jan 2022 10:25:31 GMT, Andrew Leonard wrote: > The new reproducible build jar & jmod feature exposed a problem on Alpline > linux whereby when --with-source-date is specified the IS_GNU_DATE setting > was actually wrong. It was assuming the BusyBox "date&quo

Re: RFR: 8279834: Alpine Linux fails to build when --with-source-date enabled

2022-01-11 Thread Andrew Leonard
;date" tool on Alpine linux >> was BSD compatible, but it is actually GNU syntax compatible. >> >> This PR additionally checks for BusyBox when determining the date tool >> version compatibility. >> >> Signed-off-by: Andrew Leonard > > Marked as r

RFR: 8279834: Alpine Linux fails to build when --with-source-date enabled

2022-01-11 Thread Andrew Leonard
tax compatible. This PR additionally checks for BusyBox when determining the date tool version compatibility. Signed-off-by: Andrew Leonard - Commit messages: - 8279834: Alpine Linux fails to build when --with-source-date enabled Changes: https://git.openjdk.java.net/jdk/pull/7025/files

Integrated: 8279182: MakeZipReproducible ZipEntry timestamps not localized to UTC

2021-12-23 Thread Andrew Leonard
On Thu, 23 Dec 2021 09:39:19 GMT, Andrew Leonard wrote: > MakeZipReproducible was added to enable reproducible building of src.zip. > However, as ZipEntry timestamps are a "localized" date with no zone, the > specified epoch instant was getting localized in whatever the b

Integrated: 8278766: Enable OpenJDK build support for reproducible jars and jmods using --date

2021-12-23 Thread Andrew Leonard
On Fri, 17 Dec 2021 09:18:03 GMT, Andrew Leonard wrote: > If "reproducible build" is enabled, then utilize the new --date option in the > building of jar's and jmods. > Validating the boot jdk supports --date for the building of the jars. > > Signed-off-by: Andrew Le

RFR: 8279182: MakeZipReproducible ZipEntry timestamps not localized to UTC

2021-12-23 Thread Andrew Leonard
rent zones would differ. The timestamp should be localized to UTC (like for jar, jmod entries), this PR ensures this. Signed-off-by: Andrew Leonard - Commit messages: - 8279182: MakeZipReproducible ZipEntry timestamps not localized to UTC Changes: https://git.openjdk.java.net/jdk

Re: RFR: 8278766: Enable OpenJDK build support for reproducible jars and jmods using --date [v5]

2021-12-22 Thread Andrew Leonard
> If "reproducible build" is enabled, then utilize the new --date option in the > building of jar's and jmods. > Validating the boot jdk supports --date for the building of the jars. > > Signed-off-by: Andrew Leonard Andrew Leonard has updated the pull request incremen

Re: RFR: 8278766: Enable OpenJDK build support for reproducible jars and jmods using --date [v4]

2021-12-22 Thread Andrew Leonard
On Wed, 22 Dec 2021 17:30:36 GMT, Erik Joelsson wrote: >> Andrew Leonard has updated the pull request incrementally with three >> additional commits since the last revision: >> >> - 8278766: Enable OpenJDK build support for reproducible jars and jmods >> using

Re: RFR: 8278766: Enable OpenJDK build support for reproducible jars and jmods using --date [v4]

2021-12-22 Thread Andrew Leonard
On Fri, 17 Dec 2021 20:04:20 GMT, Andrew Leonard wrote: >> make/common/MakeBase.gmk line 148: >> >>> 146: ifeq ($(SOURCE_DATE_ISO_8601), ) >>> 147: # GNU date format did not work, try BSD date options >>> 148: SOURCE_DATE_I

Re: RFR: 8278766: Enable OpenJDK build support for reproducible jars and jmods using --date [v3]

2021-12-22 Thread Andrew Leonard
On Wed, 22 Dec 2021 15:28:52 GMT, Andrew Leonard wrote: >> make/InitSupport.gmk line 317: >> >>> 315: export SOURCE_DATE_EPOCH := $$(SOURCE_DATE) >>> 316: ifeq ($$(IS_GNU_DATE), yes) >>> 317: SOURCE_DATE_ISO_8601 := $(shell $(DATE) --utc >

Re: RFR: 8278766: Enable OpenJDK build support for reproducible jars and jmods using --date [v4]

2021-12-22 Thread Andrew Leonard
On Fri, 17 Dec 2021 17:47:02 GMT, Erik Joelsson wrote: >> Andrew Leonard has updated the pull request incrementally with three >> additional commits since the last revision: >> >> - 8278766: Enable OpenJDK build support for reproducible jars and jmods >> using

Re: RFR: 8278766: Enable OpenJDK build support for reproducible jars and jmods using --date [v4]

2021-12-22 Thread Andrew Leonard
> If "reproducible build" is enabled, then utilize the new --date option in the > building of jar's and jmods. > Validating the boot jdk supports --date for the building of the jars. > > Signed-off-by: Andrew Leonard Andrew Leonard has updated the pull request

Re: RFR: 8278766: Enable OpenJDK build support for reproducible jars and jmods using --date [v4]

2021-12-22 Thread Andrew Leonard
On Fri, 17 Dec 2021 20:08:43 GMT, Andrew Leonard wrote: >> But I think the code in InitSupport will be executed always; Init.gmk is our >> "bootstrapper" / "trampoline" which wraps all calls to make (and >> InitSupport.gmk contains gory implementation de

Re: RFR: 8278766: Enable OpenJDK build support for reproducible jars and jmods using --date [v3]

2021-12-22 Thread Andrew Leonard
On Wed, 22 Dec 2021 13:41:53 GMT, Erik Joelsson wrote: >> Andrew Leonard has updated the pull request incrementally with one >> additional commit since the last revision: >> >> 8278766: Enable OpenJDK build support for reproducible jars and jmods >> usin

Re: RFR: 8278766: Enable OpenJDK build support for reproducible jars and jmods using --date [v3]

2021-12-22 Thread Andrew Leonard
On Wed, 22 Dec 2021 13:41:53 GMT, Erik Joelsson wrote: >> Andrew Leonard has updated the pull request incrementally with one >> additional commit since the last revision: >> >> 8278766: Enable OpenJDK build support for reproducible jars and jmods >> usin

Re: RFR: 8278766: Enable OpenJDK build support for reproducible jars and jmods using --date [v3]

2021-12-22 Thread Andrew Leonard
On Wed, 22 Dec 2021 13:50:32 GMT, Andrew Leonard wrote: >> make/InitSupport.gmk line 317: >> >>> 315: export SOURCE_DATE_EPOCH := $$(SOURCE_DATE) >>> 316: ifeq ($$(IS_GNU_DATE), yes) >>> 317: SOURCE_DATE_ISO_8601 := $(shell $(DATE) --utc >

Re: RFR: 8278766: Enable OpenJDK build support for reproducible jars and jmods using --date [v3]

2021-12-22 Thread Andrew Leonard
On Wed, 22 Dec 2021 13:41:53 GMT, Erik Joelsson wrote: >> Andrew Leonard has updated the pull request incrementally with one >> additional commit since the last revision: >> >> 8278766: Enable OpenJDK build support for reproducible jars and jmods >> usin

Re: RFR: 8278766: Enable OpenJDK build support for reproducible jars and jmods using --date [v3]

2021-12-22 Thread Andrew Leonard
> If "reproducible build" is enabled, then utilize the new --date option in the > building of jar's and jmods. > Validating the boot jdk supports --date for the building of the jars. > > Signed-off-by: Andrew Leonard Andrew Leonard has updated the pull request incremen

Re: RFR: 8278766: Enable OpenJDK build support for reproducible jars and jmods using --date [v2]

2021-12-22 Thread Andrew Leonard
> If "reproducible build" is enabled, then utilize the new --date option in the > building of jar's and jmods. > Validating the boot jdk supports --date for the building of the jars. > > Signed-off-by: Andrew Leonard Andrew Leonard has updated the pull request

Re: RFR: 8278766: Enable OpenJDK build support for reproducible jars and jmods using --date

2021-12-17 Thread Andrew Leonard
On Fri, 17 Dec 2021 16:29:21 GMT, Magnus Ihse Bursie wrote: >> Oh, that's ... interesting. (I'm pretty sure I wrote that code myself :)) >> >> I still think it would be good to keep the new code close to the old. If we >> set SOURCE_DATE to "updated", I think that should reflect in >>

Re: RFR: 8278766: Enable OpenJDK build support for reproducible jars and jmods using --date

2021-12-17 Thread Andrew Leonard
On Fri, 17 Dec 2021 17:53:12 GMT, Erik Joelsson wrote: >> But I think the code in InitSupport will be executed always; Init.gmk is our >> "bootstrapper" / "trampoline" which wraps all calls to make (and >> InitSupport.gmk contains gory implementation details of Init.gmk). > > SOURCE_DATE_EPOCH

Re: RFR: 8278766: Enable OpenJDK build support for reproducible jars and jmods using --date

2021-12-17 Thread Andrew Leonard
On Fri, 17 Dec 2021 17:54:46 GMT, Erik Joelsson wrote: >> If "reproducible build" is enabled, then utilize the new --date option in >> the building of jar's and jmods. >> Validating the boot jdk supports --date for the building of the jars. >> >> Signed

Re: RFR: 8278766: Enable OpenJDK build support for reproducible jars and jmods using --date

2021-12-17 Thread Andrew Leonard
On Fri, 17 Dec 2021 11:18:10 GMT, Magnus Ihse Bursie wrote: >> If "reproducible build" is enabled, then utilize the new --date option in >> the building of jar's and jmods. >> Validating the boot jdk supports --date for the building of the jars. >> >>

RFR: 8278766: Enable OpenJDK build support for reproducible jars and jmods using --date

2021-12-17 Thread Andrew Leonard
If "reproducible build" is enabled, then utilize the new --date option in the building of jar's and jmods. Validating the boot jdk supports --date for the building of the jars. Signed-off-by: Andrew Leonard - Commit messages: - 8278766: Enable OpenJDK bui

Integrated: 8278163: --with-cacerts-src variable resolved after GenerateCacerts recipe setup

2021-12-03 Thread Andrew Leonard
On Thu, 2 Dec 2021 21:07:30 GMT, Andrew Leonard wrote: > PR https://github.com/openjdk/jdk/pull/6647 resolved the GENDATA_CACERTS_SRC > fir --with-cacerts-src after the recipe, which meant the dependencies were > wrong. > This PR moves it before the recipe. > > Signed-off-

RFR: 8278163: --with-cacerts-src variable resolved after GenerateCacerts recipe setup

2021-12-02 Thread Andrew Leonard
PR https://github.com/openjdk/jdk/pull/6647 resolved the GENDATA_CACERTS_SRC fir --with-cacerts-src after the recipe, which meant the dependencies were wrong. This PR moves it before the recipe. Signed-off-by: Andrew Leonard - Commit messages: - 8278163: --with-cacerts-src

Re: RFR: 8278080: Add --with-cacerts-src='user cacerts folder' to enable deterministic cacerts generation [v2]

2021-12-02 Thread Andrew Leonard
On Thu, 2 Dec 2021 18:46:09 GMT, Erik Joelsson wrote: >> this was my understanding: >> https://www.gnu.org/software/make/manual/html_node/Variables-in-Recipes.html >> >> This occurs after make has finished reading all the makefiles and the target >> is determined to be out of date; so, the

Re: RFR: 8278080: Add --with-cacerts-src='user cacerts folder' to enable deterministic cacerts generation [v2]

2021-12-02 Thread Andrew Leonard
On Thu, 2 Dec 2021 17:48:04 GMT, Andrew Leonard wrote: >> you make a valid point, but i've tested this numerous times, but let me >> check again > > my assumption was the recipe gets resolved later this was my understanding: https://www.gnu.org/software/make/manual/h

Re: RFR: 8278080: Add --with-cacerts-src='user cacerts folder' to enable deterministic cacerts generation [v2]

2021-12-02 Thread Andrew Leonard
On Thu, 2 Dec 2021 17:46:35 GMT, Andrew Leonard wrote: >> I would have expected to see something like: >> >> ifneq ($(CACERTS_SRC), ) >> GENDATA_CACERTS_SRC := $(CACERTS_SRC) >> else >> GENDATA_CACERTS_SRC := $(TOPDIR)/make/data/cacerts/ >> end

Re: RFR: 8278080: Add --with-cacerts-src='user cacerts folder' to enable deterministic cacerts generation [v2]

2021-12-02 Thread Andrew Leonard
On Thu, 2 Dec 2021 17:35:36 GMT, Magnus Ihse Bursie wrote: >> make/modules/java.base/Gendata.gmk line 76: >> >>> 74: ifneq ($(CACERTS_SRC), ) >>> 75: GENDATA_CACERTS_SRC := $(CACERTS_SRC) >>> 76: endif >> >> Does this even work?! You are reassigning the variable after it has been >> used.

Integrated: 8278080: Add --with-cacerts-src='user cacerts folder' to enable deterministic cacerts generation

2021-12-02 Thread Andrew Leonard
On Wed, 1 Dec 2021 18:30:06 GMT, Andrew Leonard wrote: > Addition of a configure option --with-cacerts-src='user cacerts folder' to > allow developers to specify their own cacerts PEM folder for generation of > the cacerts store using the deterministic openjdk GenerateCacerts tool. &g

Re: RFR: 8278080: Add --with-cacerts-src='user cacerts folder' to enable deterministic cacerts generation [v2]

2021-12-02 Thread Andrew Leonard
On Thu, 2 Dec 2021 14:29:00 GMT, Sean Mullan wrote: > I don’t have any major concerns with this change, as long as the default > cacerts are still the ones that are in the JDK. As an aside, using Mozilla's > root certificates might be fine for TLS certificates, but if you need to > support

Re: RFR: 8278080: Add --with-cacerts-src='user cacerts folder' to enable deterministic cacerts generation [v2]

2021-12-02 Thread Andrew Leonard
On Wed, 1 Dec 2021 18:47:41 GMT, Erik Joelsson wrote: >> Andrew Leonard has updated the pull request with a new target base due to a >> merge or a rebase. The incremental webrev excludes the unrelated changes >> brought in by the merge/rebase. The pull request contai

Re: RFR: 8278080: Add --with-cacerts-src='user cacerts folder' to enable deterministic cacerts generation [v2]

2021-12-02 Thread Andrew Leonard
> Addition of a configure option --with-cacerts-src='user cacerts folder' to > allow developers to specify their own cacerts PEM folder for generation of > the cacerts store using the deterministic openjdk GenerateCacerts tool. > > Signed-off-by: Andrew Leonard Andrew Leon

Re: RFR: 8278080: Add --with-cacerts-src='user cacerts folder' to enable deterministic cacerts generation

2021-12-02 Thread Andrew Leonard
On Thu, 2 Dec 2021 00:09:31 GMT, Sergey Bylokhov wrote: > I have a question related to the custom cacerts which can be added to the > OpenJDK bundle. How do you pass the tests like > test/jdk/sun/security/lib/cacerts/VerifyCACerts.java using that custom jdk > bundle? Probably we can add an

RFR: 8278080: Add --with-cacerts-src='user cacerts folder' to enable determinsitic cacerts generation

2021-12-01 Thread Andrew Leonard
Addition of a configure option --with-cacerts-src='user cacerts folder' to allow developers to specify their own cacerts PEM folder for generation of the cacerts store using the deterministic openjdk GenerateCacerts tool. Signed-off-by: Andrew Leonard - Commit messages

Integrated: 8277762: Allow configuration of HOTSPOT_BUILD_USER

2021-12-01 Thread Andrew Leonard
On Wed, 24 Nov 2021 16:49:31 GMT, Andrew Leonard wrote: > To better allow "reproducible builds", a new configure parameter is added to > set the USERNAME build variable, rather than always using the current user: > --with-build-user= > HOTSPOT_BUILD_USER is then

Re: RFR: 8277762: Allow configuration of HOTSPOT_BUILD_USER [v2]

2021-12-01 Thread Andrew Leonard
On Wed, 1 Dec 2021 15:22:09 GMT, Andrew Leonard wrote: >> To better allow "reproducible builds", a new configure parameter is added to >> set the USERNAME build variable, rather than always using the current user: >> --with-build-user= >> HOTSPOT_BUILD_

Re: RFR: 8277762: Allow configuration of HOTSPOT_BUILD_USER [v2]

2021-12-01 Thread Andrew Leonard
> To better allow "reproducible builds", a new configure parameter is added to > set the USERNAME build variable, rather than always using the current user: > --with-build-user= > HOTSPOT_BUILD_USER is then set to a reproducible USERNAME if required. > > S

Re: RFR: 8277762: Allow configuration of HOTSPOT_BUILD_USER [v2]

2021-12-01 Thread Andrew Leonard
On Wed, 24 Nov 2021 18:59:03 GMT, Erik Joelsson wrote: >> it's also used in make/hotspot/lib/CompileJvm.gmk, but agree moving to >> jdk-versions.m4 makes sense > > Right, I meant the only place it's used within configure. done, moved - PR:

Re: RFR: 8277762: Allow configuration of HOTSPOT_BUILD_USER

2021-11-24 Thread Andrew Leonard
USER is then set to a reproducible USERNAME if required. >> >> Signed-off-by: Andrew Leonard > > make/autoconf/basic.m4 line 99: > >> 97: >> 98: # Setup username (for use in adhoc version strings etc) >> 99: AC_ARG_WITH([build-user], [AS_HELP_STRING([

RFR: 8277762: Allow configuration of HOTSPOT_BUILD_USER

2021-11-24 Thread Andrew Leonard
To better allow "reproducible builds", a new configure parameter is added to set the USERNAME build variable, rather than always using the current user: --with-build-user= HOTSPOT_BUILD_USER is then set to a reproducible USERNAME if required. Signed-off-by: Andr

Integrated: 8276743: Make openjdk build Zip Archive generation "reproducible"

2021-11-12 Thread Andrew Leonard
On Tue, 9 Nov 2021 12:59:17 GMT, Andrew Leonard wrote: > This PR adds a new openjdk build tool MakeZipReproducible, which if > ENABLE_REPRODUCIBLE_BUILD is enabled, generates a final "zip" file in a > deterministic way, ensuring ordering and timestamps are set as sp

Re: RFR: 8276743: Make openjdk build Zip Archive generation "reproducible" [v5]

2021-11-12 Thread Andrew Leonard
On Fri, 12 Nov 2021 14:34:50 GMT, Erik Joelsson wrote: >> Sorry, I'm mis-reading the lines here. It's of course for docs-zip. Then >> it's perfectly in order! :-) > > It's using JarArchive.gmk, which is a similar, but different and probably > also needs the same treatment. We should probably

  1   2   >