Re: RFR: 8264449: Enable reproducible builds with SOURCE_DATE_EPOCH [v3]
On Fri, 16 Apr 2021 19:05:34 GMT, Kevin Rushforth wrote: >> John Neffenger has updated the pull request incrementally with one >> additional commit since the last revision: >> >> Include WebKit shared library for Windows >> >> Enable reproducible builds of the native WebKit shared library for >> Windows (jfxwebkit.dll) when SOURCE_DATE_EPOCH is defined. > > Note that this PR is dependent on PR #422 . > > @jgneff As an alternative, you could close that PR (since all of the changes > from that one are currently included here), and list both issues in this PR > with Skara's `/issue add ...` command. It's up to you. @kevinrushforth I'll take the opportunity provided by this *merge-conflict* to simplify as you suggested. I'll close pull request #422, list both issues in this pull request, add Bernhard as a contributor, and add a commit that makes the `all` task reproducible. That will consolidate what would have been three pull requests into one and will leave only the build path problem for a follow-up request (fix number 4 in my first comment). - PR: https://git.openjdk.java.net/jfx/pull/446
Re: RFR: 8264449: Enable reproducible builds with SOURCE_DATE_EPOCH [v3]
On Sun, 4 Apr 2021 16:36:27 GMT, John Neffenger wrote: >> This pull request allows for reproducible builds of JavaFX on Linux, macOS, >> and Windows by defining the `SOURCE_DATE_EPOCH` environment variable. For >> example, the following commands create a reproducible build: >> >> >> $ export SOURCE_DATE_EPOCH=$(git log -1 --pretty=%ct) >> $ bash gradlew sdk jmods javadoc >> $ strip-nondeterminism -v -T $SOURCE_DATE_EPOCH build/jmods/*.jmod >> >> >> The three commands: >> >> 1. set the build timestamp to the date of the latest source code change, >> 2. build the JavaFX SDK libraries, JMOD archives, and API documentation, and >> 3. recreate the JMOD files with stable file modification times and ordering. >> >> The third command won't be necessary once Gradle can build the JMOD archives >> or the `jmod` tool itself has the required support. For more information on >> the environment variable, see the [`SOURCE_DATE_EPOCH`][1] page. For more >> information on the command to recreate the JMOD files, see the >> [`strip-nondeterminism`][2] repository. I'd like to propose that we allow >> for reproducible builds in JavaFX 17 and consider making them the default in >> JavaFX 18. >> >> Fixes >> >> There are at least four sources of non-determinism in the JavaFX builds: >> >> 1. Build timestamp >> >> The class `com.sun.javafx.runtime.VersionInfo` in the JavaFX Base module >> stores the time of the build. Furthermore, for builds that don't run on the >> Hudson continuous integration tool, the class adds the build time to the >> system property `javafx.runtime.version`. >> >> 2. Modification times >> >> The JAR, JMOD, and ZIP archives store the modification time of each file. >> >> 3. File ordering >> >> The JAR, JMOD, and ZIP archives store their files in the order returned >> by the file system. The native shared libraries also store their object >> files in the order returned by the file system. Most file systems, though, >> do not guarantee the order of a directory's file listing. >> >> 4. Build path >> >> The class `com.sun.javafx.css.parser.Css2Bin` in the JavaFX Graphics >> module stores the absolute path of its `.css` input file in the >> corresponding `.bss` output file, which is then included in the JavaFX >> Controls module. >> >> This pull request modifies the Gradle and Groovy build files to fix the >> first three sources of non-determinism. A later pull request can modify the >> Java files to fix the fourth. >> >> [1]: https://reproducible-builds.org/docs/source-date-epoch/ >> [2]: https://salsa.debian.org/reproducible-builds/strip-nondeterminism > > John Neffenger has updated the pull request incrementally with one additional > commit since the last revision: > > Include WebKit shared library for Windows > > Enable reproducible builds of the native WebKit shared library for > Windows (jfxwebkit.dll) when SOURCE_DATE_EPOCH is defined. > As an alternative, you could close that PR ... and list both issues in this > PR ... I prefer to keep them separate because they have distinct attributions and descriptions. Few developers will read #422 if we merge them. They also pose distinct questions for reviewers. The implicit question in #422 is, should we define an external environment variable or just remove the build timestamp entirely? The implicit question in this pull request is, given the environment variable, should we also use it as a flag for reproducible builds? Furthermore, which sources of non-determinism should be removed unconditionally, and which should be conditional on the flag? I chose conservative answers to those questions in my changes, but I'm open to other suggestions. - PR: https://git.openjdk.java.net/jfx/pull/446
Re: RFR: 8264449: Enable reproducible builds with SOURCE_DATE_EPOCH [v3]
On Sun, 4 Apr 2021 16:36:27 GMT, John Neffenger wrote: >> This pull request allows for reproducible builds of JavaFX on Linux, macOS, >> and Windows by defining the `SOURCE_DATE_EPOCH` environment variable. For >> example, the following commands create a reproducible build: >> >> >> $ export SOURCE_DATE_EPOCH=$(git log -1 --pretty=%ct) >> $ bash gradlew sdk jmods javadoc >> $ strip-nondeterminism -v -T $SOURCE_DATE_EPOCH build/jmods/*.jmod >> >> >> The three commands: >> >> 1. set the build timestamp to the date of the latest source code change, >> 2. build the JavaFX SDK libraries, JMOD archives, and API documentation, and >> 3. recreate the JMOD files with stable file modification times and ordering. >> >> The third command won't be necessary once Gradle can build the JMOD archives >> or the `jmod` tool itself has the required support. For more information on >> the environment variable, see the [`SOURCE_DATE_EPOCH`][1] page. For more >> information on the command to recreate the JMOD files, see the >> [`strip-nondeterminism`][2] repository. I'd like to propose that we allow >> for reproducible builds in JavaFX 17 and consider making them the default in >> JavaFX 18. >> >> Fixes >> >> There are at least four sources of non-determinism in the JavaFX builds: >> >> 1. Build timestamp >> >> The class `com.sun.javafx.runtime.VersionInfo` in the JavaFX Base module >> stores the time of the build. Furthermore, for builds that don't run on the >> Hudson continuous integration tool, the class adds the build time to the >> system property `javafx.runtime.version`. >> >> 2. Modification times >> >> The JAR, JMOD, and ZIP archives store the modification time of each file. >> >> 3. File ordering >> >> The JAR, JMOD, and ZIP archives store their files in the order returned >> by the file system. The native shared libraries also store their object >> files in the order returned by the file system. Most file systems, though, >> do not guarantee the order of a directory's file listing. >> >> 4. Build path >> >> The class `com.sun.javafx.css.parser.Css2Bin` in the JavaFX Graphics >> module stores the absolute path of its `.css` input file in the >> corresponding `.bss` output file, which is then included in the JavaFX >> Controls module. >> >> This pull request modifies the Gradle and Groovy build files to fix the >> first three sources of non-determinism. A later pull request can modify the >> Java files to fix the fourth. >> >> [1]: https://reproducible-builds.org/docs/source-date-epoch/ >> [2]: https://salsa.debian.org/reproducible-builds/strip-nondeterminism > > John Neffenger has updated the pull request incrementally with one additional > commit since the last revision: > > Include WebKit shared library for Windows > > Enable reproducible builds of the native WebKit shared library for > Windows (jfxwebkit.dll) when SOURCE_DATE_EPOCH is defined. Pending review by @johanvos -- I bumped the number of reviewers, so he will have a change to review before it is approved. - PR: https://git.openjdk.java.net/jfx/pull/446
Re: RFR: 8264449: Enable reproducible builds with SOURCE_DATE_EPOCH [v3]
On Sun, 4 Apr 2021 16:36:27 GMT, John Neffenger wrote: >> This pull request allows for reproducible builds of JavaFX on Linux, macOS, >> and Windows by defining the `SOURCE_DATE_EPOCH` environment variable. For >> example, the following commands create a reproducible build: >> >> >> $ export SOURCE_DATE_EPOCH=$(git log -1 --pretty=%ct) >> $ bash gradlew sdk jmods javadoc >> $ strip-nondeterminism -v -T $SOURCE_DATE_EPOCH build/jmods/*.jmod >> >> >> The three commands: >> >> 1. set the build timestamp to the date of the latest source code change, >> 2. build the JavaFX SDK libraries, JMOD archives, and API documentation, and >> 3. recreate the JMOD files with stable file modification times and ordering. >> >> The third command won't be necessary once Gradle can build the JMOD archives >> or the `jmod` tool itself has the required support. For more information on >> the environment variable, see the [`SOURCE_DATE_EPOCH`][1] page. For more >> information on the command to recreate the JMOD files, see the >> [`strip-nondeterminism`][2] repository. I'd like to propose that we allow >> for reproducible builds in JavaFX 17 and consider making them the default in >> JavaFX 18. >> >> Fixes >> >> There are at least four sources of non-determinism in the JavaFX builds: >> >> 1. Build timestamp >> >> The class `com.sun.javafx.runtime.VersionInfo` in the JavaFX Base module >> stores the time of the build. Furthermore, for builds that don't run on the >> Hudson continuous integration tool, the class adds the build time to the >> system property `javafx.runtime.version`. >> >> 2. Modification times >> >> The JAR, JMOD, and ZIP archives store the modification time of each file. >> >> 3. File ordering >> >> The JAR, JMOD, and ZIP archives store their files in the order returned >> by the file system. The native shared libraries also store their object >> files in the order returned by the file system. Most file systems, though, >> do not guarantee the order of a directory's file listing. >> >> 4. Build path >> >> The class `com.sun.javafx.css.parser.Css2Bin` in the JavaFX Graphics >> module stores the absolute path of its `.css` input file in the >> corresponding `.bss` output file, which is then included in the JavaFX >> Controls module. >> >> This pull request modifies the Gradle and Groovy build files to fix the >> first three sources of non-determinism. A later pull request can modify the >> Java files to fix the fourth. >> >> [1]: https://reproducible-builds.org/docs/source-date-epoch/ >> [2]: https://salsa.debian.org/reproducible-builds/strip-nondeterminism > > John Neffenger has updated the pull request incrementally with one additional > commit since the last revision: > > Include WebKit shared library for Windows > > Enable reproducible builds of the native WebKit shared library for > Windows (jfxwebkit.dll) when SOURCE_DATE_EPOCH is defined. Note that this PR is dependent on PR #422 . @jgneff As an alternative, you could close that PR (since all of the changes from that one are currently included here), and list both issues in this PR with Skara's `/issue add ...` command. It's up to you. - PR: https://git.openjdk.java.net/jfx/pull/446
Re: RFR: 8264449: Enable reproducible builds with SOURCE_DATE_EPOCH [v3]
On Sun, 4 Apr 2021 16:36:27 GMT, John Neffenger wrote: >> This pull request allows for reproducible builds of JavaFX on Linux, macOS, >> and Windows by defining the `SOURCE_DATE_EPOCH` environment variable. For >> example, the following commands create a reproducible build: >> >> >> $ export SOURCE_DATE_EPOCH=$(git log -1 --pretty=%ct) >> $ bash gradlew sdk jmods javadoc >> $ strip-nondeterminism -v -T $SOURCE_DATE_EPOCH build/jmods/*.jmod >> >> >> The three commands: >> >> 1. set the build timestamp to the date of the latest source code change, >> 2. build the JavaFX SDK libraries, JMOD archives, and API documentation, and >> 3. recreate the JMOD files with stable file modification times and ordering. >> >> The third command won't be necessary once Gradle can build the JMOD archives >> or the `jmod` tool itself has the required support. For more information on >> the environment variable, see the [`SOURCE_DATE_EPOCH`][1] page. For more >> information on the command to recreate the JMOD files, see the >> [`strip-nondeterminism`][2] repository. I'd like to propose that we allow >> for reproducible builds in JavaFX 17 and consider making them the default in >> JavaFX 18. >> >> Fixes >> >> There are at least four sources of non-determinism in the JavaFX builds: >> >> 1. Build timestamp >> >> The class `com.sun.javafx.runtime.VersionInfo` in the JavaFX Base module >> stores the time of the build. Furthermore, for builds that don't run on the >> Hudson continuous integration tool, the class adds the build time to the >> system property `javafx.runtime.version`. >> >> 2. Modification times >> >> The JAR, JMOD, and ZIP archives store the modification time of each file. >> >> 3. File ordering >> >> The JAR, JMOD, and ZIP archives store their files in the order returned >> by the file system. The native shared libraries also store their object >> files in the order returned by the file system. Most file systems, though, >> do not guarantee the order of a directory's file listing. >> >> 4. Build path >> >> The class `com.sun.javafx.css.parser.Css2Bin` in the JavaFX Graphics >> module stores the absolute path of its `.css` input file in the >> corresponding `.bss` output file, which is then included in the JavaFX >> Controls module. >> >> This pull request modifies the Gradle and Groovy build files to fix the >> first three sources of non-determinism. A later pull request can modify the >> Java files to fix the fourth. >> >> [1]: https://reproducible-builds.org/docs/source-date-epoch/ >> [2]: https://salsa.debian.org/reproducible-builds/strip-nondeterminism > > John Neffenger has updated the pull request incrementally with one additional > commit since the last revision: > > Include WebKit shared library for Windows > > Enable reproducible builds of the native WebKit shared library for > Windows (jfxwebkit.dll) when SOURCE_DATE_EPOCH is defined. *IEEE Software* just published a good article that describes the problems solved in part by this pull request. The article is called "Reproducible Builds: Increasing the Integrity of Software Supply Chains," by Chris Lamb and Stefano Zacchiroli. It's an easy read of 10 pages, available at the links below: * [IEEE Xplore](https://ieeexplore.ieee.org/document/9403390) - Registration required * [arXiv.org](https://arxiv.org/abs/2104.06020) - Free PDF download - PR: https://git.openjdk.java.net/jfx/pull/446
Re: RFR: 8264449: Enable reproducible builds with SOURCE_DATE_EPOCH [v3]
On Sun, 4 Apr 2021 16:36:27 GMT, John Neffenger wrote: >> This pull request allows for reproducible builds of JavaFX on Linux, macOS, >> and Windows by defining the `SOURCE_DATE_EPOCH` environment variable. For >> example, the following commands create a reproducible build: >> >> >> $ export SOURCE_DATE_EPOCH=$(git log -1 --pretty=%ct) >> $ bash gradlew sdk jmods javadoc >> $ strip-nondeterminism -v -T $SOURCE_DATE_EPOCH build/jmods/*.jmod >> >> >> The three commands: >> >> 1. set the build timestamp to the date of the latest source code change, >> 2. build the JavaFX SDK libraries, JMOD archives, and API documentation, and >> 3. recreate the JMOD files with stable file modification times and ordering. >> >> The third command won't be necessary once Gradle can build the JMOD archives >> or the `jmod` tool itself has the required support. For more information on >> the environment variable, see the [`SOURCE_DATE_EPOCH`][1] page. For more >> information on the command to recreate the JMOD files, see the >> [`strip-nondeterminism`][2] repository. I'd like to propose that we allow >> for reproducible builds in JavaFX 17 and consider making them the default in >> JavaFX 18. >> >> Fixes >> >> There are at least four sources of non-determinism in the JavaFX builds: >> >> 1. Build timestamp >> >> The class `com.sun.javafx.runtime.VersionInfo` in the JavaFX Base module >> stores the time of the build. Furthermore, for builds that don't run on the >> Hudson continuous integration tool, the class adds the build time to the >> system property `javafx.runtime.version`. >> >> 2. Modification times >> >> The JAR, JMOD, and ZIP archives store the modification time of each file. >> >> 3. File ordering >> >> The JAR, JMOD, and ZIP archives store their files in the order returned >> by the file system. The native shared libraries also store their object >> files in the order returned by the file system. Most file systems, though, >> do not guarantee the order of a directory's file listing. >> >> 4. Build path >> >> The class `com.sun.javafx.css.parser.Css2Bin` in the JavaFX Graphics >> module stores the absolute path of its `.css` input file in the >> corresponding `.bss` output file, which is then included in the JavaFX >> Controls module. >> >> This pull request modifies the Gradle and Groovy build files to fix the >> first three sources of non-determinism. A later pull request can modify the >> Java files to fix the fourth. >> >> [1]: https://reproducible-builds.org/docs/source-date-epoch/ >> [2]: https://salsa.debian.org/reproducible-builds/strip-nondeterminism > > John Neffenger has updated the pull request incrementally with one additional > commit since the last revision: > > Include WebKit shared library for Windows > > Enable reproducible builds of the native WebKit shared library for > Windows (jfxwebkit.dll) when SOURCE_DATE_EPOCH is defined. Thank you, Alexander and Arun, for looking over the Makefiles and verifying the builds. > I'd like to take a week to get feedback and mull over the code changes ... After realizing how little I knew about Gradle, I did some research last week and took four courses on Groovy, Gradle, and the Gradle Java Plugin. Although I now see a couple things I could change, I'm still happy with the commits as they are. I welcome any comments or questions. Once #422 and this pull request are integrated, I can follow up with changes to make the `all` task reproducible (fixing `javafx-sdk-17.zip`) and to remove the build path from the `.bss` files (fix number 4 in the first comment). - PR: https://git.openjdk.java.net/jfx/pull/446
Re: RFR: 8264449: Enable reproducible builds with SOURCE_DATE_EPOCH [v3]
On Sun, 4 Apr 2021 16:36:27 GMT, John Neffenger wrote: >> This pull request allows for reproducible builds of JavaFX on Linux, macOS, >> and Windows by defining the `SOURCE_DATE_EPOCH` environment variable. For >> example, the following commands create a reproducible build: >> >> $ export SOURCE_DATE_EPOCH=$(git log -1 --pretty=%ct) >> $ bash gradlew sdk jmods javadoc >> $ strip-nondeterminism -v -T $SOURCE_DATE_EPOCH build/jmods/*.jmod >> >> The three commands: >> >> 1. set the build timestamp to the date of the latest source code change, >> 2. build the JavaFX SDK libraries, JMOD archives, and API documentation, and >> 3. recreate the JMOD files with stable file modification times and ordering. >> >> The third command won't be necessary once Gradle can build the JMOD archives >> or the `jmod` tool itself has the required support. For more information on >> the environment variable, see the [`SOURCE_DATE_EPOCH`][1] page. For more >> information on the command to recreate the JMOD files, see the >> [`strip-nondeterminism`][2] repository. I'd like to propose that we allow >> for reproducible builds in JavaFX 17 and consider making them the default in >> JavaFX 18. >> >> Fixes >> >> There are at least four sources of non-determinism in the JavaFX builds: >> >> 1. Build timestamp >> >> The class `com.sun.javafx.runtime.VersionInfo` in the JavaFX Base module >> stores the time of the build. Furthermore, for builds that don't run on the >> Hudson continuous integration tool, the class adds the build time to the >> system property `javafx.runtime.version`. >> >> 2. Modification times >> >> The JAR, JMOD, and ZIP archives store the modification time of each file. >> >> 3. File ordering >> >> The JAR, JMOD, and ZIP archives store their files in the order returned >> by the file system. The native shared libraries also store their object >> files in the order returned by the file system. Most file systems, though, >> do not guarantee the order of a directory's file listing. >> >> 4. Build path >> >> The class `com.sun.javafx.css.parser.Css2Bin` in the JavaFX Graphics >> module stores the absolute path of its `.css` input file in the >> corresponding `.bss` output file, which is then included in the JavaFX >> Controls module. >> >> This pull request modifies the Gradle and Groovy build files to fix the >> first three sources of non-determinism. A later pull request can modify the >> Java files to fix the fourth. >> >> [1]: https://reproducible-builds.org/docs/source-date-epoch/ >> [2]: https://salsa.debian.org/reproducible-builds/strip-nondeterminism > > John Neffenger has updated the pull request incrementally with one additional > commit since the last revision: > > Include WebKit shared library for Windows > > Enable reproducible builds of the native WebKit shared library for > Windows (jfxwebkit.dll) when SOURCE_DATE_EPOCH is defined. Changes looks good. Verified the build on all 3 platforms. - Marked as reviewed by ajoseph (Committer). PR: https://git.openjdk.java.net/jfx/pull/446
Re: RFR: 8264449: Enable reproducible builds with SOURCE_DATE_EPOCH [v3]
> This pull request allows for reproducible builds of JavaFX on Linux, macOS, > and Windows by defining the `SOURCE_DATE_EPOCH` environment variable. For > example, the following commands create a reproducible build: > > $ export SOURCE_DATE_EPOCH=$(git log -1 --pretty=%ct) > $ bash gradlew sdk jmods javadoc > $ strip-nondeterminism -v -T $SOURCE_DATE_EPOCH build/jmods/*.jmod > > The three commands: > > 1. set the build timestamp to the date of the latest source code change, > 2. build the JavaFX SDK libraries, JMOD archives, and API documentation, and > 3. recreate the JMOD files with stable file modification times and ordering. > > The third command won't be necessary once Gradle can build the JMOD archives > or the `jmod` tool itself has the required support. For more information on > the environment variable, see the [`SOURCE_DATE_EPOCH`][1] page. For more > information on the command to recreate the JMOD files, see the > [`strip-nondeterminism`][2] repository. I'd like to propose that we allow for > reproducible builds in JavaFX 17 and consider making them the default in > JavaFX 18. > > Fixes > > There are at least four sources of non-determinism in the JavaFX builds: > > 1. Build timestamp > > The class `com.sun.javafx.runtime.VersionInfo` in the JavaFX Base module > stores the time of the build. Furthermore, for builds that don't run on the > Hudson continuous integration tool, the class adds the build time to the > system property `javafx.runtime.version`. > > 2. Modification times > > The JAR, JMOD, and ZIP archives store the modification time of each file. > > 3. File ordering > > The JAR, JMOD, and ZIP archives store their files in the order returned > by the file system. The native shared libraries also store their object files > in the order returned by the file system. Most file systems, though, do not > guarantee the order of a directory's file listing. > > 4. Build path > > The class `com.sun.javafx.css.parser.Css2Bin` in the JavaFX Graphics > module stores the absolute path of its `.css` input file in the corresponding > `.bss` output file, which is then included in the JavaFX Controls module. > > This pull request modifies the Gradle and Groovy build files to fix the first > three sources of non-determinism. A later pull request can modify the Java > files to fix the fourth. > > [1]: https://reproducible-builds.org/docs/source-date-epoch/ > [2]: https://salsa.debian.org/reproducible-builds/strip-nondeterminism John Neffenger has updated the pull request incrementally with one additional commit since the last revision: Include WebKit shared library for Windows Enable reproducible builds of the native WebKit shared library for Windows (jfxwebkit.dll) when SOURCE_DATE_EPOCH is defined. - Changes: - all: https://git.openjdk.java.net/jfx/pull/446/files - new: https://git.openjdk.java.net/jfx/pull/446/files/afeb3fe0..cb8510c6 Webrevs: - full: https://webrevs.openjdk.java.net/?repo=jfx=446=02 - incr: https://webrevs.openjdk.java.net/?repo=jfx=446=01-02 Stats: 9 lines in 2 files changed: 9 ins; 0 del; 0 mod Patch: https://git.openjdk.java.net/jfx/pull/446.diff Fetch: git fetch https://git.openjdk.java.net/jfx pull/446/head:pull/446 PR: https://git.openjdk.java.net/jfx/pull/446
