On Thu, 4 Nov 2021 20:56:45 GMT, Andrew Leonard <aleon...@openjdk.org> wrote:
> This PR enables reproducible Jars, Jmods and openjdk image zip files > (eg.src.zip). > It provides support for SOURCE_DATE_EPOCH for Jar, Jmod and underlying > ZipOutputStream's. > It fixes the following keys issues relating to reproducibility: > - Jar and ZipOutputStream are not SOURCE_DATE_EPOCH aware > - Jar and ZipOutputStream now detect SOURCE_DATE_EPOCH environment setting > - Jar and Jmod file content ordering was non-determinsitic > - Fixes to Jar and Jmod main's to ensure sorted classes content ordering > - openjdk image zip file generation used "zip" which is non-determinsitic > - New openjdk build tool "GenerateZip" which produces the final > determinsitic zip files as part of the build and also detects > SOURCE_DATE_EPOCH > > Signed-off-by: Andrew Leonard <anleo...@redhat.com> > I think this is going to require discussion, a PR may be too premature. Is > your goal to get the JDK build and run-time images creates with jlink to use > the SOURCE_DATE_EPOCH? Are you looking for project builds that use the > jar/jmod/etc. tools to use it? Or are you looking to have every library and > application that uses the java.util.zip APIs to read it? If it includes the > latter, and the patch in the PR suggests you are, then it will require > analysis to work through the API spec changes. > > One suggestion is to look at JDK-8231640 and PR 5372 [1]. The original > proposal was to use SOURCE_DATE_EPOCH. After a lengthy discussion we > converged on the system property java.properties.date rather than the env > variable. I suspect that much of that discussion applies here. > > -Alan > > [1] https://github.com/[/pull/5372](https://github.com/openjdk/jdk/pull/5372) @AlanBateman Hi Alan, thank you for the comments. So yes, totally agree this has a way to go, I thought creating a PR would kick things off...! So just to clarify the initial objectives which you raise above. My current aim is actually purely an OpenJDK JDK image perspective, building on the work @magicus has been doing with SOURCE_DATE_EPOCH in the build, hence my natural approach to this PR. So from that perspective, it's purely a "build" change. However, the openjdk build obviously uses openjdk Jar, Jmod tooling to build itself, hence the direction I took in this PR. Magnus's PR #5372 is useful thank you, I had not discovered that change, as you say the direction there is similar to mine, and I understand the potential pitfalls of adding doPrivileged's etc. So that resolved to a new system property java.properties.date, which makes sense. So with that in mind, and given the objective of JDK image reproducibility, this would imply presumably an approach of a new cmd line option to Jar and Jmod tools (eg.--source-date <date>), since these tools are cmds? One of the key changes in this PR is to ZipOutputStream where it sets the ZipEntry.xdostime if not set, to the current time. Now I am thinking given the "objective", if we fix all upstream tooling to ensure they set ZipEntry times, using "--source-date" (or whatever mechanism), then we can then avoid changing ZipOutputStream I believe. JDK tools like jar, jmod generate/update extra files like Manifests, and in fact I think the current jmod implementation writes all its content using the current time. So something along these lines: jar -cf myjar.jar --source-date "<date>" * jmod create --source-date "<date>" ... my.jmod The jar, jmod,.. tooling then ensure all ZipEntry times are set to "source-date". I chose "source-date" name based on being synonymous with SOURCE_DATE_EPOCH, implying a similar "use case". So taking this approach for JDK tooling should achieve the reproducible JDK image objective, without a java API behavior change I am hoping. Thoughts? Regards Andrew ------------- PR: https://git.openjdk.java.net/jdk/pull/6268
