great feedback, thank you: this proves the initial set of only 3 plugins is a good basis. For checking the many output artifacts from a build, future buildinfo file [1] should help a lot. I still have many usability concerns for Maven multi-module builds (starting with: should we generate only the root buildinfo or one buildinfo per Maven module?) Let's keep the ball rolling: today, it's plugins release day!!!
Regards, Hervé [1] https://reproducible-builds.org/docs/jvm/ Le vendredi 1 novembre 2019, 13:34:32 CET Christofer Dutz a écrit : > Hi, > > so I just updated the versions of the 3 plugins and gave the Apache PLC4X > build a little spin ... > https://github.com/apache/plc4x/tree/feature/reproducible-builds > > Did two executions of this: > mvn -U -P > with-boost,with-java,with-dotnet,with-cpp,with-python,with-proxies,with-san > dbox,with > -DaltDeploymentRepository=snapshot-repo::default::file:./local-snapshots-di > r clean deploy (but with a different second local repo location) > > Then I used "diff" and "cmp" to compare individual files and it didn't show > differences with the artifacts I chose ... now I guess I'll have to whip > up some little bash script to sort of compare the artifacts from one > directory with the other (Unfortunately the SNAPSHOT timestamps are a > little annoying :-/ > We do have some C++, C# and Python modules ... but I wouldn't expect the C++ > and C# to be reproducible. > Chris > > > Am 01.11.19, 12:04 schrieb "Hervé BOUTEMY" <herve.bout...@free.fr>: > > Le vendredi 1 novembre 2019, 09:40:40 CET Christofer Dutz a écrit : > > > Hi Herve, > > > > thanks for that will try it asap and report any findings back. > > > > But good to know that there is a difference between JDK major versions > > and > > OSes ... so it would probably be best to stage releases on Linux with > > an > > OpenJDK of the minimum supported version? > > Just thinking how to make it > > possible to verify without having to buy Mac or Windows licenses ... > > guess > > on every machine you could whip up a Ubuntu VM for verification. Just > > thinking about it ... perhaps it would be best to create a Docker > > image for doing the reproducible stuff ... > > just to be clear: the difference is on newline only, then Windows vs > anything else. You get the same result on Linux, MacOS, FreeBSD, or any > other Unix. > And if I want to be complete, if you get a source tarball from Windows, > extract it to a Linux/MacOS/... box and build with "mvn - > Dline.separator='\r\n'", in general, you get the same result as a build > under Windows: I tested multiple times, it worked, but we'll see if it > works always or just "in general" > > > > Are there any plans on creating a plugin to allow verification? > > > > Sort of something like this: > > "mvn package release:verify-reproduicble > > -DstagingRepoUrl=a.b.c.de/repo/blahblahblah" > > (Which doesn't deploy the > > artifacts, but instead download them and do a binary comparison) > > yes, but for now it was completely another form: this is the "Buildinfo > file" proposal https://reproducible-builds.org/docs/jvm/ > that I stopped to work on 1 year ago given I had no chance to get the > same output: it's now a good time to restart working on it as next steps > > > > Also it could be great if the release-plugin could automatically set > > the > > property: > > a) if it finds the "project.build.outputTimestamp" set to some > > placeholder value b) if some switch tells it to prepare a > > reproducible > > build by using some sort of "switch" parameter > > Guess that would sort of close the loop to get the biggest benefit out > > of > > the reproducible builds. > > +1 > issue has been created > https://issues.apache.org/jira/browse/MRELEASE-1029 But I didn't work on > it yet, help welcome > > > > I would be happy to help as I think this is one > > of the greatest new features. (Ok ... perhaps besides the > > sound-output-extension I learned about yesterday ;-) ) > > +1 > the current step is important, but it's only the beginning of the story > from an effective usage perspective > > Regards, > > Hervé > > > > > > Chris > > > > > > Am 01.11.19, 09:24 schrieb "Hervé BOUTEMY" <herve.bout...@free.fr>: > > > > > > Le jeudi 31 octobre 2019, 17:26:52 CET Christofer Dutz a écrit : > > > > > > > > > Hi all, > > > > > > as I can see you're voting on releasing the reproducible build > > > extended > > > plugin versions. > > > > > > > > Is there any documentation on how to use this new > > > > > > > > > feature? > > > > > > I had a look at the confluence page, but that seemed like a > > > brainstorming > > > session. > > > > > > > > ok, the Wiki page [1] started as a brainstorming session, was > > updated to > > > > a proposal (the "Output Archive Entries Timestamp" parapgraph). > > And now I > > > probably should order paragraph a little bit, and add a "Making your > > build > > reproducible" section for end uses to have a quick explanation. > > > > I'll write the explanation here as a first try before working on > > the > > > > Wiki: > > > > > 1. upgrade your plugins to reproducible version, particularly > > > > mpaven-source-plugin, maven-jar-plugin and maven-assembly-plugin to > > vesion > > 3.2.0 > > 2. add project.build.outputTimestamp property with the timestamp > > > value that will be used in zip/jar/tar archives: <properties> > > > > > > > > <project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build.out > > putTi mestamp> > > </properties> > > > > > Notice: > > - there is no Maven version prerequisite, everything happens at > > plugins > > > > level > > - Reproducible Builds require to have no version ranges in > > > dependencies, generally gives different result on Unixes vs Windows, > > and > > generally depend on the major version of JDK used to compile (see "Out > > of > > Scope" paragraph) > > > > You have the basis configured, the output should be reproducible > > now. > > If something is still not reproducible, use diffoscope to find > > the > > > > unstable output, find the plugin that generated this output and check > > if > > there is a reproducible version available: if not, please open an > > issue to > > help plugin maintainers improving Reproducible Builds support at > > every > > plugin level. > > > > > [1] > > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682 > > 318 > > > > > > > > > > > > > > > > > > I would love to add this to the PLC4X build asap. > > > > > > > > I'd love to have feedback > > Don't hesitate to ping me. > > > > > > > > > > > > So I would like to test the release-candidates and vote too. > > > > > > > > I would love to have many tester and votes :) > > > > > > > > > > > > > > Chris > > > > > > > > > > > > Am 16.10.19, 14:42 schrieb "Hervé BOUTEMY" > > > <herve.bout...@free.fr>: > > > > > > > > > > > > Le mercredi 16 octobre 2019, 13:40:48 CEST Andreas Sewe a > > > écrit : > > > > > > > > > > > > > > > > > > > Emmanuel Bourg wrote: > > > > > > > > > > > > > > > > > Le 16/10/2019 à 08:35, Hervé BOUTEMY a écrit : > > > > > > > > > > > > > > > > > > > >> last question: now that we seem to fully understand > > > > >> each > > > > >> other, > > > > >> does it > > > > >> mean that you don't need any more "seconds since the > > > > >> epoch" > > > > >> format > > > > >> support for the property? > > > > > > > > > > > > > > > > > > > > > > > > > If Maven supports the SOURCE_DATE_EPOCH environment > > > > > variable > > > > > I > > > > > don't > > > > > think that's necessary, otherwise it would be nice to be > > > > > able > > > > > to > > > > > invoke > > > > > > > > > > Maven with: > > > > > > > > > > > > > > > > > > > > mvn package > > > > > -Dproject.build.outputTimestamp=$SOURCE_DATE_EPOCH > > > > > > > > > > > > > > > > > > > > > > > > > and this means supporting a timestamp formatted as > > > > > seconds > > > > > since > > > > > the > > > > > epoch. > > > > > > > > > > > > > > > > > > > > +1 > > > > > > > > The above would be a nice, simple way of bridging the gap > > > > between > > > > SOURCE_DATE_EPOCH and project.build.outputTimestamp. > > > > > > > > > > > > > > > > > > told like that, ok, why not > > > > > > > > > > > > > > > > > > > > > > > > > > If it is not too much trouble to implement the "\d+ -> > > > > seconds > > > > since > > > > epoch" heuristic, them I would love to see it included. > > > > > > > > > > > > > > > > > > ok, I'll do and prepare the release > > > > > > Regards, > > > > > > Hervé > > > > > > > > > > > > > > > > > > > > > > > > > > Best wishes, > > > > > > > > Andreas > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------ > > > ------ > > > --- > > > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > > > For additional commands, e-mail: dev-h...@maven.apache.org > > > > > > > > > > > > > > > > > > > > > ---------------------------------------------------------------- > > > ----- > > > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > > > For additional commands, e-mail: dev-h...@maven.apache.org > > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------ > > --- > > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > > For additional commands, e-mail: dev-h...@maven.apache.org > > > > > > > > > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > > > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
