maven-site-plugin and sisu-inject-plexus
Hi, maven-site-plugin has a dependency on sisu-inject-plexus [1] but it doesn't seem to be used. The project still builds and the tests pass without it. Is it safe to assume it can be removed? Emmanuel Bourg [1] https://github.com/apache/maven-site-plugin/blob/maven-site-plugin-3.9.1/pom.xml#L284 - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: Talk: Bootstrapping the Java Ecosystem
Le 20/11/2020 à 10:06, Tamás Cservenák a écrit : > Also, according to your explanation, the problem is now solved once for > all, right? You do have (those distros you mention, like Guix) Maven 3.6.3 > built now, so you do not have to repeat this anymore? That's correct, in Debian this was done only once, future versions of Maven will be built with the version 3.6.3 currently packaged. The bootstrapping is really useful to get an initial package in a distribution, or to provide an irrefutable proof that a set of sources generate a given binary when you care about reproducibility. > (Another joke) By the way, what do you do with npm packages? I leave this mess to the JS folks :) Emmanuel Bourg - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: Talk: Bootstrapping the Java Ecosystem
On 19/11/2020 09:51, Tamás Cservenák wrote: > Without starting any flame wars, am really curious: why are you > repackaging Maven? > > I'd understand for OS/distro native packages, but > why do you rebuild JVM bytecode as well? > > Again, am not to start any flame war, am just curious! Short answer: why not? This is an Open Source project, not an Open Binary project. Anyone should be able to rebuild the code, and in an ideal world where every project is reproducible, get byte identical binaries. Long answer: Debian, Fedora, and I assume Guix are "closed" ecosystems where you can rebuild every component from sources without needing tools or libraries outside of the distribution. If you were alone on a desert island with just a laptop, the sources and no internet connection, you would be able to rebuild any part of the distribution from scratch. This really goes to the roots of the open source philosophy, open source projects are meant to be built from sources, and if it's not possible then there is a problem somewhere. Assuming every project becomes reproducible at some point (see https://reproducible-builds.org for why it matters) the question of knowing who produced the binaries become irrelevant, because everyone get the exact same binaries. > 3) What are you really building? As in video, it is said > several times that you "mutilate" some package to build > it, then use it to "bootstrap" some other package, and finally > you rebuild the target package. Given in the process there > was once a "mutilated" tool, how are you certain, that output > of the build is correct (I have no doubts about > reproducibility)? How do you prove that output is what > it is thought/assumed to be? In Debian the Maven package we rebuild from sources is itself used to build all the other Maven based projects packaged in Debian (that's over 600 projects currently), so regressions are caught pretty quickly (it's rare but it happens sometimes when the binary compatibility is broken in a core library like maven-shared-utils). > 3) (Joker) What is the overall CO2 footprint of distros like > these? I believe you did not use Apple M1 for this work... :) Probably a tiny fraction of what bitcoin mining, Travis CI and Youtube/Netflix 4K videos generate ;) Emmanuel Bourg - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: Talk: Bootstrapping the Java Ecosystem
Hi Björn, Nice presentation, the packaging of Maven in Debian followed a similar path but we never documented the process. Did you go as far as recording the exact steps and build order required to build from scratch? Spoiler for the next part of your quest toward packaging the Android SDK: Maven was the easy part, Gradle and Kotlin are many leagues above in term of circular dependencies and headache. We have been trying to bootstrap Kotlin for 2 years in Debian and haven't found the right path yet. Emmanuel Bourg On 18/11/2020 21:29, Björn Höfling wrote: > Dear Maven Developers, > > more than 4 years ago I naively asked you on how to build Maven from > sources without using Maven. > > If you are interested in a declarative, bootstrappable, reproducible > and effectively executable answer to this question, Julien Lepiller > recorded a video on how he bootstrapped Maven and a maven-build-system > for GNU Guix with only using the ant-build-system. He shows how to > bootstrap Maven only from sources, which difficulties he had and how he > mastered the dependency cycles and other problems. > > You can find a link to the video recordings in this announcement: > > https://guix.gnu.org/en/blog/2020/online-guix-day-announce-2/ > > If you have any questions, you can join the discussions on Guix day, > the discussions for this talk will be on Sunday 2020-11-22, 16:00–16:25 > UTC. > > Happy Hacking, > > Björn > - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: Dynamic phases proposal
Le 25/10/2019 à 21:01, Stephen Connolly a écrit : > https://cwiki.apache.org/confluence/display/MAVEN/Dynamic+phases > > Thoughts? Sounds interesting. You may want to forbid the before/after prefix on the deprecated phases (such as after:pre-clean). The name of the phases use to start with a verb, that would no longer be true with the proposed 'sources' and 'resources' phases. Since it isn't that common to invoke these phases directly from the command line maybe the longer name could be retained to preserve the consistency (generate-sources, etc). Emmanuel Bourg - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: last review of Reproducible Builds proposal
Le 16/10/2019 à 08:35, Hervé BOUTEMY a écrit : > last question: now that we seem to fully understand each other, does it mean > that you don't need any more "seconds since the epoch" format support for the > property? If Maven supports the SOURCE_DATE_EPOCH environment variable I don't think that's necessary, otherwise it would be nice to be able to invoke Maven with: mvn package -Dproject.build.outputTimestamp=$SOURCE_DATE_EPOCH and this means supporting a timestamp formatted as seconds since the epoch. Emmanuel Bourg - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: last review of Reproducible Builds proposal
Le 10/10/2019 à 19:28, Hervé BOUTEMY a écrit : > the only little mis-interpretation is that it is a pure build information, > then I don't see why this property would appear in a consumer POM Thank you for the clarification, that makes perfectly sense. And I now see the benefit of using a property that can be inherited. In a multi modules project it's only necessary to define the timestamp once in the root pom. Parent poms deployed to Maven Central will never include a timestamp and there is no risk of affecting other projects deriving from the pom. Emmanuel Bourg - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: last review of Reproducible Builds proposal
Le 07/10/2019 à 14:40, Andreas Sewe a écrit :
> - Can we get something analogous to maven.build.timestamp.format? One
> could then write the following to be compatible with env.SOURCE_DATE_EPOCH:
>
>
>
>
> ${env.SOURCE_DATE_EPOCH}
>
> ...
>
>
+1, some kind of support for SOURCE_DATE_EPOCH would be nice. Either
this (but maybe with only one property supporting both formats) or by
overriding automatically the property when SOURCE_DATE_EPOCH is set.
Emmanuel Bourg
-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org
Re: last review of Reproducible Builds proposal
Le 06/10/2019 à 20:13, Hervé BOUTEMY a écrit : > no, it does not add any dependency on developer configuration: > 2019-10-05T18:37:42Z == 2019-10-05T20:37:42+02:00 == 2019-10-05T16:37:42-02:00 yes but: "2019-10-05T18:37:42Z" != "2019-10-05T20:37:42+02:00" != "2019-10-05T16:37:42-02:00" The point is, two developers may generate a different pom if the local timezone is used. A fixed timezone is necessary to ensure the pom itself is reproducible. > when will this value be written in the pom.xml is independant: currently, in > my PoC, I wrote the values by hand. In the future, it will probably be > updated > by maven-release-plugin, and we'll have to choose if the timestamp is written > in Z or if it is written in local timezone with its offset: both ways of > expressing the timestamp are valid and will give reproducible result The jar generated is reproducible, but not the pom. I suspect that if the jar includes the pom it'll break its reproducibility too (this is the default for maven-jar-plugin, but I don't know if it embeds the original pom without the timestamp, or the generated pom with the timestamp). > once again, war files taken apart for web servers, who looks at timestamp in > zip files? archive timestamps are just the tip of the iceberg. There are more visible timestamps elsewhere, for example in the javadoc headers, in .properties files, in OSGi attributes, sometimes in the source files... Emmanuel Bourg - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: last review of Reproducible Builds proposal
Le 06/10/2019 à 09:43, Hervé BOUTEMY a écrit : > Notice that you can also express a timezone (as digits), as seen in the unit > tests. I know but that's not desirable, otherwise the formatted timestamp would depend on the timezone of the developer and that would harm the reproducibility of the pom (I assume the timestamp is added automatically to the pom during the build, is that right?). >> Would it be possible to prevent this property from being inherited? > AFAIK no. > And I find that default inheritance from parent to child is a nice feature. Why do you think this is a nice feature? I can only see the downside of child projects having a timestamp clamped in the past and leaving the developers clueless about the reason. This would be especially bad if a widely used parent pom like org.sonatype.oss:oss-parent or org.apache:apache had it. Emmanuel Bourg - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: last review of Reproducible Builds proposal
Le 05/10/2019 à 19:52, Hervé BOUTEMY a écrit : > based on the feedback I got, I updated the proposal: > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318 > > The archives entries timestamp is now configured with > project.build.outputTimestamp property, in ISO-8601 format > > > > 2019-10-02T08:04:00Z > > You may want to specify explicitly that the timestamp must be formatted with the UTC timezone (as in your example). Would it be possible to prevent this property from being inherited? Emmanuel Bourg - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: [DISCUSS] Maven 3.7.0
Le 03/10/2019 à 16:54, Karl Heinz Marbaise a écrit : > Hm.. first Java 7 is out for eight years now (2011) (End of live) and > has no public updates for security/bug fixes etc. since 2015 RedHat still maintains OpenJDK 7 until June 2020 [1]. Emmanuel Bourg [1] https://access.redhat.com/articles/1299013#OpenJDK_Lifecycle_Dates_and_RHEL_versions - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: [DISCUSS] configuration for Reproducible Builds
Le 29/09/2019 à 20:46, Hervé BOUTEMY a écrit : > This is exactly how I see Reproducible Builds for the future: > - select versions of plugins that bring reproducible output > - either inherit or define a local timestamp > > et voilà, it's so easy (once plugins support)... How do you plan to capture the elements of the build environment necessary to build identical artifacts (JDK used, command line parameters) ? As project properties too? Emmanuel Bourg - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: [DISCUSS] configuration for Reproducible Builds
Le 28/09/2019 à 17:55, Hervé BOUTEMY a écrit : > Putting that parameter as a pom property with a well known name and value > format permits to share the configuration between every packaging plugin. > This also has the advantage that child poms will inherit from parent value, > and eventually override. It seems a bit odd to me to tie the build timestamp to the pom. The fact that it could be inherited is disturbing, i.e. if you forget to override it in a subproject your build time will be wrong. > WDYT? Any other idea? I thought the timestamp would rather go to a separate file deployed along the pom and capturing the build environment. What Maven needs then is a command line parameter to force a specific build time (and/or support for the SOURCE_DATE_EPOCH environment variable). Emmanuel Bourg - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: Reproducible Builds for Maven
Le 23/09/2019 à 01:52, Hervé BOUTEMY a écrit : > after a few years of testing, thinking, procrastination and hard work (thank > you Thomas for your talk at Devoxx France 2016 [1]), I think I achieved a key > step this week-end toward native Reproducible Builds with Maven [2]: Maven > core itself can be built in a reproducible way! An important milestone on a very long journey. Well done! Emmanuel Bourg - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: Fwd: FOSDEM 19 Debian Java talk
Le 15/02/2019 à 17:34, Hervé BOUTEMY a écrit : > IIUC you don't use the same versions of dependencies (because of uniqueness > of each lib in Debian [1]) as those used in a "normal unconstrained" build > from source [2] > I don't yet understand the details of Debian build system, if the symlinks > are used at build time or only at runtime, but even if you the resulting jars > of the Debian build from Maven source are the same binaries, the execution of > Maven does not run with the same dependencies, then there may be subtle > differences during the run (for example from jansi, or guice, or commons-cli > or ...) I try to stick as much as possible to the same versions of the dependencies, especially the sensitive ones like Guice or Aether, but there are still some divergences. I've reviewed the dependencies installed in the /usr/share/maven/lib directory by the maven/3.6.0 package in Debian unstable and we currently have the following differences: MavenDebian commons-lang3 3.8.13.8 commons-io 2.5 2.6 guava 25.1 19.0 jansi 1.17 1.17.1 jansi-native1.17 1.18 maven-shared-utils 3.2.13.3.0 plexus-utils3.1.03.1.1 wagon 3.2.03.3.1 The versions are slightly ahead to the ones in Maven 3.6.0, except for Guava. Guava is only used by Guice for Preconditions, immutable collections and other basic operations. I don't think the difference is significant. At least, I've verified that the Guice 4.2.1 tests pass with Guava 19.0. > IIUC, that uniqueness of every library is a Debian principle, then a Maven > distribution from Debian cannot be equivalent to a "normal" distribution > where library versions are purely selected by Maven source's pom.xml, without > adding any constraint > This is why having Debian's distribution display its name in "mvn -v" would > make that more clear. > > Let's discuss a little bit about plugins: when Debian launches the "mvn" > command to build some Debian package, does it use Maven Central like anybody? > = where do the Maven plugins used during build come from? There are two different cases: 1. What Maven does when launched by a user 2. What Maven does when it's used in a Debian package build 1. When the user launches "mvn" to build a project it behaves as usual by downloading the dependencies from Maven Central or the other repositories defined in the pom. The Maven plugins also available as Debian packages are never used. The only code compiled by Debian that is executed is the set of dependencies found in the Maven lib directory (/usr/share/maven/lib). The default plugin versions are also exactly identical. Before Maven 3.5.0 we patched default-bindings.xml to align them with the versions packaged in Debian. We stopped to preserve the same default behavior. This is new in Debian 10 "Buster" (and the latest Ubuntu 18.04 LTS), the current stable version Debian 9 "Stretch" has Maven 3.3.9 with different default plugins. 2. Debian builds are offline, completely disconnected from the network. It's impossible to download dependencies or plugins from Maven Central. That's why we recreate a Maven repository in /usr/share/maven-repo/ and instruct Maven to fetch the dependencies there instead of Maven Central when building a Debian package. This repository is populated by installing the packaged plugins and dependencies. If there are Maven developers interested I can do a live demo showing how Maven projects are built into Debian packages. Emmanuel Bourg - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: Fwd: FOSDEM 19 Debian Java talk
Hi Robert, Some comments with my Maven package maintainer hat. Le 12/02/2019 à 20:09, Robert Scholte a écrit : > I'm also wondering how you build Maven, since Maven is > being built with Maven. That should be a challenge to also rebuild all > plugins, etc. The maven package is rebuilt with the version of Maven previously packaged, that's not really a problem now. In the past we used a convoluted Ant build mimicking the Maven phases but that's history now. The source code is fetched from the release tags on the Git repositories (the tags are polled everyday and we're warned when a new release is available). One project (library, plugin, etc) is mapped to one package, and the upgrade process is done manually. It isn't terribly difficult but that's quite time consuming. If there are Debian/Ubuntu users in the Maven community interested in helping they are very welcome. > And how do you test this and confirm that it works as the official > distribution? > Sure, *IF* Maven is 100% reproducible then you can rely on our > test-infra, but that's not the situation. Debian has a CI infra rebuilding 550+ packages using Maven as build tool, so regressions tend to be caught fairly quickly (and immediately [1] reported [2][3]). Also the version provided in the stable release is available after months of user testing. So I'm pretty confident the Debian package is true to the Apache binaries. > So here are my main questions: > - Are you making clear that you're not using the official Maven > distribution, e.g. by adjust the info from 'mvn --version'? No we aren't, but that's worth considering. Note that as the Maven reproducibility improves this will become unnecessary because at some point we'll be able to build binaries strictly identical to the Apache ones. > - What is the optimum way of distributing Maven sources? For example: > also providing compile and package scripts (instead of calling > maven-plugins)? The current system is mostly fine to us. The only issue I got recently was the embedding of the SLF4J sources at build time, because we don't build binary packages installing the sources artifacts of the libraries and they aren't available at build time (we could though, that's a little more work). In this case I patched the build [4] to embed the compiled classes with the shade plugin instead (the patch was refused though [5], but that's a fairly minor divergence). > Interesting :) We've been discussing how we could get more contributors > and Kotlin was one idea, but that one didn't make it. > Even though we as Maven developers don't like the wrapper, the community > is asking for it, so we're seriously considering to add it as part of > Maven Core. The wrapper would have no significant impact on the Debian packaging, we just remove it before assembling the source package (no binaries are allowed). Emmanuel Bourg [1] https://issues.apache.org/jira/browse/MJAVADOC-504 [2] https://issues.apache.org/jira/browse/MPLUGIN-339 [3] https://issues.apache.org/jira/browse/SUREFIRE-1422 [4] https://salsa.debian.org/java-team/maven/blob/master/debian/patches/slf4j-compatibility.patch [5] https://github.com/apache/maven/pull/118 - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: can we remove ant build.xml in maven core?
Hi, I'm co-maintaining the Maven package in Debian and I'd like to mention that the Ant bootstrap is also used in the build process. I'd be very grateful if you could keep maintaining it, the Maven packaging is already quite difficult (we are blocked on the 3.0.5- 3.1.1 upgrade) and removing the Ant script is likely to be disrupting. Emmanuel Bourg Le 26/02/2015 19:15, Igor Fedorenko a écrit : Sorry if this is a dup of a recent discussion, but I couldn't find anything so here it goes. Why do we use ant to run Maven CI build at https://builds.apache.org? Seems odd we need to maintain both maven and ant builds for maven core. I am quite certain maven is mature enough to bootstrap itself without problems. -- Regards, Igor - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
