Re: Evolving the POM format

[Robert Scholte]

Op Thu, 19 Jun 2014 18:07:33 +0200 schreef Stephen Connolly  
<stephen.alan.conno...@gmail.com>:

Yeah I thought about that... and I don't like it.
First off pom reading is not a pure java world. The ship has sailed.  
There
are people parsing the pom using Ruby, people parsing the pom using other
languages that don't even run on the JVM. Thus the only common  
denominator
is to provide an XSLT transformation to map the newer XML format down to
the older XML format.

Publishing all formats is just ridiculous.

My proposal, and I have not seen a better one, is to limit ourselves to  
at
most three poms:

1. The 4.0.0 flattened consumer pom
2. The post 4.0.0 flattened consumer pom
3. The build pom


The flatten-maven-plugin is a very nice experiment which taught me that  
some repository managers require extra information inside the pom.xml  
beside the consuming parts. So apart from the dependencies it expects  
developers, scm and a few other elements. We have to be sure that this  
consumer pom is accepted by all repo managers one way or another.
For 2 it is only useful if frameworks like Aether want it and will use it.  
I think they should be in the lead for the format so it is optimized for  
them. You might even wonder if it is Maven Core who should create it. I  
can imagine that either the repo managers extract that information from  
the artifact and generates the consumer pom OR that the  
maven-deploy-plugin adds it, so you're not bound to the Maven version to  
generate such files.

Robert

So for 1 we know what that will look like. We just change the rules so  
that
it is always flattened, it never has a parent, we strip out some things,
we're explicit on dependency exclusions, etc.  In other words it makes it
as simple as possible for any pom 4.0.0 parser to construct the  
transitive
dependency tree that we believe it should have. All other use is
unnecessary. We don't publish a 4.0.0 consumer pom with
<packaging>pom</packaging> because

* they are unnecessary for other 4.0.0 consumer poms as none of them will
reference a parent
* it removes the problem of people wanting to use the 4.0.0 consumer poms
as a parent, and we are setting the post-modelVersion-4.0.0 rule as your
parent must be of a modelVersion <= the child's modelVersion

For 3, we have no fucking clue what that will look like. I think we need  
to
agree that the first line of it needs some hint as to the model  
version...
but if we stick to the rule that you need a Maven version that is >= the
model version of the pom you are building to build... well we can always
just try all the parsers we have until one of them works... and if it  
don't
work... well we can safely bomb out as you are trying to build with a
version of Maven that is too old.

Now we have the hard problem, what does 2 look like?

Well 2 is produced and consumed by machines only. So I think it will be
either json or xml. Both are problematic formats for people to write
correctly, but are simple for machines to create and parse. YAML is out  
in
my view as it becomes too tempting for people to try and hand craft...  
also
have you seen the YAML spec?

If we go for XML, we need to use namespaces and define up front what to  
do
with unknown namespaces.

If we go for JSON, we need to define the extension mechanism and how to
handle unknown extensions.

Perhaps the simplest thing to do is to have extensions include a flag  
that
says understanding them is mandatory or not.

In any case this post 4.0.0 flattened consumer pom only has to worry  
about
expressing dependencies and dependency like constructs (such as
platforms/runtimes and "provides" information).

Which poms we deploy depends on the packaging:

packaging == pom: deploy build pom only
packaging != pom : deploy 4.0.0 consumer pom and post 4.0.0 consumer poms
only

That is my vision... the bit that I feel worried about is ensuring that  
we
get the post-4.0.0 consumer pom flexible enough that it can handle future
stuff that we have not anticipated... as if we get that wrong then 2-3
years down the road we end up with a 4.0.0 consumer pom, a 5.0.0 consumer
pom and a post-5.0.0 consumer pom. I would hate to see us deploying 15
different consumer poms just to publish a 1k utility jar


On 19 June 2014 16:49, Paul Benedict <pbened...@apache.org> wrote:

I definitely agree that there should be two kinds of POMs: building and
consuming. Those are two separate concerns and there's no reason to  
pollute
dependency consumption with build information (or site information!). I  
am
on board with separating the two out. When it comes to evolving the
building POM, that's clear cut: upgrade to the Maven X.Y that can read  
that
kind of POM. However, you're still "stuck" (for a lack of better word)  
with
evolving the POM that everyone consumes, as I said, due to (i) how far  
back
in time you want to support and (ii) publishing all the formats that  
could
exist.

Here is my armchair idea to give for consideration:

1) The POM reading code is refactored out of core and becomes a plugin  
that
runs in a new phase at the very beginning (before "validate"). Either  
the
plugin supports all POM versions, there exists a plugin per specific POM
version, or it's configurable which versions you want to read.

2) At publish to repo time, for whatever plugin actually generate the
"consuming" POM, the builder configures how many POM versions he wants  
to
publish: 3.x, 4.0, 4.1, etc.

My idea is not to force either party into saying "I must read or write
umpteen formats". Leave it all to the builder and consumer to decide  
which
are important to each respective use cases.

Paul



Cheers,
Paul


On Thu, Jun 19, 2014 at 10:32 AM, Jason van Zyl <ja...@takari.io> wrote:

>
> On Jun 19, 2014, at 11:24 AM, Paul Benedict <pbened...@apache.org>
wrote:
>
> > I am curious why you interoperability as a requirement? Perhaps
> questioning
> > that may seem outrageous, but I see no problem with saying that you
need
> to
> > upgrade to Maven X.Y to read newer POM formats.
>
> To build a project that is certainly fine. If a project starts using a
> Ruby-based POM you will need a newer version of Maven to build the
project.
> The issue of interoperability arises when projects that want to use  
what
> was built by the project using the Ruby-based POM. Say the Guice  
project
> starts using a Ruby-based POM: you do not want to force everyone who  
uses
> Guice to use a version of Maven that has the capability to read a
> Ruby-based POM. I think it would be an enormously limiting move to
require
> that. Build with whatever you want but let's make it easy to consume  
for
> everyone which means making those dependencies usable by all the  
existing
> versions of Maven. The build-time information is mostly, if not
completely,
> irrelevant at consumption time.
>
> > If a vendor wants to
> > backport their project into older POM formats, that should be the
> vendor's
> > shoulders and not be a concern of Maven core. If Maven does publish
older
> > formats of POMs, you then have to decide how many older formats do  
you
> want
> > to publish? One day there will be a 4.1 or 5.0, and it's going to
> > complicate the world if Maven takes on the burden of  
interoperability.
> >
> >
> > Cheers,
> > Paul
> >
> >
> > On Thu, Jun 19, 2014 at 9:57 AM, Stephen Connolly <
> > stephen.alan.conno...@gmail.com> wrote:
> >
> >> On 19 June 2014 15:48, Igor Fedorenko <i...@ifedorenko.com> wrote:
> >>
> >>>
> >>>
> >>> On 2014-06-19, 10:30, Stephen Connolly wrote:
> >>>
> >>>> - Igor is*mostly*  right in that we should not deploy the pom  
that
is
> >> used
> >>>>
> >>>> to build to the repository...
> >>>> - Where Igor is wrong is that for <packaging>pom</packaging> we
should
> >>>> actually deploy the build time pom to the repository... probably
with
> >> the
> >>>> classifier `build`... this is safe as `pom` does cannot have a
> >> classifier
> >>>> in model version 4.0.0.
> >>>> - You couple that with a simple and obvious restriction... your
parent
> >>>> must
> >>>> be the same or earlier version of Maven. You cannot have as a
parent a
> >>>> newer version of Maven than the child is built with.
> >>>>
> >>>
> >>> I think there is more to this.
> >>>
> >>> At very least we need to decide what to do with <parent> in 4.0.0
> >>> compatible poms. Maybe we should always deploy effective pom with
> >>> build-related elements removed.
> >>>
> >>
> >> Well I think the simple thing is that the 4.0.0 pom is fully  
resolved
> when
> >> you have a builder pom
> >>
> >>
> >>>
> >>> I am also not sure if it is enough to deploy "build" parent poms  
as
is.
> >>> Your suggested "parent must be the same or earlier version of  
Maven"
> >>> implies new versions of Maven can read older pom formats, which I
think
> >>> will significantly limit our flexibility to evolve pom format.
> >>
> >>
> >> They need to be able to parse it into the model. Perhaps they do  
the
> >> parsing by downloading a parser. But I think it is reasonable to
expect
> >> that we can convert older build pom formats into newer ones and  
just
let
> >> Maven take care of it... if we cannot do that then we are really
> creating a
> >> brand new build tool rather than evolving Maven
> >>
> >>
> >>> I wonder
> >>> if we can have different solution for force parent poms, like
> >>> org.apache:apache, which are used by multiple projects and  
different
> >>> versions of maven and project-specific parent poms, where it is  
much
> >>> easier to require specific version of maven.
> >>>
> >>
> >> Well the simple way is we have deployed the 4.0.0 parent pom as
> >> org.apache:apache:14:pom if you want to upgrade your parent to
> >> org.apache:apache:15 then you will need to require Maven 4.0+ to
build.
> If
> >> you don't want to upgrade your build time requirements, then don't
> upgrade
> >> the parent... and if we need to "fix" things in the parent for  
4.0.0
> >> consumers, we can always deploy org.apache:apache:14.1 as a newer
4.0.0
> >> model pom
> >>
> >>
> >>> --
> >>> Regards,
> >>> Igor
> >>>
> >>>
> >>>  
---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> >>> For additional commands, e-mail: dev-h...@maven.apache.org
> >>>
> >>>
> >>
>
> Thanks,
>
> Jason
>
> ----------------------------------------------------------
> Jason van Zyl
> Founder,  Apache Maven
> http://twitter.com/jvanzyl
> http://twitter.com/takari_io
> ---------------------------------------------------------
>
> Selfish deeds are the shortest path to self destruction.
>
>  -- The Seven Samuari, Akira Kurosawa
>
>
>
>
>
>
>
>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org