On 16 Dec 06, at 12:10 AM 16 Dec 06, Jason Dillon wrote:
On Dec 15, 2006, at 7:51 PM, Jason van Zyl wrote:
IMO the remote repos are for user connivence *ONLY*,
That is horse shit. What they contain is a product of what you
people put in them. You guys for example have something that is
tagged, which you released and which is not reproducible. That's
not Maven's fault, that's your fault. You are hosing anyone trying
to consume your stuff. And if people are doing that to you then
your should irate like anyone should be who looks at your tag for
1.1.1. That's why the release plugin behaves the way it does. It
screamed at you when you tried to release that 1.1.1 tag I bet and
you side stepped what you should have done and did something
manually.
First off... this is *my opinion*... not sure how you can jump to
the conclusion that my oppinon is horse shirt... or any mammal shit
for that matter. But I have been know to say that mvn is crap many
times before... so if you feel better stating that my opinion is
shit... well, then go for it.
Second, what assurance does any project have any any given artifact
in the central repo will remain there asis for the foreseeable future?
It's always been our policy that artifacts that we place in the
repository are not removed. From syncing partners like Codehaus,
Apache, ObjectWeb, or Mortbay they have to tell us that they do not
want us to match the source they give us. We have never culled the
repository.
There are are already situations where bits have been added and
removed (at least one which I requested to get around a sync
problem from ASF) and a few others which I have heard about through
others.
Only if the repository is diddled on the source side. We cannot
control in all places what is done. You guys seem to manually delete
artifacts from this side which wreaks havoc.
There is no audit trail for central or any other popular mvn repo,
so any release manager with half their wits is going to think twice
about trusting any content which is pulled from it.
Two separate issues. People behind firewalls generally do not and
cannot use the central repository directly but they manage ones that
do rely on central. And they can for releases because we do not
delete anything. Everything release be signed and that will get
easier to do automatically so though it's not fully automated to
check the signatures you can verify what you have. We will have a
transition period where unsigned artifacts will be accepted but
shortly anything coming in by any means will have to be signed and
the metadata will carry with it a reference to its source.
one critical error I believe that everyone is making is thinking
that deployed artifacts are permanent...
Deployed releases that we place in the repository are most
certainly permanent.
What assurance do I have that artifacts are going to exist in the
repo in 10 years? 20?
That is our policy and we will bolster our infrastructure but Ibiblio
gives us the assurance that what we give them stays in perpetuity.
That's their policy. That's why we still mirror things there even
though we are using our own dedicated box. I also have backups from
day one that are stored in an offsite along with my other backups.
How about the assurance that they have not been tampered with?
Signatures that you are supposed to use for releases here since
always. We don't enforce that yet but anything coming from Apache
should have them.
Digest files don't do jack... as if someone alters an artifact, and
I download it into an empty repo... then it all looks fine to the
consumer.... and its highly regular for users to nuke their repo
when problems happen... and problems are regular too.
It will not be long before it will simply be mandatory to have a PGP
key sign your goods if they are going to make it into the repository.
Snapshots can be transient depending on the policy of the
repository. What I've been told is that infrastructure is telling
you to remove old versions from the repository at Apache which can
have disasterous effects. Your artifacts deployed to central
should not disappear so I'll go make sure that we're not deleting
files that have been removed from the source. But we generally
assume when we are syncing from a source that the source knows
what it's doing. If you delete stuff on this end then we expect
you to understand what effect that will have on your clients.
Many users may understand, but there is always one that will not,
which throws off the entire system.
Which users? You as a user supplier of artifacts populating your
repository?
If one weak link decided to re-release version 2.0 of something
that effectively breaks compatibility with consumers of the
previous 2.0 release, then then entire system is compromised... who
is to say that any other artifact might just change under the covers.
There are lots of the issues we know about and are fixing, lots of
stuff in the wiki about maven-artifact changes in 2.1. You're not
saying anything yet we haven't heard before. That's not really
germane to this your issues anyway. You can't even get non-SNAPSHOT
so it wouldn't matter if it was secure, it would still be useless.
Granted that does not happen often, but it does happen, and will
most certainly continue to happen.
It's happened once that was very noticeable which was the overlay of
a binary incompatible version of commons-logging 1.0 was laid down.
Using the release plugin helps and when we make a deployment
transactional we can enforce not being allowed to deploy the same
version again. So it's not like we're sitting on our laurels, we've
seen these issues and over time we add measures.
Maybe its possible to remove the chance of it happening from
central, but projects do not just depend upon central... its easy
enough to setup a repo, then include that into your project. And
the managers of that repo may remove add/change artifacts at will.
So my comment about the transitive nature of all mvn repos is much
more general... and certainly not mean to know mvn, but more as a
warning that artifacts on remote repos are much more transient that
many people (like you) believe they are.
Then don't use those repos, or label them as snapshot repos. As far
as Geronimo is concerned why do you need anything more then central
as a source? Aside from your SNAPSHOT dependencies.
This will only stop when Archiva is in full effect. The only way to
submit anything to central will be via Archiva. Any project who
wishes to have the same stability will only take artifacts that have
passed through and instance of Archiva. You'll know you're using an
instance of Archiva because we'll have a wagon for doing that and it
will be configured. It will eventually be the default. It will simply
be the Grizzly client and Jetty using the Grizzly connector.
which they are most certainly not, so we should generally always
build from source to ensure that everything is being included at
the right version.
No you shouldn't. That defeats the entire purpose of Maven.
Certainly not Jason...
Yes, building from source when not required defeats the purpose of
Maven. It is generally not required.
and I'm surprised to hear this from you as I have heard you talk
about the plug-ability of build functionality as much more
important to the purpose of maven than the artifact remoting. IMO
the plugin framework of mvn is much more important to the purpose
of mvn than remote artifact handling... and more so the lossy
artifact handling is more of a detriment to mvn than anything
else. I'd like to throw the remote artifact capabilities into the
trash... and then we'd find a reliable and trustworthy build
platform in mvn.
The remote artifact handling is one of Maven's greatest strengths. It
relies on the integrity of the repository and when everyone expect
us, the Maven developers, to clean up their shit it's not going to
work. If you don't want to play nice with everyone else then stick
your repository in SVN and be cut off from everyone else. When the
central repository gets more robust through the use of Archiva it
will become an invaluable resource. it's far more important then you
think.
Projects are not accountable for stuff they put in repositories is
what make the public space unreliable in its default mode. Maven
entirely reliable when the repositories you use are and many people
do maintain their own repos. That is the norm for corporate usage. If
you also really really want to you can use the SCM Wagon which works
off an SCM. The unreliability factor is almost always the over use of
SNAPSHOTs and people being cavalier with repositories. Maven will
only be able to improve this over time when the repository management
tools come into effect. It's not a small problem to try and make this
all work and unfortunately we've determined people are too negligent,
lazy, and don't think it's important to maintain the repositories
even though they have become a critical resource.
I can't even count the number of times that mvn builds have broken
due to external dependency changes... no local sources changes, but
come in to the office the next day and the build is just fucked.
SNAPSHOTs. Nothing with a SNAPSHOT is reliable. And by that the
"SNAPSHOT" identifier. It is only an identifier, on the remote side
there are no artifacts with "SNAPSHOT" in them. That identifier maps
to a timestamped version. So if you wanted reliability from an ever
changing mass, once you had a build that worked you could write a
plugin that passed over them all, looked up the last timestamp from
the metadata in the repository and plug those in. You have zero
guarantee of a working build with a SNAPSHOT. None. Never have, never
will.
You have unmaintainable and non-reproducible builds because you
have such a massive number of SNAPSHOTs and you never make any
incremental releases. What do you expect when you're building upon
sand?
Ya could be... mvn's snapshot mechanism is more than enough rope
for a team to hang them selves 10x over. Is that the teams fault
or mvn's fault for handing over the tools to hang themselves?
That's your fault, SNAPSHOTs mean unstable by their very definition.
You don't have to use SNAPSHOTs it's a convenience for change
__within__ your project.
Probably a mix of the two. Which I why I tend to try to limit
the usage of that garbage... but since its in mvn, and other folks
just assume that if its there it must be good, they tend to use it...
I don't who told you to use SNAPSHOTs but they are bad news when you
are using an external project SNAPSHOTs. You have squat in terms of
reliability. Use all non SNAPSHOT versions, and central and if you
have a breakage it's guaranteed to be in your project. Period. Can't
be anything else. The only time really bad things have happened is
the accidental commons logging thing and when we have failures at
places like Codehaus, and Ibiblio. We've moved to fix that by using
Contegix, trying out S3 and trying to get more mirrors. Again, we're
not resting on our laurels.
but more its also that g is a different type of project that has a
bunch of dependencies on external projects, which are moving at
similar (if not faster) speeds... and IIUC this is what SNAPSHOT
artifacts are meant to address, though I think that the actually
implemented, that with inconsistent deployment of snaps, has
completely failed as a solution.
Then you're really one project if you're so highly coupled and maybe
you should just put it all together. I think that's a clear
indication of a problem.
for example, G server depends on OpenEJB... but who knows when the
latest snap was deployed for OpenEJB, so to ensure that your G
server build actually works you *MUST* build OpenEJB from source.
No it doesn't. Once you have a build that works, you can use the
timestamp as the version and stop letting it swing. Ask OpenEJB to
deploy everything to one repository and have no separate SNAPSHOT
repository so that they won't be nuked. If you cannot stop using a
SNAPSHOT something is wrong.
This is not a mvn problem by itself, but a process problem for not
getting regular snaps deployed. But assuming that there was a
nightly deploy of all that stuff, then there is still a window
during the day when the code could have changed in OpenEJB and G
server which would not be picked up due to the SNAPSHOT not being
deployed until hours later...
Absolutely, don't use them all the time. They have to do this
periodically to give you something stable.
and then even once it was deployed, the default policy of waiting
daily before updating snapshots... basically makes a huge window of
times when a user might get the wrong code and end up with a failed
build.
THE ONLY WAY TO SOLVE THIS IS TO BUILD IT LOCALLY.
No it's not. I think you have a fundamental misunderstanding about
how versioning works in Maven. At least in Maven 2.x. OpenEJB can
deploy as many snapshots as they like, you have found one to work.
You have found a set of snapshots to work then lock them down.
This only happens for a small number of deps, where there is tight
coupling... but it does happen... and since G server and OpenEJB
are so intimate with each other, we run into this all of the time.
And there really is no solution to solve the problem, short of
decoupling them, or building from source.
And since the chances of these being decoupled anytime soon is slim
to none... we really have no choice but to build openejb2 as a
dependency.
That's simply nuts. That you have to do that means you're screwed.
That you, at no point in time, can get a stable release is not good.
Everyone needs to across your project must settle on a version as the
sign post.
Get the producers of your artifacts to release things more
frequently, this ensures that SNAPSHOTs are removed from the
chain. I look in your server build and you've got SNAPSHOTS
**all** over the place. OpenEJB, ActiveMQ, Tranql, your spec JARs.
You control, or have relationships with most of these projects.
Give me a break dude... we have relationships with Maven too... but
have had little to no luck in getting some critical bugs fixed
which affect our builds, like how we have to split up our build
into two stages to avoid the bug of extensions that are built in
the same cycle from being picked up.
Like the one Brett applied last week for you? LIke the RAR plugin
that was put into the cycle for you? Like the XMLBeans plugin I have
tried to get released for you? LIke the plugins I'm making to try and
make staging easier and licensing easier that Dain used yesterday to
make your release?. I took your GPG plugin and integrated it into
Maven plus I field the bitching and fix shit for people all the time.
It doesn't get all fixed at once. But I deal with this shit all time
where Maven is getting pegged for shit that the tool is not
responsible for. There are lots of bugs because we have tons of users
and they find shit all the time. I could give a rat's ass what anyone
thinks because we're honestly can't do anymore then we already are.
The same shit happens for other projects too. G depends on a lot
of other projects, and its non feasible to wait for our dependency
projects to deploy the right versions for use to move forward all
the time.... otherwise we would be waiting forever for other groups
to move.
A timestamp will do as a version. Always has in Maven 2.x. You don't
need a final release. You need something that will not change out
from underneath you which a SNAPSHOT is meant to do. So you have:
http://snapshots.repository.codehaus.org/org/codehaus/swizzle/swizzle-
jira/1.3.1-SNAPSHOT/
Now if you look in:
http://snapshots.repository.codehaus.org/org/codehaus/swizzle/swizzle-
jira/1.3.1-SNAPSHOT/maven-metadata.xml
You will see that 20061122.035631 was the actual name of the JAR that
corresponds to the latest SNAPSHOT. That will work as a version that
will not change. For all intents and purposes that SNAPSHOT should
stick around long enough for you to get a release.
For example... if we wanted for your team to fix MNG-1911 before we
finished our m2 conversion... well, we'd still be using m1 today...
and probably for the foreseeable future.
Well, what does any software project do given many demands and
limited resources? It's got 0 votes. Scream louder. I use them to
gauge what I do in plugins:
http://repo1.maven.org/reports/plugins/plugin-issues.txt
What else are we supposed to do with the unsatisfied multitudes?
That... and the fact that mvn releases each damn little part of a
project severally forces each of these projects to go through legal
hell to make sure that everything is up to policy with legal files
etc... which normally would only have applied to source files and
to the finally distribution.... but mvns opened that up to include
almost every single damn file that is part o a build... which is
ridiculous.
Modularization is a good thing. Crappy problems like the legal
problem can be solved when the group of Maven users needs something.
That crappy legal problem has been fixed and we're testing it and it
worked for your release yesterday. Dain used it and you don't have to
copy any legal crap into every JAR. It's done transparently now, or
will be when you inherit from the ASF POM:
http://idisk.maven.org/jvanzyl/Public/RemoteResourcesHandling.png
That one I listened to and fixed.
You don't wait until it's time to release to get the producers of
your dependencies to crap out some junk you can use. Get them
release, lock your stuff down to it, move on. Repeat. You should
have no SNAPSHOTs other then ones for your own projects in your
POMs. If everything is changing everywhere it's total insanity
This is not always a possibility... if any of these groups releases
there goods a slow or you our we do, then nothing is going to
move. Using SNAPSHOTS, or building from source is really the only
thing that is allowing us to move forward, because there is so many
changes going into each of these components which are required for
G to function. Its simply not possible to lock step our progress
with the release schedule of other projects, we must for
development take incremental snapshots of code.
That's fine but you still can lock down on a timestamp. You have to
at some point draw the line. The second you lock one thing it forces
others to lock down. You declare that for this week using this
version of OpenEJB, everyone must follow along and you prepare for
the following week to use the SNAPSHOT built on X day. Everyone
trying to look at sources to match signatures is simply not scalable.
Locking down to a version, even if it is a timestamp, is your way of
communicating "you're going to have to coordinate with others to
align things and live with with for the week". Things cannot change
so radically that everything breaks. People have to use feature
branches or be more careful about what they are expelling. Additions
should not affect consumers of artifacts, and if it is only additions
(which I assume because you're all following JEE standards) then you
have to plan that better for consumption. If people are breaking
binary compatibility five times a week they should be tarred and
feathered:
http://wiki.eclipse.org/index.php/Evolving_Java-based_APIs
You can use CLIRR to nail offenders:
http://clirr.sourceforge.net/
And look at that ugly Maven site and you know there's a plugin.
Build everything from source is not something you should be doing
because you should not have to. It is one of the most obvious
signifiers that you are doing something very wrong.
I do not meant to build everything from source... though if I did
feel that was needed, it is most certainly not a sign of doing
something wrong. If I wanted to know exactly what changes went
into the system... or wanted to know what changes affected which
tests, then that is really the only way to tell.
Again it's not. You have crappy releases and crappy planning. You
have too much change to absorb reliably at once and is unsustainable.
I'm not suggesting that we want to build commons-logging or howl
all by hand... but for the projects that change the most (those
which we are using SNAPSHOTS for), then for the automated scenario
is it ideal ti build from source, and eliminate any guess work that
might come in to play when using the lossy mvn SNAPSHOT downloading
mechanism.
You're using the SNAPSHOT mechanism incorrectly. It's for internal
use on a project and external dependencies a rarity.
This is especially key, when build take a while to perform, and
during the time it takes to run a longer build that one or more
snapshot builds of a dependency are created.... unless the
automation system actually tracks the exact artifacts produced and
hands them to the target build, there is no way that any
correlation from build to build can be done. This is a CRITICAL
flaw in the mvn SNAPSHOT mechanism.
No it's not you use it incorrectly. You always have the exact version
number for a SNAPSHOT. Always.
We don't need to guess what changes were include in a build... but
for large projects and distributed builds... mvn offers no
assurance to what artifacts are actually used.
You can always resolve the exact version. You guys have to many which
would make it annoying to look in the repo to find them. But we could
easily make a tool.
You have way too many points of variation and all your projects
seem to be in constant swing with one another. If that's the case
it begs the question whether they really separate and why not just
lump them all together if they are so coupled together. Which
generally the case when you cannot eliminate SNAPSHOTs from your
build. When you see SNAPSHOT not from your project in our POM it
is a warning sign.
Dunno about that... the most tightly coupled SNAPSHOT we have now
is OpenEJB... and really, I think that its so tightly coupled that
it really should be in the same codebase. BUT, that is not what
other key plays believe, so then we are left trying to work the
build system to generate something that is predictable given the
circumstances. Unfortunatly, mvn does not have a good answer for
this problem... all it has is SNAPSHOT... and IMO that is not worth
a damn.
Not the way you use it because that's not what it was intended for.
Its more trouble than it is worth... and more so since all you
people are using it everywhere, its very difficult to educate
people to its harm.
I don't think people use it the way you do. Most people seem to
understand it's for dealing with variation on the project you are
working on. And that SNAPSHOTs typically occur within your own reactor.
You guys seem to want to use the release plugin for releasing
things with SNAPSHOTs in them. That's not what it was made for and
again it's a function of you not managing your relationships with
your dependent projects. A release with SNAPSHOTs is not a
release. Tagging something that contains a POM with a SNAPSHOT
identifier is completely useless. I mean look at your 1.1 release:
http://svn.apache.org/repos/asf/geronimo/server/tags/1.1.1/
That's insane. That's a tag! How did you do a release with
SNAPSHOTs? That's not Maven's fault if you can't built that again
if you were careless enough to tag something with SNAPSHOTs. You
have got to use releases of plugins, parent POMs, and dependences
more frequently. Maven has lots of bugs, sure, but this mess is
largely your own doing. There are lots of Maven users with way
more source code and modules then you and don't have nearly the
number of problems you do.
I had nothing to do with any of that release... I have been trying
to unfuck a bunch of issues with the release process in general....
and for the record its not all mvn related stuff that needs
unfucking. Its more policy on how one uses mvn that needs work...
some issues with the default plugins, some bugs, but more than
anything it more the policy about how one uses mvn to build/release
projects rather than mvn itself.
The stuff done of late for resources will help everyone at Apache.
The legal and signing nonsense will be taken care and I will take
measures to clean up the repository fuckups.
I really feel that the little short-cuts and side-steps around
maven problems are going to kill us. And really, I don't see how
any medium to large sized group can effectively use maven to
manage releases of their projects...
Lots of projects do it fine. Your dependency chain is fragile
which is the problem. Your coupling is too high which will not let
you do releases as you have to, or you are not working with the
producers of your dependencies to create releases. In trunk right
now you have a SNAPSHOT for the POM that seems to contain the
dependencies for everything you need. That's wrong. You need to
release those parent POMs and the dependencies in them. It is
natural to have SNAPSHOTs for the project in question, for CI and
while you're developing. But a SNAPSHOT in a dependency means that
you, as a last resort, had to use something in flux because it
cannot work otherwise. This immediately should signal you to
contact that project to fix it, and release it. And again, because
all your dependent projects seem to share many of the developers
you should be able to do that.
Lots of projects do fine... sure. Smaller projects with less
requirement for durable/repeatable and stable builds should work
fine. But larger projects, with more moving parts that need to
have builds repeatable over 10 years+ (with no changes to tags)
will almost certainly fall flat on their faces when using mvn...
UNLESS certain precautions and limitations are heeded.
I'm building several projects way bigger then yours for mission
critical systems that have governments involved if they don't work
and it builds fine, and it will build fine in 10 years.
The same is true for almost any other build tool... make and ant
both can end up in the same situation... BUT... the key difference
is that mvn makes it way to easy to get into a situation where your
builds are non-durable and non-stable right out of the box. Its
got instabliltiy and non-durability built right into it with its
remote repository... a problem which make and ant do not suffer
from... and a reason why ant is more prevalent in commercial shops
than mvn is.
That's just not true. You seem to fundamentally misunderstand the use
of a SNAPSHOT. And a build from 3 years ago running against Ibiblio
done as a proper release will build. In Norway more people use
Maven :-) And more and more people are using Maven. Our stats are
going through the roof and repository usage rate goes up all the
time. Corporate users also manage their own repositories so they
don't have any problems.
* * *
Anyways, I'm gonna stop here. I could certainly go on and respond
to everything in this mail... but I don't see the point.
My purpose was not to diss mvn here... but more so to suggest that
there are more effective ways to use mvn which fit better into our
integration model... there are better ways to use mvn ASIS with the
requirements/restrictions we have now.
But even more so... my point was... that changes that have been
made recently have really fucked over the work I have been doing to
produce automated reliable durable builds for our projects... and
yes, based on all those SNAPSHOTS of all those projects.
I had it working and it was working well... until the model of
versioning specs changed... more specifically since the specs/trunk
branch could not actually build and now it builds artifacts which
are not being using by anything... the bits that are needed are now
only available to build off of little spec module branches, which
is a huge pain to automate.
IMO the only problem specific to maven here.... is that this
versioning model was allowed by maven and even more so promoted by
mvn since that is how you folks manage your plugin modules. So we
have the lemmings following by example... right off the cliff.
And lastly... none of this is anything I consider mine... or my
own... its all junk that I have picked up and been trying to fix.
And at one point I did believe that mvn was going to help me do
that... in retrospect... Ant would have been a better tool.... less
bugs... less mystery... less time trying to figure out just what
the fuck broke.
I doubt it. When it came down to releases and updating artifacts and
trying to tie everything all together. Lots of people might have
problems but even now I bet there are people who could help with the
build if necessary. If you did it in Ant you would be the only person
who would know how it worked. You would get the nice Inner Platform
Effect with a build your size:
http://thedailywtf.com/forums/69415/ShowPost.aspx
I'd bet my life you would have more overall problems using Ant. Just
because you could get it to work doesn't mean it would scale or be
something anyone else could comprehend. It's probably already hard
enough with what you have.
I cant' even count the number of times I have had to nuke my local
repo, rebuld specs, openejb, g by hand to debug someone else's mvn
problem... even more prevalent is the number of times I have heard
of other people doing the same. We spend more time fucking with
mvn than we do actually writing code for Geronimo... and that is
certainly a sign that something is wrong.
Yes, there are a handful of problems with maven-artifact that cause
some grief. Can't fix everything all at once.
--jason
