On Dec 16, 2006, at 5:51 AM, Jason van Zyl wrote:
The central repository itself has always been pretty stable with no
safeguards. I realize not having these safeguards is not great but
things don't just disappear off that machine. We have a huge
problem, it appears, with the syncs we are pulling in
automatically. Organization wide syncs are soon going to stop and
it's going to be per-project so that when garbage appears we will
know immediately who's polluting the repository, Archiva will also
keep track of deletions. So yes, I agree on one hand that we need a
watchdog in place but we are not randomly jumbling stuff around on
the central repository. We're getting burned from our source syncs
and the misuse of SNAPSHOT repositories for the most part.
Put the contents of central under svn, and expose the change log...
that would be a good start. That will help reduce the window of
error, though it does not completely eliminate it. If you also allow
the central repo to be pulled via svn's versioned dav protocol, then
you can also eliminate problems when an artifact is being changed...
so that when the build runs, it will always use the same rev# when
retrieving artifacts (picking whatever is the latest at the time when
its initialized and keeping that for the entire cycle).
I think that all mvn repos should behave like this actually... and if
they did I would have much less to complain about, since that would
provide a rich audit trail and atomic access to artifacts.
Only thing left would be to allow projects to add specific tags to
those artifacts on central during release time, so that at any point,
a project could always get the complete set of artifacts it needed to
build.
Another comment I will make is that I am fairly sure there are
severe bugs in the maven artifact resolution process when
snapshots are present.
There are a huge number, I believe it's completely unreliable and
it's going to need an overhaul. It was very apparent from my last
round of travels that in many cases especially when snapshots are
used there are severe problems. I think we underestimated the use
snapshots and how prevalent their use would be for external
dependencies.
Ya, maybe... I think almost everyone is using them for development,
and most people who are working with other groups to get changes made
are pulling in those snapshots.
This appears to be the recommended way to handle these types of
dependencies which change rapidly using Maven. IMO its lossy and
harmful in many cases... and that is why I prefer using source repos
to handle this, so that I can always get a snapshot of the source
code, build it and integration. I believe mvn's snapshot's are
really only a convenience for folks that don't want to build a bunch
of stuff, but are willing to live with the possibility that they are
not using the latest version of the codebase.
But, personally... and specifically for build automation, I would
much rather work directly from source code so that I was 100% sure
what was getting pulled in, and have a changelog to correlate with.
And... well, currently Maven does more to get in the way to implement
that solution than it does to help get there.
This is because if I remove all org.apache.geronimo.modules
artifacts from my local repo and build the corresponding part of
geronimo, if I build online I usually get errors together with
downloaded timestamped artifacts whereas if I build offline the
build succeeds.
Yup, that's a patch we applied for Jason to provide a stopgap
solution. Where no snapshots will be updated when building.
Ya, thanks by the way. Not sure when I will actually get to be able
to use that though... I don't think I want to recommend that anyone
use an pre-release of Maven to get around issues with the current
version.
Note carefully that I am only building geronimo artifacts and
there is no change whatsoever in non-geronimo artifacts in my
local repo. I think nearly every time we've made a change
involving more than one module since we started using m2 and
pushing snapshots to the snapshot repo we've had user complaints
that the build is broken, and the solution always is to build
offline.
Snapshots are an inherit instability but there are definitely error
in working with snapshots in maven-artifact and it's bad. I see it
as the most critical problem with 2.0.x. But moving toward using
less of them even if that's locking to some timestamped versions
will help greatly.
Timestampped versions cause enough problems by themselves... and I
don't recommend using them directly.
The thing is that... we need to have a simple mechanism to easily
pick up changes made by other dependency projects with out needing
them to make a release and/or changing our poms to pick up new bits.
It would be a PITA if everytime a change was made to OpenEJB to
support Geronimo that they had to make a release and we had to modify
poms to use a new version. That could potentially happen 3 or 4
times a day, if not more.
SNAPSHOTS here are very useful for developers who don't want to
checkout everything and build it, who are willing to assume some risk
of strange build failures due to slop.
Your complaints about any already released geronimo artifacts are
totally irrelevant unless you want to recommend we move back to
m1 since the 1.2-beta and 2.0-M1 are the first releases we've
tried to do with m2 (except for specs, which got messed up in
various other ways but have not been a giant problem until recently).
With m1 or m2 a release with snapshots is deadly. The practice
seems to be something present regardless of what version of Maven
you're using. The concept of a SNAPSHOT is the same in both
versions though implemented differently.
Even in the face of the instability with SNAPSHOT handling in m2 I
think you can eliminate a lot of it by getting off many of your
SNAPSHOTs and I am trying to get out 2.0.5 which now contains a fix
that always takes SNAPSHOTs locally if you have them.
I agree... best to limit which SNAPSHOTS are used... BUT, its really
not that easy to just go and get a project to make a release so that
you can move forward. So I think that there will always be SNAPSHOTS
used in this manner.
--jason
