Hello,
I talked with Maven developers on IRC today. I asked for the problem[1] we are currently facing. They give me interesting advices and
thoughts that may interest you. I paste cleaned up log file of the talk:
[15:32:18] <grek> hi, I'm Grzegorz Kossakowski, a member of PMC of Apache Cocoon
[15:33:07] <grek> I'm looking for help with Maven issues that seem to stop us
from releasing artifacts :(
[15:34:28] <grek> being more precise, we are affected by this issue very badly:
http://jira.codehaus.org/browse/MNG-2743
[15:35:14] <grek> Is there anyone eager to help?
[16:32:14] <wsmoak> grek: it's marked for 2.0.4. can you confirm it also
happens with 2.0.6 ?
[16:33:31] <wsmoak> and... after a quick read, can you duplicate
repository elements where you need them, so you can proceed with the release ?
[16:34:01] <grek> wsmoak: yes, it happens with 2.0.6
[16:35:22] <grek> wsmoak: yes, we can do this (and done it before) but we have
to ask our *users* to put repository elements in *their* poms
[16:35:46] <grek> I'll give you more links explaining the situation
[16:36:03] <wsmoak> so it's not just a problem at release time?
[16:36:45] <wsmoak> why can't the artifacts in that other repo be put in
the central repo?
[16:37:30] <grek> they are waiting in your JIRA ready for upload AFAIR ;)
^^^^^^^^^^^^^^^^^^ here I confused commons-jci with xreporter artifacts that
waited for upload ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[16:38:56] <grek> but the problem still persists, Cocoon has _lots_ of dependencies, some of the stuff is just in between of conversion to
the Maven
[16:39:05] <maxb> Isn't central supposed to be selfcontained anyway?
[16:39:30] <grek> what do you mean by self-contained?
[16:39:40] <wsmoak> ok. assuming no problems with the upload bundle and
carlos gets them into central, is there still a problem?
[16:39:48] <jvanzyl> if you mean a complete transitive closure, yes but we
don't have a way to check just yet
[16:40:50] <wsmoak> poms in the central repo shouldn't define additional
repositories
[16:41:14] <wsmoak> it makes users in corporate environments nervous to
see Maven trying to download from random places
[16:41:28] <grek> hmmm, good point
[16:41:47] <grek> let me explain our situation and policy at Apache Cocoon
[16:42:46] <jvanzyl> wsmoak: that would be a good rule to check for upload
bundles actually
[16:42:58] <jvanzyl> no external repository definitions
[16:43:03] * jvanzyl makes a note
[16:43:24] <grek> if we want release something as RC (or final) we have to make sure that we do not depend on snapshot versions and all our
dependencies are at central
[16:44:22] <grek> however, we also release milestones which can depend on snapshots and can define it's own repositories (for those
unreleased artifacts)
[16:45:39] <grek> and that's the case now, we want to release a milestone of our own Maven plug-in so users can test it and report back if
everything works
[16:45:50] <wsmoak> grek: "release" meaning you want these milestones in
the central repo?
[16:46:08] <grek> wsmoak: yes
[16:46:28] <wsmoak> then snapshots simply aren't allowed... you'll have
to release milestones of your dependencies.
[16:46:49] <wsmoak> but these aren't really "users" you're asking to help
test, are they? they're more involved than that.
[16:47:16] <wsmoak> what's wrong with staging it on people.apache.org/builds/cocoon and asking them to add a repository to their
settings.xml to test ?
[16:47:49] <grek> wsmoak: we have done exactly what you suggest
[16:48:08] <grek> wait a minute, I'll provide links
[16:48:48] <wsmoak> (btw, the Struts PMC has the same policy... snapshot
dependencies are allowed in alpha releases.)
[16:49:35] <grek> http://thread.gmane.org/gmane.text.xml.cocoon.devel/73469 and
my answers in this thread
[16:50:11] <grek> http://thread.gmane.org/gmane.text.xml.cocoon.devel/73495 <-
here is the user affected by the MNG-2743
[16:55:13] <wsmoak> ok. sounds like you're waiting on carlos to close a
MAVENUPLOAD issue.
[16:56:18] <BrianFox> grek: you can use the enforcer plugin to make sure
people are using 2.0.6 to build cocoon 2.2
[16:56:30] <wsmoak> Kenney's 2006-01-07 comment on COCOON-1975 sounds
right on (and that issue is closed).
[16:57:03] <grek> BrianFox: we use it already, but will it work if someone just
uses the artifacts but does not build them?
[16:58:26] <BrianFox> no. I just saw you comment in that thread asking
Joakim which version he used. With Enforcer you would know ;-)
[16:59:11] <BrianFox> the prerequisites would affect "users" of plugins if
that's what you mean
[17:00:31] <grek> we require Maven 2.0.6 not only for plug-ins/building but also when people just depend on our artifacts, it's due to
dependencyManagment fixed in 2.0.6
[17:00:56] <BrianFox> i see
[17:01:27] <wsmoak> I don't think you can force them to build their own
projects with 2.0.6.
[17:01:44] <wsmoak> you can keep them from using your plugins *unless*
they are on 2.0.6, with the prerequisite element.
[17:01:52] <BrianFox> right
[17:02:00] <grek> I see
[17:02:17] <grek> so we will have to put big, fat warnings everywhere to use
2.0.6
[17:02:33] <wsmoak> so at this point it's just education-- encourage them to upgrade to 2.0.6 and point them at the docs on how to prepare
for upgrade.
[17:03:07] <grek> yeah, but the original problem still persists I think
[17:03:37] <wsmoak> true, but it's not even marked with a fix-for
version, so I don't think waiting is really an option
[17:04:06] <grek> and I just realized that the dependency we are talking about here (commons-jci-fam) does not wait for upload, I confused
it with other artifacts waiting for upload
[17:05:40] <grek> wsmoak: I'm aware of that you'll not fix it tomorrow but we really have to find a solution, we have a lot of depdencencies
for which we have to prepare poms and upload them
[17:06:36] <grek> waiting for each upload is not an option for us when we talk
about milestone release, it happens to slowly I guess
[17:07:11] <wsmoak> are these your own artifacts we're talking about ?
[17:08:25] <grek> no, in this particular case it's a commons-jci-fam that is
not a part of Cocoon, obviously
[17:09:44] * wsmoak will be back in ~30 min
[17:24:48] <grek> after thinking about it for a while I'm starting to realize that you are right and we should really push other folks to
upload their artifacts to central
[17:29:02] <grek> on the other hand, the issue I'm constantly referring to does not allow us to make internal (not published on central)
alpha/milestone releases
[17:29:44] <grek> that could depend on unreleased stuff, of course
[17:32:14] <esm> grek: totally unrelated to your issue - i like your solution
using an alternate testing-cocoon-settings.xml file
[17:33:21] <grek> esm: thanks :)
[17:34:05] <esm> having uses rm -rf ~/.m2/repository doesn't make sense and
now I know a better way :-)
[17:36:09] <grek> esm: yeah, and if you work with mutliple projects you can test one with empty, separate local repository and after testing
continue the development of the another one without having to re-download the dependencies
[17:36:32] * esm nods
[...]
[17:44:22] <grek> wsmoak: can you give us an advice on making this internal
releases I talked above?
[17:46:12] <grek> in order to gather some early feedback from users we really need to make those releases that depend on snapshots or
released artifacts not available at central yet and I guess that we would be fine if Maven just been doing what we expect from him to do
[17:46:45] <wsmoak> grek: seems like you're mostly there. "release" your milestone, but deploy it to a staging area and have users add that
repo to their settings.xml
[17:47:29] <wsmoak> since you can't release with snapshot dependencies,
push the other projects to release to central.
[17:48:09] <grek> starts to make sense
[17:49:18] <grek> if so, why Maven allows to define repositories at pom level
if they do not work and are discouraged in a fact?
[17:58:55] <jvanzyl> a mistake that will be corrected in 2.1
[17:59:14] <jvanzyl> and you just won't be able to put them in a POM
[17:59:25] <jvanzyl> but that's a big change so we can't do it in 2.0.x
[17:59:30] <grek> ah I see
[17:59:37] <jvanzyl> but you can still abide by the practice of doing what
is in 2.1
[17:59:48] <jvanzyl> define all your repos up front in a shared
settings.xml
[18:00:08] <jvanzyl> so that your POMs don't change when you use different
repos underneath for whatever reason
[18:00:21] <jvanzyl> your infrastructure requirements should not be passed
on to your clients
[18:00:32] <grek> yes, I totally agree
[18:01:48] <grek> I'm only disappointed that you don't document this good practices and don't refer to them whenever someone has the problem
similar to our
[18:04:03] <grek> jvanzyl: one more thing if we have this shared settings file how we can tell Maven that it will should use that file while
building Cocoon?
[18:04:37] <grek> I guess that people will not be happy with -s option or
replacing their default settings file
[18:06:27] <jvanzyl> we could do something like map repository settings to
a groupId
[18:06:40] <jvanzyl> so you would have them in your settings for work on
cocoon, haven't quite figured it out
[18:06:46] <wsmoak> grek: I use different sets of repos by putting them
all in one settings.xml, inside profiles.
[18:07:00] <jvanzyl> so we'll try to make it convenient
[18:07:10] <jvanzyl> but defining repos in POMs was a definite mistake
[18:07:20] <kenney> maybe it's an idea to export a part of the settings,
like the profile, as a 'project settings' artifact
[18:07:59] <kenney> it would contain properties needed by the projects to build (like j2ee app locations or database info), and the
repositories required to build/use that project
[18:08:14] <grek> wsmoak: but then you still have to define which profile to
use with -P option, right?
[18:08:36] <kenney> grek: not necesarily, you can active a profile in any
pom
[18:08:36] <wsmoak> yes... these are your project developers we're
talking about, right ?
[18:08:39] <jvanzyl> yes, we'll make it easier
[18:08:57] <jvanzyl> no reason we can't generally map some settings for a
groupId
[18:09:01] <wsmoak> there are many ways to activate a profile, as long as
all of you agree on some conventions.
[18:09:11] <jvanzyl> so all your junk for org.apache.cocoon gets picked up
[18:10:09] <esm> grek: we use this at Pluto:
http://wiki.apache.org/portals/Pluto/Maven2Tips
[18:10:21] <esm> alluding to what wsmoak said
[18:10:31] <esm> just a profile devs activate to test builds, etc
[18:10:43] <grek> activating profiles that contain infrastructure info in poms does not seem to be a neat idea, still you have your poms
bloated with infrastructure stuff
[18:11:20] <esm> we ask the devs to modify ~/.m2/settings.xml instead of
putting stuff inpoms
[18:11:21] <grek> jvanzyl: what about simplest solution possible: searching for
settings.xml file along pom.xml ?
[18:11:52] <kenney> grek: there's already profiles.xml that serves that
purpose
[18:11:52] <jvanzyl> esm: right that's what generally happens
[18:12:30] <grek> kenney: any documentation?
[18:12:38] <jvanzyl> grek: i've really just started trying to collect
everything i've found working at big places with 500+ devs
[18:12:48] <kenney> grek: euh, maven.apache.org ? :)
[18:12:59] <jvanzyl> and i'm about to start another one and they've agreed
to let me publish the information
[18:13:25] <grek> kenney: I was asking if profiles.xml is somewhere documented,
but I hope it really is :)
[18:13:31] <kenney>
http://maven.apache.org/guides/introduction/introduction-to-profiles.html
[18:14:42] <grek> jvanzyl: does it mean that we'll see your conclusions on dev
list soon?
[18:15:13] <grek> kenney: thanks, starting to read this
[18:15:32] <jvanzyl> no, i'm just starting a project that will probably
last a year
[18:15:37] <jvanzyl> but as i go i will
[18:15:46] <jvanzyl> first deliverable is in 8 weeks so probably around
then
[18:16:06] <jvanzyl> but 2.1 will move along quickly after next week
[18:16:41] <grek> I'm getting lost here, do you want to say that you are going
to propose replacement for Maven?
[18:16:47] <jvanzyl> no
[18:17:13] <jvanzyl> this is a client project where i'm using maven
[18:17:18] <grek> ah, I see
[18:17:37] <grek> and why do you mean that 2.1 will move along quickly?
[18:18:43] <grek> from lurking at dev I had an impression that 2.1 is not going
to be released soon (I mean, a year period)
[18:19:16] <wsmoak> it can look that way, then jvanzyl stays up for three
days straight and suddenly all the bugs are closed ;)
[18:19:42] <grek> wsmoak: heheh ;)
[18:19:57] <jvanzyl> yah, it's not moved in a while as we try to bridge
2.0.x with 2.1.x so there's no shock
[18:20:03] <jvanzyl> all 2.0.x stuff will run in 2.1.x
[18:20:26] <jvanzyl> it's actually quite stable
[18:20:29] <wsmoak> ... except stuff that defines repositories in the pom
? if it's no longer allowed, how's that going to work?
[18:20:58] <jvanzyl> pom readers used keyed of the version of the POM
[18:21:09] <wsmoak> ah, the model version
[18:21:16] <jvanzyl> that will be harder
[18:21:19] <jvanzyl> the plugins are easier
[18:22:08] <grek> if I can have a huble request... could you write really good
docs explaining what and why was changed and how to migrate?
[18:23:07] <grek> Cocoon has suffered for long time because of lack of
migration guides and I think Maven should not repeat others' mistakes
[18:23:19] <jvanzyl>
http://docs.codehaus.org/display/MAVEN/Maven+2.1+Backward+Compatibility+Information
[18:23:52] <jvanzyl> all here:
[18:23:58] <jvanzyl> http://docs.codehaus.org/display/MAVEN/Home
[18:24:06] <jvanzyl> those top three is where you'll see everything
[18:25:06] <esm> btw the docs for upgrading to 2.0.6 were nice - the
dependencyManagment fixes
[18:25:16] <grek> great, I'm happy to hear these all good news
[18:25:20] <grek> yes, I agree with esm
[...]
[18:49:08] <grek> argh, I'm having some problems with internet connection
[18:49:29] <kenney> is there a plugin out there that can check wheter
something is done with the exception in a catch clause?
[18:49:42] <kenney> esp. stuff like this: catch ( Exception e ) { throw new
FooException("bar!"); }
[18:50:06] <grek> it's sign to go offline, so I would like to thank all of you
for the information and tips you have given to me
[18:50:19] <grek> thanks! :)
[18:50:36] <esm> kenney: can you configure a PMD rule to find those?
[18:50:43] <esm> grek: ttyl
[18:50:46] <kenney> hm, maybe
[18:54:10] <grek> ok, it's time leave, thanks again for your help, have a nice
evening, bye!
Ok, it's time for conclusions:
1. Defining repositories is considered harmful and we should get rid of all of
them
2. I think that we should move repository information to profiles.xml, but it's
a subject of discussion
3. We cannot put milestones that depend on snapshot versions on central. It
means that we can't upload following artifacts:
org.apache.cocoon:cocoon-maven-plugin:1.0.0-M1
org.apache.cocoon:cocoon-rcl-spring-reloader:1.0.0-M1
org.apache.cocoon:cocoon-rcl-webapp-wrapper:1.0.0-M1
But we still can have it in our staging repository and ask our early testers to customize their settings.xml for or create new one. I don't
see it as a big deal really, but I'm thinking about other implications. Especially, not all milestones will be available on the central,
there will be "gap" in version number. On the other hand, milestones are just our internal releases and I think we can live with these gaps.
What do you think?
--
Grzegorz Kossakowski
http://reflectingonthevicissitudes.wordpress.com/
