2008/6/30 Ishaaq Chandy <[EMAIL PROTECTED]>: > Hmm, so in short you're telling me that I should completely lock down my > build process simply because maven can't differentiate between plugin deps, > test deps and compile/runtime deps. >
my perspective (as a maven user) is that if I was concerned about dragging in artifacts with certain licenses then I'd want to be 100% sure I had everything locked down build-wise. that seems easier to defend (legally-speaking) than relying on scopes, where you could drag in something accidentally and may only find out when it's too late Look, I know I'm starting to sound like a whining complainer and I wouldn't > blame you if you got annoyed, but look at it from my perspective: put aside > for a moment what maven can and can't do, and how it implements dependency > management. Now consider this: > FYI, I'm not a core Maven developer so I won't get annoyed ;) > 1. Legally, it makes sense to vet all the compile-time and runtime > dependencies of a distributable product (commercial or otherwise). > agreed - some people vet at release time, others all the time ( that's why locking all the versions down can help - because if it was all ok at release time then there's no way an artifact could have sneaked in later on for the same release ) > 2. The build processes, the tests, the metrics are all internal processes > and are not part of the distributable product, in some scenarios the > restriction on the artifacts used to produce these can be different from > those on the compile/runtime deps. In my commercial scenario, having to > process, lock-down and manage each of these deps using the same stringent > procedures used to vet normal compile/runtime deps is just needless > make-work - there is no legal requirement to subject my dev team to this. > I guess it depends on your company - where I used to work they were just as strict with build and test artifacts because it's then easier to maintain a "cleanroom" status of course with build and test artifacts there's an upfront cost but that should hopefully level-off once everyone's using the same setup Now, it would be perfectly reasonable to state that maven does not do what I > need it to do, that the maven developers are not interested in solving this > scenario and that I should look elsewhere. > even if it's possible technically, I think it's valid to ask whether this is the right thing to do - of course that's just my opinion ( i.e. not everything that can be implemented should be ;) I guess that what I was just hoping for was that these would in fact be > considered reasonable requirements; not as niche as, it turns out, you > obviously think they are and that, on the contrary, there is a way to solve > this using maven. Unfortunately, given your answers, it seems to be that > this is not the case. A pity. > imho, discussion helps everyone, and presenting a counterpoint is usually a good way to check requirements - so even if you decide to look elsewhere, I hope this has been semi-helpful Thank you again for your patience, > Ishaaq > > 2008/6/30 Stuart McCulloch <[EMAIL PROTECTED]>: > > > 2008/6/30 Ishaaq Chandy <[EMAIL PROTECTED]>: > > > > > Hi Jörg, > > > Thanks for your reply. > > > > > > Unapproved transitive deps can "creep" in because we do not lock down > > > plugin > > > dep versions - for e.g. even maven's compiler plugin could conceivably > > have > > > transitive deps - we do not explicitly lock down the version numbers of > > > each > > > and every plugin we use - yes, this opens up the risk of having > > > unrepeatable > > > builds but these risks of this happening in the short term are pretty > low > > > IMHO so we can live with it. However, one of the problems is that I > > cannot > > > differentiate between the transitive deps based on the scope of the > > > original > > > artifact. > > > > > > hmm, maybe I should backtrack a bit and set out re-explaining what I > was > > > trying to achieve: > > > > > > I was setting up a couple of central repositories for our team: > > > > > > 1. A locked down repo in which only "approved" versions of some deps > > exist. > > > > > > 2. An open repo which proxied on to maven central. > > > > > > Note that the process of approving an artifact is not trivial - we need > > to > > > vet each dep and each transitive dep of said deps - don't really want > to > > do > > > all this rigmarole in the case of test-scope and plugin deps. > > > My naive assumption was that I would be able to direct all test-scope > and > > > plugin deps along with their transitive deps on to the second repo and > > all > > > compile/runtime deps to the first. > > > > > > Yes, a developer can usually find a way to break rules, my objective is > > not > > > to try and fix all possible holes, just to have some structure around > how > > > developers go about adding deps to our projects. As it stands currently > > > with > > > how maven manages deps I can either open up everything, giving up on > what > > I > > > set out to achieve or lock it down completely which will turn out to be > > > highly unproductive and frustrating for our team - simply because there > > is > > > no real legal reason to lock down test and plugin deps, the only real > > legal > > > reason for lock-downs applies to compile/runtime deps. > > > > > > Even if maven somehow communicated the scope of the artifact it is > > > requesting to the repository I could work around this at the repository > > > level; i.e., for e.g., the open-proxied repo would reject all requests > > for > > > compile/runtime dependencies and only allows requests for test and > plugin > > > dependencies and their transitive dependencies, but maven swallows up > > this > > > information before making the call to the repository. > > > > > > Is sending this information along with the call to the repository > > > difficult? > > > I have no idea how maven resolves dependencies so have no idea as to > how > > > easy/difficult this is. > > > > > > > Part of the problem is that once an artifact is downloaded to your > > local repository, then there's no clear record of where it came from, > > so it wouldn't be easy to exclude it from later dependency searches > > > > also, I'm not sure that 'legally' it makes sense to firewall artifacts > > based on scope - it sounds very fragile and easy for artifacts with > > 'incompatible' licenses to leak through > > > > imho the best approach is: > > > > 1) lock down all versions (use dependency+enforcer plugins) > > 2) use a controlled corporate repository > > 3) check transitive dependencies as part of release process > > > > I think the key thing with 2) is that you should try to streamline the > > process to approve artifacts rather than attempt to work around it. > > > > just my 2 sen > > > > > > > Thanks for your patience. > > > > > > Ishaaq > > > > > > 2008/6/30 Jörg Schaible <[EMAIL PROTECTED]>: > > > > > > > Hi Ishaaq, > > > > > > > > Ishaaq Chandy wrote: > > > > > Well, not knowing who else uses maven out there I have no > > > > > reasonable way to > > > > > verify or deny your claim that this is not useful for 95%. I > > > > > can only say > > > > > that I find it hard to believe that only 5% of maven users > > > > > would conform to > > > > > both of the following criteria > > > > > > > > Maybe those users simply use Maven in a different way? ;-) > > > > > > > > > - but then again, I don't really know: > > > > > > > > > > 1. You're writing a commercial app (or any app for that > > > > > matter which has a > > > > > potentially incompatible license with some or all of its > > > > > dependencies, so, for e.g. a BSD app with a GPL dependency). > > > > > > > > > > and > > > > > > > > > > 2. If said app is supposed to be redistributable (or, in some > > > > > other way, > > > > > violate the license agreement with its dependencies). > > > > > > > > > > > > > > > If you are in the unfortunate situation of satisfying both > > > > > criteria then you > > > > > only have 3 options w.r.t. implementing your build system with > maven: > > > > > > > > > > 1. Just continue to use the standard maven dependency system > > > > > and don't worry > > > > > about dependencies with incompatible licenses and hope that no > lawyer > > > > > notices. Possibly works for a lot of small projects, not > > > > > really an option > > > > > for me - and I'm sure not a practice that Apache or Maven would be > > > > > officially encouraging. > > > > > > > > > > 2. Same as 1 above, but have a post-build manual process to > > > > > make sure that > > > > > no incompatible dependencies have crept in. > > > > > > > > How should an incompatible dependency have "crept" in? If someone on > > your > > > > side add a new dep, it is his repsonsibility to clear up the dep and > > its > > > > transitive deps also. Since a released artifact never changes, its > deps > > > will > > > > also never change. > > > > > > > > > The problem with > > > > > this approach > > > > > is that you have to make sure you catch these incompatible > > > > > dependencies early on else it may be hard to fix your source code > > > > > after-the-fact. Note > > > > > also that incompatible dependencies can creep in due to an updated > > > > > transitive dependency, not just a dependency explicitly > > > > > listed in your own > > > > > pom.xml. > > > > > > > > Even an update of a dependency must have been done by someone on your > > > side. > > > > Same responsibilities apply. > > > > > > > > > 3. Lock down your repository completely so that all > > > > > dependencies will have > > > > > to be vetted - a bit over the top IMHO - considering that in > > > > > reality you > > > > > only need to vet the compile/run-time scope dependencies - > > > > > and certainly not > > > > > the test-scope and the plugin dependencies (along with their > > > > > respective transitive dependencies). > > > > > > > > This is *exaclty* what we do. Simply use a company POM where you > > declare > > > > your versions in the dependencyManagement (and lockdown the versions > of > > > the > > > > plugins). Remember that those versions will talke precedence about > the > > > > versions of the transitive deps. You may even declare dependencies > with > > > > "unwanted licenses for product" with test scope. And you should > really > > > lock > > > > down plugins, otherwise a release is not repeatable in a defined way. > > > > > > > > > As for your solution of setting up multiple environments, I > > > > > am not sure that > > > > > it is practical - I have a large multi-project build that I > > > > > need to be able > > > > > to run in one go using my continuous build environment. That > > > > > build not only > > > > > compiles the projects but also runs tests, check coverage, metrics, > > > > > publishes site info etc. It would be really hard and > > > > > cumbersome to have to > > > > > split this out into multiple environments. In short, I'd have > > > > > to do this in > > > > > at least two passes - one pass would compile the build and > > > > > the second pass > > > > > would run tests etc. However, even this is problematic as the > > > > > first pass > > > > > still requires plugins (for e.g. the Antlr plugin) whose transitive > > > > > dependencies would leak across into the non-plugin repository (as > > > > > mentioned in my previous post). > > > > > > > > > > Am still happy for someone to tell me I am being an idiot and > > > > > that this is > > > > > really simple to setup with maven (that would solve a number > > > > > of issues for > > > > > me), but I fear this is not the case. > > > > > > > > If somebody want to break the rules, he will find a way. If you do > not > > > want > > > > your ordinary devs to take the responsibility for the transitive > deps, > > > you > > > > may setup a company wide repo with approved artifacts and force > > everyone > > > to > > > > use that repo as mirror of central. Then you'll need a specialized > team > > > to > > > > approve and upload new artifacts manually like it is done for > > > MAVENUPLOAD. > > > > > > > > - Jörg > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > > > > > > > > > > > > -- > > Cheers, Stuart > > > -- Cheers, Stuart
