> -----Original Message----- > From: Jesse McConnell [mailto:[EMAIL PROTECTED] > Sent: lundi 14 novembre 2005 15:09 > To: Maven Developers List > Subject: Re: [M2][Proposal] J2EE builds best practices and conventions
[snip] > ok, this is where you are losing me...what exactly are these extra > directory > structures gives us? You mention functional tests...is that was you are > aggregating under the applications directory? >From my first email on this thread: " [...] * build J2EE modules (JAR, WAR, EJB, EAR, etc) * support storing in the project the container's configuration files - for weblogic that's the domain config - for jboss that's the server config - for tomcat that's server.xml, tomcat-users.xml, etc * support building for different target environments - support container-specific deployment descriptor for example - support container config files that depend on the environment. For example you may want to run on port 8080 in your dev env and on port 80 in your production env. You may want to run on a single server in the dev env but in cluster on the production env, etc. - support different deployment models per environment. For example in one environment you may want to have the web server and EJB server on the same machine and in another environment you may want them located on 2 different machines. * support deploying config and J2EE modules to a container * support automated functional tests execution " The first bullet point is what you have covered in your mail. I'm interested in covering the other and for those I'm recommending the applications/ directory structure (and the system/ one). Application = programs that can be executed (i.e. bundle of different components + config + possibly start/stop scripts). Here some details from that same initial email: " Some explanations: * The applications/ directory contains subprojects (aka modules in m2 speak) that contain executable applications, i.e. things that will make up the full system. Applications are made of 2 things: - modules - configuration files including scripts (e.g. start/stop scripts, etc) * The database/ application represents the database. This is where schema and data could be located. This is also where database script files can be stored for starting/stopping/analyzing logs/profiling/etc. * The business/ application would be the business logic executing in the J2EE container. It would package the J2EE modules (EARs and possibly others) and the container configuration files. * The wsclient/ application is just an example showing a client application (ws stands for web services here). In this example this would be a sample swing application. * All the application projects above will generate a distribution (zipped file, tar.gz, etc) per environment containing a directory structure with scripts files in (say) bin/. Basically they should be self-running with only pre-requisite software required to be installed on the target machines (OS, JVM, container install, etc). * The system/ project represents the full system. It is made of applications. It'll also deliver a distribution (zipped file, tar.gz, etc) which will aggregate all the distributions from the other applications. This represents the full system for a target environment. Other considerations ==================== [...] * functional/system tests would be run in system/ (or application/) * It would be nice to be able to automate the installation of the distributions generated by the different application projects. This means the ability to install locally but also remotely. Alternatively this would mean building with m2 on all target machines to always install locally, which is also possible using M2 in embedded mode for example and the repo as a way to distribute the binary files - Actually this sounds cool :-). But this still leaves the issue of triggering the executions of the embedded m2 remotely. This requires some new plugin with some xml file to configure IPs and access to all the machines where to install the system. * Cargo can probably be used somewhere (to run functional tests, to create container configurationsn, to deploy to running containers, to start/stop containers, etc). I still need to figure out the usage of cargo vs shell scripts. " > This starts to get into the discussion we had a couple of months ago on > irc > in regards to the different testing methodologies to support under maven > direction. The simplest is the surefire launched unit tests sitting in > each > subprojects src/test/java directory. Next up would be integration tests > between different subprojects, different components in your above > example...is that purpose of the applications directory? Not quite. The applications directory has a goal: producing executable applications (i.e. installers or zips containing not only the components but also config files to configure the containers into which those components will go + start/stop scripts to start/stop those containers, etc). Obviously if you want to run functional tests you need a running environment, i.e. you need to have your containers configured and your components deployed into it. This is why running the functional tests in the applications/ project is a good place. Imagine you have an application that access a database and you want to run integration tests say in the ear component. If the database is not up and running with valid schemas and data your tests will fail. It is the role of the applications/database project to ensure that the database is up with a valid schema and proper data. OTOH, it is possible to use tools like the Cargo m2 plugin in the WAR or EAR component projects to perform integration tests using a *default* configuration. But that will not be the *target* configuration. I'm recommending to define the target configurations in applications projects and then possibly use Cargo there, using this defined configuration instead of generating a default one. There's work to be done in this area of generating/defining configurations. ATM Cargo is able to generate a default configuration that can be slightly tuned (ports, logging levels, security, etc). We would need to work so that users could override more of that. Smartfrog is also a good idea in this domain as it has a language to define configurations. > I think on that irc discussion this was about as far as we figure maven at > this stage would try and formalize since post integration testing it get > really localized to implementation... > > I would think that integration testing in a j2ee build, or any build > really > would just be another subproject that declared all of artifacts as > dependencies. If you had too many different integration tests or wanted to > break them out by functionality then adopt the same mechanism I mentioned > earlier, name them function-integration, function2-integration, > function3-integration...and if you get 5 or more then nest them in a > directory called integration. Yes, but it's not just about testing. It's about generating executable distributions. If your project don't want to generate an executable distribution then I agree you don't need the applications/ projects. > what artifacts are you seeing will be produced in the applications > directory? The war and ear artifacts? aren't they really just dependencies > to other artifacts... The war and ears are components produced by the components projects. The applications project generate zips and the like. > if applications is where you are envisioning storing configuration or > different containers and whatnot then I am not sure why that means that > the > component directory needs to nest everything else.. The separation between components/ and applications/ is there simply to distinguish them. Obviously if you wanted to have everything flat you could also have that. For a project the size of the daytrader app though I would recommend have a clean directory structure separating them. > In your comment you > mention starting and stopping the container and things..that almost sounds > like a tools or utility functionality which I would just make a subproject > for called deployment-tools or something...and in there either the pom > that > uses the relevant plugin or subprojects if there are multiple behaviours.. There are 2 things here: - data - tools that use this data to start/stop The applications projects store the data. The tools are indeed generic plugins (like the Cargo one or the db one that I'm planning to write). In any case these tools will not be developed as part of the applications/ projects. They would just get used there. <sidenote>We would also need a best practice to tell users where they should put build tools (i.e. plugins) that they develop for their build.</sidenote> > thanks for the response :) it looks like we are agreeing on a lot of the > fundamentals which should let jason actually start writing about these > things :P I'm trying to go beyond the fundamentals but we don't have to make those best practices. I'm just interested in writing about those in the J2EE chapter I'm writing for the m2 book because I think they are real-world things. There are probably different ways of doing this. I just know that this strategy has worked quite well for me in the past on several projects. As Steve was saying there are probably more robust solutions like using a proper tool (Smartfrog for example) outside of your build for doing this. I think the "Saint-Graal" in term of automated build if the ability to fully automate the process from checking out the sources to deploying a working solution on a pre-production platform. I used to call this "Agile Delivery" in some of my past presentations. I've been able to get very close to this once. One critical item is having automated end to end tests. Am I making sense? Thanks -Vincent > On 11/14/05, Vincent Massol <[EMAIL PROTECTED]> wrote: > > > > Hi Jesse, > > > > Thanks for restarting this thread! > > > > See below > > > > > -----Original Message----- > > > From: Jesse McConnell [mailto:[EMAIL PROTECTED] > > > Sent: lundi 7 novembre 2005 20:00 > > > To: Maven Developers List > > > Subject: Re: [M2][Proposal] J2EE builds best practices and conventions > > > > > > ok, digging this old thing out the dust bin.. > > > > > > I have abandoned my old layout (and the current structure provided by > > the > > > j2ee archetype) and shifting towards something that is more like the > > > current > > > state of the maven repo...and I am trying to decide the lvl of nesting > > > that > > > is useful for the components when factoring vincents threads above.. > > > > > > first off, I concede the usefulness of having actual source in the war > > > files...while my architecture doesn't really require/use that kinda > > thing, > > > it would be silly to design around the idea in any sort of best > > practices > > > way :) (that being that war's and the like don't need any compiled > > source > > > at > > > all, just suck in the associated dependencies) > > > > Cool :-) > > > > > ok...I'll start with the basics of the maven layout since that is a > > pretty > > > simple no brainer place to start..and for this discussion to make > sense > > as > > > a > > > best practices discussion I'll invent a little project and if it makes > > > sense > > > to later we can apply the daytrader app to it (I don't know enough of > > the > > > daytrader to do it justice atm). I know brett alluded to an exercise > > like > > > this in the initial mail above.. > > > > > > Lets say we have a mythical project called dropit... > > > > > > flat: > > > > > > dropit > > > dropit-logging > > > dropit-config > > > dropit-interfaces > > > dropit-frontend-servlet > > > dropit-backdoor-servlet > > > dropit-site > > > dropit-war > > > dropit-ear > > > > > > pretty simple project, three discrete source modules with a seperate > > > interfaces module for other systems to program against..the site, a > war > > > and > > > an ear.. > > > > Could you explain what the dropit-*-servlet projects are? Shouldn't they > > be > > located in the dropit-war project? > > > > > starting simple I guess there are a couple of questions we ought to > > answer > > > if we want to arrive at a best practice module layout... > > > > > > 1. do we want to distingish wars, servlets, ears and other packaging > > > concepts by the name of the directory or by nesting them? if so, how > > many > > > lvls deep? > > > > > > dropit (the primary source, sister element) > > > wars > > > wars/dropit-frontend > > > wars/dropit-backdoor > > > > +1 > > > > > or go for the gusto? > > > > > > ear/wars/dropit-frontend > > > ear/wars/dropit-backend > > > > I don't think this can be considered a best practice. It may happen in > > very > > particular circumstances that it'll work but not generally speaking. > > That's > > because the same jar/war/etc can be packaged into different wars, ears, > > etc. > > > > Also it sounds very weird to me to have a directory structure like: > > > > My project > > |_ src/... > > |_ pom.xml > > |_ dropit-frontend/ > > |_ dropit-backend/ > > > > Mixing src/ and subprojects at the same level doesn't resonate too well > I > > think. > > > > > 2. if we follow another common approach of nesting > > > > > > component/dropit > > > > What is this dropit project inside component? > > > > > component/dropit-logging > > > component/dropit-config > > > packaging/war/dropit-frontend > > > packaging/war/dropit-backdoor > > > packaging/dropit (type ear, generates dropit-1.0.ear) > > > > For me a jar, war, rar, even ear are all the same: they are modules (aka > > components). > > > > > this kinda approach still kind of rubs me raw with there being actual > > > source > > > in the wars since that means you have source under packaging > > > subprojects... > > > > Yep agreed. I don't like it either. I prefer the directory layout I had > > proposed in my earlier posts. > > > > > now we can move on to my personal psychosis :) > > > > :-) > > > > > I think my personal problem with trying to figure out a nice clean > > module > > > layout is that with maven's multiproject abilities it makes it > difficult > > > to > > > seperate my previous layout experiences stemming from one overriding > > > build.xml file containing everything woven together, from the more > clean > > > seperation of discrete units that maven2 encourages...working through > > the > > > repository for unit artifacts as opposed to other directories in the > > > project > > > really frees things up a lot. > > > > > > to be honest, working out my current setup and reading through the > > backlog > > > of this message I find myself leaning towards a best practices j2ee > > setup > > > of > > > very close to the maven-components one, just utilizing a naming > > convention > > > for j2ee components. > > > > > > dropit > > > dropit-logging > > > dropit-config > > > dropit-interfaces-api > > > dropit-frontend-servlet > > > dropit-backdoor-servlet > > > dropit-war > > > dropit-ear > > > > Yes, I fine with this too except that I'd name it a bit differently: > > > > dropit/ > > |_ logging/ > > |_ config/ > > |_ interfaces-api/ > > |_ frontend-servlet/ > > |_ backend-servlet/ > > |_ war/ > > |_ ear/ > > > > But again this is a very simple project that doesn't have everything > > automated? What about functional tests? Where do they fit? What about > > container configuration files? What about deployment to the container? > > What > > about database start/stop/loading of data? Etc... > > > > This is why I was suggesting an additional directory level in my > previous > > posts: > > > > dropit/ > > |_ components/ > > |_ [...] > > |_ applications/ > > |_ [...] > > > > But I agree that if, as a user, you don't want to automate all those > then > > a > > single directory level is enough. > > > > > For a simple project like this, there is no need to venture into lots > of > > > convoluted subdirectories that carry along a meaning with them (like > the > > > packaging thing and my hangup with there being source in there). > > > > Agreed. > > > > > I would even go so far as to say that if this where a best practices > > > layout > > > that a general rule of thumb could be that if you have 5 or more > > *-servlet > > > artifacts that you nest them into a servlets directory > > > > > > dropit > > > dropit-logging > > > dropit-config > > > dropit-interfaces-api > > > servlets/dropit-frontend-servlet > > > servlets/dropit-backdoor-servlet > > > servlets/dropit-fun-servlet > > > servlets/dropit-silly-servlet > > > servlets/dropit-humor-servlet > > > dropit-war > > > dropit-ear > > > > > > and the same thing would hold for the ejbs, wars, ears, whatever... > > > > +1 > > > > > the > > > only > > > exception being artifacts that compile to be jars...then would all > exist > > > at > > > the root of the project directory unless it really makes sense to nest > > > components that have maybe 5 or more of a particular thing. > > > > I'd treat jars in the same manner as wars, ejbs, ears, etc. > > > > > which...when you look at it like this closely models the current > > structure > > > of the maven-components svn module and how maven-plugins was nested in > > it > > > with a whole lot of plugins in it. > > > > > > Perhaps we don't necessarily want to promote a j2ee best practices > > layout > > > beyond a general rule of thumb for maven2 projects having all discrete > > > compiling units at the base directory of the project. That and a > > suggested > > > naming convention and nesting convention for j2ee artifacts -war, - > ear, > > > -ejb. > > > > Not sure about this naming conventions. > > > > > that accomplishs a lot: > > > > > > 1) it gets us out of the old habit of nesting things just to nest > things > > > 2) promotes the concept that a project is made up of a lot of > discretely > > > compiling dependencies > > > 3) holds all artifacts as equals, nesting only based on shared > > underlying > > > functionality > > > 4) all 'best practices' for maven2 archetype projects are the > > same...they > > > already have an established layout internal to the sub-project, src, > > > target,etc.. > > > 5) best practices for different meta project organizations is reduced > to > > > naming conventions, j2ee using -war, -ejb, -ear...others can follow > > their > > > own setups... > > > 6) following this kinda approach ties closely with the multi-project > > > setup...embraces the flatness really...maybe making it easier to > explain > > > how > > > inner project dependencies work without any sort of implied knowledge > > > based > > > on project directory layout... > > > > > > does this rambling make any sense to anyone else? :) > > > > Sure, it does! But it doesn't cover the full story. > > > > Thanks > > -Vincent > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > -- > jesse mcconnell > jesseDOTmcconnellATgmailDOTcom --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
