Bryan--
Hey; good proposal. I definitely agree that we need to revisit the structure of the distribution in the coming releases to simplify the Ant we provide and to address shipping zips for doc, samples, libraries, and the whole thing.
Couple of general comments:
1) It'd be great to understand more specifics about some of the changes you propose to both SVN and the distribution. Asked some questions about this below. :)
2) I'm still not convinced that Beehive should be in the business of having Ant in SVN and the distribution that is aware of various application containers.
In SVN:
We definitely need some container support so that we can run tests on a running server. But, the number of these SVN-only integrations would presumably be small.
In the distribution:
For builds, it seems like they should be customizable to adapt to various containers (more on this below). For deployment, this seems like an exercise best left to the developer. Two solutions would be to make container integrations available in a Beehive contrib/ directory (that doesn't ship but is checked in) or to have the community put this type of Ant on the Beehive wiki.
Thoughts from anyone else on this?
For full disclosure, there are a couple of us that are doing work to revamp the entire SVN build system to remove the use of the OS BEEHIVE_HOME variable, to reduce the dependence on hard-coded classpaths and JAR locations, and to remove Tomcat and Ant from SVN. Xibin and I have been doing this on and off for a couple of weeks now. Should be done in another week or two.
Other comments are in-line below.
Eddie
Bryan Che wrote:
Hi, this is a proposal for an updated Beehive build system that more broadly addresses support for various application servers (http://issues.apache.org/jira/browse/BEEHIVE-102) and other issues.
Please look it over and comment. If we get consensus on proceeding with something like this, then I will go ahead and do the implementation.
Bryan
------------------------------------------------------------------------
Proposal For a New Apache Beehive Build System
by
Bryan Che ([EMAIL PROTECTED]
2005-01-06
************************************************ Contents
1 Introduction 2 Background 3 New Build System Use Cases And Requirements 4 Proposed New Beehive Distributions 5 Proposed New Beehive Source Tree Development Environment 6 Conclusion
************************************************
1 Introduction
This is a proposal for modifying Apache Beehive's current build/test/distribution system so that it will support broader user needs, increase support for various application servers, and improve Beehive's maintainability.
2 Background
Beehive has two main build environments: the distribution environment and the source-tree environment. The distribution environment is the environment that Beehive users setup when downloading Beehive for developing and/or running Beehive-based applications. The source-tree environment is the environment where developers develop on, build, and test Beehive itself.
2.1 Distribution Environment
Currently, the Beehive distribution is available as one large download which includes Beehive jars, third-party jars, sample applications, a set of build tools for building Beehive applications, and documentation. The build tools include ant tasks for deploying Beehive applications to specific application servers and starting/stopping those application servers. Additionally, the build tools have extension points for adding support for other application servers.
The Beehive distribution seems targeted at developers who want to develop applications using Beehive and to work principally with Beehive's build tools and environment. Because it uncompresses directly into a usable development environment, it makes starting to develop a Beehive application straightforward. However, this distribution format has several deficiencies:
- It does not support users who want to install Beehive but not develop with it. This includes users who want to deploy their own applications or users who want to install other applications that depend upon Beehive - It does not support developers who want to use Beehive in the context of a different or larger development environment - It does not well support multiple developers working on the same machine
If I understand this and the solution below, this is facilitated by having the "Beehive development tools" distribution installable by each user, correct?
- It only supports the specific application servers for which it includes configuration files - It includes copies of third-party libraries, which could conflict with other versions of those libraries elsewhere on a user's system
The 3rd party libraries that Beehive ships in the distribution are those which are *required* for the runtime. For example, Beehive doesn't work with the v1 of Apache XMLBeans, which is why we ship v2 in the distribution. Classes in a Beehive webapp are application scoped, and hopefully don't conflict with libraries elsewhere in a user's application. It may be possible to swap out things like Log4J 1.2.8 for a 1.2.9 or 1.3.0, but this depends on the stability of those APIs.
What currently distributed libraries would you like to see removed?
Thus, the current Beehive distribution works great for a specific set of users--those who are primarily developing Beehive on a clean system and with an application server that Beehive supports. It does not work well for many other users.
2.2 Source-Tree Environment
Beehive's source-tree environment is easy to configure and setup for developers who want to work on Beehive itself. It provides ant tasks for setting up an work environment, and it also includes automated unit tests. These tests include mechanisms for testing Beehive inside of the Tomcat application server. This environment, though, also has a couple deficiencies:
- It only supports Tomcat and no other application servers - It includes a large set of third party software that would be better left for the user to obtain and install
Adding support for other application servers and minimizing the set of included software would make Beehive more portable and easier to maintain.
3 New Build System Use Cases And Requirements
3.1 Use Cases
An improved Beehive build system should support the following use cases:
It should support two main groups of users: people who want to obtain a Beehive distribution, and people who want to work on Beehive itself.
3.1.1 Distribution Use Cases
For people who want to work on a Beehive distribution, the build system and environment should enable them to work in several manners: - As developers of Beehive-based applications - As developers of large applications that depend upon Beehive - As users of Beehive applications
Furthermore, multiple users should be able with Beehive on the same machine, and they should be able to work with the application servers of their choice.
3.1.2 Source-Tree Use Cases
Developers working on the Beehive source-tree should be able to setup a work environment, develop on Beehive, build Beehive, and test Beehive. Furthermore, they should be able to differentiate Beehive itself from other software like application servers.
3.2 Requirements
Based upon the preceding use cases and also experience with the current Beehive build system, a new Beehive build system should have the following requirements:
3.2.1 Distribution Requirements
- The distribution should support two different sets of users equally well: those who just want to use Beehive, and those who want to develop Beehive applications - The distribution should be able to setup a proper development environment for those users who want one - The distribution should support multiple users per box - The distribution should support any application server - The distribution should support mechanisms for improving integration with specific application servers
Seems to me that there are two major issues for app container integration -- application build and application deployment.
On the application build side, we need two things:
1) decompose the application build infrastructure so that the various steps (XMLBean build, controls build, JPF build, etc) are separated and can be composed or worked into an existing app build by a Beehive user to build Beehive artifacts. Ant 1.6 provides several good mechanisms for this.
2) make it possible to customize application classpaths to include container specific JARs used during a build. For example, on Tomcat, the JSP / Servlet APIs are located in common/lib, and on WebLogic, these are in weblogic.jar. This user-defined <path> would be used by the "compile JPF" Ant snippet provided in (1).
For deployment, Beehive would not support any application server OOTB -- users need to use their own Ant to manage the application (deploy / undeploy / redeploy).
:)
Again, perhaps we can add these to a contrib/ or to the wiki.
3.2.2 Source Tree Requirements
- The source-tree environment should support developers working on Beehive - The environment should support automated unit testing on various application servers - The environment should minimize bundling of third-party libraries
Specifically, which ones?
- The environment should be easy to setup for working
4 Proposed New Beehive Distributions
The Beehive distribution should be available in multiple binary
packages: - Beehive Libraries
- Beehive Developer Tools
- Beehive Documentation
- Full Beehive
4.1 Beehive Libraries
Beehive libraries would consist of just jars of Beehive code and nothing else. This distribution would be for users who have no intention of developing with Beehive--they just need it as a dependency. Also, it would benefit packagers who want to re-distribute Beehive with other software.
Definitely! We need to have a library-only distribution a la the Struts Library one.
4.2 Beehive Developer Tools
The Beehive Developer Tools would come with a set of scripts to setup and support development environments for working with Beehive. This environment would be similar to the current Beehive distribution environment. However, it would have several differences: - It would not include jars in itself. Rather it would depend upon the Beehive Libraries distribution for providing Beehive jars and require installation of third-party software (just like the current Beehive distribution requires the user to install ant, Tomcat and Java). - It could be installed in multiple places on the same machine. It would have configuration files pointing to where canonical Beehive and other third-party libraries are located - It would build and deploy wars of Beehive-based applications. Currently, deploying is contained in application-server-specific build files. The new developer tools would have an application-server-neutral deploy task that builds a war and copies it to a user-specified location. Extra, application-server-specific build files would add further functionality for popular servers - It would provide tools to verify that the development environment is properly setup and that necessary libraries are present
If this is something that the community would be interested in, I think this would be great!
One question, though -- is this meant for devs to customize their "Beehive Developer Tools" environment individually? If several users are sharing a Beehive installation, would it be the case that it's easier to customize the one install for a particular container?
4.3 Beehive Documentation
Comprehensive Beehive documentation would be available as its own download.
Agreed.
4.4 Full Beehive
A full Beehive download would include all the other Beehive distributions: the libraries, developer tools, and documentation.
5 Proposed New Beehive Source Tree Development Environment
The new Beehive source tree development environment would remain similar to the current one. However, it would have the following changes: - The build files would be refactored to support multiple application servers rather than just Tomcat. The end product would be similar to the build files in the current (not new) Beehive distribution build files: there would be application-server-specific files for running/deploying servers and a configuration switch for changing between application servers. Since Beehive is an open-source project, it could rely to some extent on its community to contribute application-server files for servers that people want to see supported
As with the distribution, I'm concerned about having container-specific integrations in trunk/. We need at least one container (presumably Tomcat) in trunk/ for testing. But, I'd vote for making it possible to *enable* cleanly plugging in other containers rather than trying to to *solve* it for various containers.
That being said, we may have to make some changes to the way the tests run to make it easier to run test webapps on other servers.
- The tree would not include Tomcat or Ant and would also remove other third-party libraries that are common and easy to obtain. The build scripts would be updated to verify that this software is present
Definitely. We're working on this now, and it should be done in a week or two.
- The source tree build tools would be able to build the various types of new Beehive distributions
Yep. I'd love to see a distribution.xml Ant file in trunk/ that knows how to build all of these things. Want to take a crack at this?
;)
6 Conclusion
By making these changes to Beehive's build system, Beehive would be able to suit a broader audience beyond just developers using Tomcat. In particular, the distribution changes would make things much simpler for Beehive users who do not develop. And, they would bring Beehive packaging more in line with that of other software like Struts.
Providing specific support for various application servers in the source-tree environment would allow people to test that Beehive works on servers other than Tomcat. Conversely, it would also allow application-server providers to certify Beehive on their own application server.
Absolutely -- the container specific testing is important. We just need to figure out how to get there.
