Hello all, I was reading through my backlog of Daily Python URL's and saw that the topics of deployment and configuration managment seem to be getting some attention lately. We have been having issues similar to this at my company, and I was hoping for some feedback on a solution I devised for these problems.
In our situation: - we have only a handful of developers, maintaining a very large legacy code base. - different applications use different versions of the same library on the same machine - developers still want to push bug fixes out without visiting every application - developers want to write new (non-compatible) versions of their libraries - developers want other developers to use their new libraries instead of the old ones The plan I have borrows a bit from the Java and Python development communities, and relies heavily on setuptools, policy, and some developer dicipline (we are professionals, after all). *Outline* - Developers outline Families of modules for common enterprise-wide tasks - Developers build Packages (python .egg files) from Families - Developers build Applications which are composed of Packages *Packages* - Packages are deployed to a SharedDirectory - Package versions have three number parts, Major.Minor.Bugfix - New package features, or *any* change in the package functionality, changes _at least_ the Minor version number. - Bugfixes should not change any functionality, and thus they only update the Bugfix version number - Developers write ChangeLogs for Packages before deployment *SharedDirectory* - Servers mount the packages SharedDirectory *ChangeLogs* - ChangeLogs outline any new features or Bugfixes in a Package - ChangeLogs are sent to all other developers after deployment *Applications* - Applications find their dependancies in the SharedDirectory - Applications have startup scripts that 'require' the developer-specified packages (for example, setuptools entry_points and automatic script generation do this) - Applications 'require' only the first two numbers of the package version, Major.Minor Some observations for deployers: - All applications should automatically upgrade if there is a bugfix version. - Changes in functionality are isolated from legacy applications. - Because applications are self-contained in setuptools .egg files, we can specify the shared directory as a global installation source when using easy_install. - Applications can be installed locally in a fashion similar to Java applications, which are often a collection of .jar files and data thrown into a common directory. We can do the same with "easy_install -f /opt/eggs -zmad /opt/myapp". - The application script itself could be shared from the central drive, if we write a custom script that does not use the shebang line. This has the side-effect of automatically upgrading *everyone's* version, which can be a blessing or a curse when you take versioning of run-time data into account. Some observations for developers: - The version numbering policy is flexible, but one *must* end up with a deployment policy where business-critical applications do not have even the most minor of functionality changes foisted upon them (for one solution see Java-style installations, outlined above). - Developers know what is happening to a package at a grain higher than that presented by Subversion, thanks to a published ChangeLog. - Developers are free to chose any version of the available packages, because they know that they are all available at install-time. They can upgrade and downgrade versions at will. - Developers are still responsible for upgrading legacy applications with more functional library versions, but that is what unit tests are for (you /do/ write unit tests, don't you?) - It helps to use a setuptools alias to easy_install when building packages for deployment. You can then type "python setup.py deploy" in the source directory as a quick and easy shortcut for building the required .egg. There are a few concerns that cross into release handling, and code library maintenance and care: - Regular library reviews and a comfortable package end-of-life schedule should be used to help prune the "supported packages" tree. - A relaxed package release schedule, with more features per-release, should help slow things down to a comfortable balance between time spent on upgrades, bugfixes, and features. - If the version numbers are climbing too fast, ask whether there should be more features per-release, or if the package should be put into 'alpha' or 'beta' status until it stabilizes. - Ask if anybody else is using the package you are maintaining. If not, ask yourself if it has be pre-maturely extract from an application (remember 'You Aint Gonna Need It'). - Ask if a quickly-rising package version indicates a family of modules and functionality that should be re-arranged. This could happen if someone is developing a new application and dumping too much application-specific functionality into an existing family. Can the family be split along functional lines? Should the functionality be kept in the application? (YAGNI again). Well, that's about it. You almost need to resurrect the role of 'Project Librarian' to keep track of it all! ;) I would love to hear people's feedback on this idea, as I am sure that I am not the first to tread this path. Maris _______________________________________________ Distutils-SIG maillist - [email protected] http://mail.python.org/mailman/listinfo/distutils-sig
