[RT] blue-sky container

[Neeme Praks]

Hi all!

My day-job company is in the process of refactoring our in-house-built 
"middleware" into "something better"(tm). That "middleware" is currently 
using plain Tomcat as a container and Spring for "component 
configuration/wiring", publishing its services over SOAP. Due to 
increased requirements (uptime, clustering, monitoring, 
ease-of-maintenance, etc), we are looking into a major architecture 
redesign.

Functionally, the "middleware" is a /telco-grade/ piece of software that 
is used to build services that interface with the (mobile) operator 
systems (billing, SMSC, positioning information, etc) and provide a user 
interface over a variety of channels (wap, web, SMS, MMS, etc).

As an apache contributor, I would love to use (read: push ;-)) Geronimo 
as the base-container for this work. However, I'm not too familiar with 
the internals of Geronimo - so I would like to ask you to give me some 
feedback about the feasibility of this plan.

I've written up my own blue sky vision/ideas for that future container 
(see attachment). It is lacking in details, but I guess you get some 
sort of fuzzy feeling about the thing I'm trying to get at :-)

The aim could be summarized as: build a container that would be
 * simple for developer (no EJBs, stick to POJO's as much as possible)
 * simple for deployer/upgrader (container support for configuration 
management)
 * simple for Operations&Maintenance people (container support for 
monitoring, alarms)
 * easy to cluster (HA and load-balancing, no state in middle tier) in 
a reliable way (we have been throwing around the idea of routing all 
method calls over JMS)

I know that everything is possible in the world of software development 
(given enough time and money :-)) so I would narrow my questions a bit:
 * is this thing a good fit, to be built on top of Geronimo? I guess yes.
 * maybe you can also give some arguments as to why Geronimo is a good 
fit for this?
 * and, going one step further, could you give some pointers as to 
how/where to start.

And, a final question: would you be interested in at least some of these 
features? As far as I can see, most of the stuff outlined in my blue-sky 
vision is not really specific to our company/industry, it is applicable 
to java development/deployment in general. So, there could be interest 
also from the broader community in implementing this beast and our 
company might be interested in contributing back the generic parts of 
our work.

What do you think? ;-)

Rgds,
Neeme
Title: MO/RefactoringProducts/PinPoint50Ideas/BlueSkyFeatures - Regio Wiki








MO/RefactoringProducts/PinPoint50Ideas/BlueSkyFeatures



an artifact
Piece of functionality packaged in a way that is ready for deployment. Consists of: version, unique identifier and type (maven2 metadata) compiled java classes - in the future this can be optional, as we have sources anyway. But initially it is more convenient to just use pre-compiled classes. java sources - used primarily for documentation and bug hunting, but can also be used for compiling JavaDoc documentation - can be generated from sources any metadata about the code that cannot be derived from the java classes and sources dependency declarations Proper artifact versioning is very important - it should be possible to determine the level of compatibility between any two versions by just comparing the version numbers. The whole concept is built on top of Maven2 project descriptors. 
CORE

The function of CORE
Publish a set of components (interfaces) to: the external world - (remote) clients can use services by including: a client library - publishes the component types (interfaces) that are publicly available a transport library - implements the details of accessing the (possibly remote) implementations over JMS/RMI/choose-your-own-transport. The simplest transport would be "no transport" - all services would be available inside the same JVM. the internal world - other components living inside the same container CORE can also be viewed as a "cloud" of middleware nodes in a cluster. Web applications and WSG would be examples of external clients that run outside the CORE. SMS handlers could be an example of internal clients that run inside the CORE (in their own "SMS-processor" component). In addition to hosting and publishing the components themselves, the container also manages a set of shared libraries that components can use to provide their services. 
a component
Component consists of: a component type - a public interface (stored in artifact) through witch the internal functionality can be accessed one or more component implementations  When a component implementation is deployed, the deployer can override any of the component runtime parameters. component implementation + runtime parameters = component deployment When component deployment is instantiated by the container (loaded into memory) it becomes a component instance. If allowed by the component implementation, the container may choose to pool more than one component instances. There can be a set of well-known component types that the container can use internally. 
a component implementation
Implements a single component type. Consists of: declaration of component type declaration of artifact where the functionality is implemented default values for runtime parameters documentation about this component implementation - will be obtained from javadocs in java sources metadata about the capabilities of this implementation - can be used to (semi)automatically choose the right implementation when choosing between different implementations. declaration of artifact where the functional tests are located - component with published interface should have (semi)automated functional/integration tests (using that published interface) that can be used to verify that the component is properly configured and fully functional after deployment (initial installation or upgrade). The tests may have input parameters that will be asked from the person conducting the deployment. The parameters can be saved so that it will be possible to automatically use the parameters from installation when doing an upgrade. Component definition can declare a dependency on other components and specify any special capabilities that it requires. 
a component deployment
Component deployments can be nested - if a component implementation depends on another component implementation that is truly internal to the implementation (needs custom configuration that is specific to the current component only) then the container can create a new (sub)component deployment to implement that dependency. Dependencies can be resolved in three ways: nested deployment (described above) default deployment - searches for default deployment from this component and up custom deployment based on capabilities - deployer can specify that this dependency has to be resolved by a deployment with particular capability. This can be used to force the dependency to be resolved to a non-default deployment. Consists of: declaration of component implementation declaration if this component should be made externally available (available at root deployment level only) declaration if this component should be a singleton inside the cluster (e.g. SMS driver that can run only on single node) declaration if this deployment is the default deployment for particular component type any non-default configuration settings any additional (custom) capabilities - can be used for customizing dependency resolution Only externally available components can be remoted inside the cluster. The decision to execute a method call on a local node or a remote node could be made at runtime: the container could use the following criteria: if ([number of requests in queue on local node] * [average time of processing a single request on local node] > 
    [number of requests in queue on remote node] * [average time of processing a single request on remote node] 
    + [overhead incurred by remoting])
then send that method call for execution on remote node.
This decision logic implies that all nodes have more-or-less up-to-date data available about the remote queue sizes, execution times and remoting overheads. Some of this data can be updated from the method calls themselves, but most of it probably has to be distributed on regular basis (e.g every 500 ms) Implementation note: when doing an upgrade the container should compare the old set of default values against the new set of default values. If default values have changed between versions, notify the deployer and ask for course of action: override the new default value with old default value or accept the new default value. would make sense to save configuration settings as XML or as JXPath:value mappings 
a component instance
Instance lives inside a separate classloader and only the component type (interface) is visible to the outside world. 
configuration
All component deployments are configured with XML configuration files - no database is needed + it is much easier to evolve XML schemas and to implement versioning. Configuration changes: all changes are versioned - it is always possible to roll back to any previous configuration. It is also possible to examine the differences between any two versions. configuration changes are broadcast over the cluster as a new set of affected XML files: the container saves the files locally (to be used on next startup). The files are versioned, timestamped and checksummed to detect corruption and mismatches. by comparing the old and new configurations, the container figures out which components it should update. the container creates new deployment definition based on the new configuration, instantiates a new component instance and applies the definition. the container stops the current deployment and allows any currently executing work to finish - no new work is accepted. the container replaces the running deployment with the new deployment the container destroys the old deployment, allowing it to be garbage collected. a single configuration change can consist of multiple updates to more than one component - all these components are swapped at the same time. By using easily distributable set of XML configuration files, we could also host a central repository of all client configurations, allowing to easily set up customer specific testing environments and to validate that all upgrades work in all customer configurations. 
installations and upgrades
Installation is done by loading a descriptor into an empty container. The descriptor defines the components to be deployed (types, implementations, etc); required artifacts and libraries are downloaded and loaded into runtime environment by the container. The download doesn't have to be remote - the container could also load the files from local directories for offline installations. However, a repository accessible over http is recommended for cluster deployments, then all members of the cluster can download their code from there. Another note about cluster installations: after setting up the first cluster node, it is enough just to start up an empty container on the other machines with valid cluster configuration - the new nodes will join the cluster, get the active configuration from the running node and deploy it automatically. Upgrades are applied by changing the component deployment descriptors - this can be a mere configuration change or a specification for a new artifact version that will be downloaded and deployed. Upgrades definition/descriptor consists of component base version declaration description of updates to be applied to the base version the new version declaration As we are dealing with XML, upgrade descriptor could be an XSLT script. But that is just one of the options, have to analyze the details when more is known about the deployment descriptors and the nature of changes that should be applied to them. Most probably XSLT is too generic and a more specialized "language" is better. Before applying an upgrade, the container can validate the upgrade by loading the new configuration and applying a set of (semi)automated functional tests to it. The deployer can inspect the results of the tests and make sure that all new and old functionality works as expected. In principle we could also implement update chaining (you have version 3.11 of some component and you tell the container to upgrade it to version 4.0 and the container can fill in the rest). But maybe this is overkill already, probably it is enough to make upgrades as easy as possible for the upgrader. 
Upgrade preparation tool
This tool can be used to automate the upgrade descriptor writing: it can take an XML deployment descriptor and compare that to a component implementation - as a result, it will highlight any conflicts between the two. This can be used to make sure that all code changes that result in structural changes to XML deployment descriptors will be accounted for in the upgrade descriptor. When it has highlighted the conflicts, it could allow the developer to make the needed changes to the configuration, recording the changes transparently and then using that information to automatically generate an upgrade descriptor. This generated descriptor might need some manual fine-tuning but at least the bulk of the work can most probably be automated. 
configuration tool
All configuration changes go through three steps: edit one or more configuration "values" validate the changes - check the types, run functional tests, etc commit - apply the changes to this and other nodes Configuration tool should be a web application with rich-client-interface, most probably it would make sense to standardaze on Firefox (or any other XUL compatible browser) and implement the interface with XUL. Would get rid of the need to work around the bugs in IE _javascript_ and CSS layout engine. And XUL is very powerful for building nice user interfaces. Hopefully operators don't mind  All writable JavaBean properties will become configurable through the admin tool. The tool allows the admin to browse the deployed component tree and modify the values in that tree. There could also be a set of commonly configured properties available as a list of JXPath expressions (with custom descriptors). The component tree is documented with content from JavaDocs and there is also possibility to see the (cross-referenced) java code behind the components. For each configurable property, the documentation comes from the setter method JavaDoc (if that is not available, then getter JavaDoc is used). In order to enable configuring dependent components, there could be a possibility to define a new component deployment descriptor: the component declares a dependency of certain type the admin chooses one of the compatible component implementations provides custom configuration for it, if needed adds a name and description to it, so it can be easily identified later tells the parent component to use this newly configured component for particular dependency In addition to configuring the properties through setters, it is also possible to monitor the health of the system by inspecting the values available from getters. 
monitoring console
(implemented with same technology as configuration tool, probably XUL) Monitoring console allows the O&M people to: view a list of data values that reflect the health of the system/cluster (probably should be implemented as a tree, to make it more manageable) define new aspects of the data to monitor in the system. Monitored data descriptor includes: JXPath path to the data (some getter on some bean in some component) - the admin can select this visually by navigating the component tree documentation from getter JavaDoc custom documentation by the person who defined this descriptor data value (or more than one value if in a cluster) if in a cluster then could also have aggregate data value - sum/avg/min/max/some-other-aggregation function to be applied to individual data values custom formatting, based on monitored values (combination of _javascript_ and CSS makes this very easy in XUL) custom alarms, based on monitored values? Implementation notes: the JXPath could be made linkable - clicking on a bean name would take the admin to that particular bean in the component tree JXPath could also provide tooltips when mouse moves over individual bean names - the documentation would come from JavaDocs when upgrade is done to the system, the upgrade system should check if the affected beans are being monitored (listener pattern) and if they are then it should automatically update the JXPath values, if needed. The updates should also be shown to the person performing the upgrade and in case of conflicts (deleted properties) manual updates may be needed. MO/RefactoringProducts/PinPoint50Ideas/BlueSkyFeatures  (last edited 2006-04-27 07:47:49 by neeme)