Re: [DISCUSS] enhance the assemble server portlet usability

[Joe Bohn]

David Jencks wrote:

On Aug 25, 2008, at 3:25 PM, Joe Bohn wrote:

Jay D. McHugh wrote:
Hey all,
I have been trying to get my thought straight on profiles/templates.
And, I think I just might have done it (we'll see).  Warning, there is
very little 'implementation' here - mostly food for thought.
First of all, I think that it would be useful to have several ways of
thinking about groups of modules.  Right now, we have maven artifacts
and plugins (that are groups of artifacts).  In this discussion, we are
trying to figure out how to divide/build up a server.  And, the idea of
profiles came up to group plugins that are necessary for a particular
function.
So far, I like everything about the direction that the discussion is
going.  But, I have two ideas that I think might improve the
managability of server building/configuration.
The first involves adding a list of profiles that the different
Gernonimo modules/artifacts would 'satisfy' into the pom's.  That would
enable us to stay away from manually building/rebuilding the list of
default included ('provided by Geronimo') profiles.
The second would be to add one more level of grouping artifacts -
templates.  The idea would be that profiles group modules that provide a
particular function and templates would group profiles that (when
combined) provide a particular server.


I think I'm getting a little lost in the terms here.  IIUC you are 
using the following groupings (from the most comprehensive to the least):

Templates (server definitions) contain Profiles
Profiles (groupings of plugins) contain Plugins
Plugins contain Geronimo modules/artifacts
modules/artifacts - core building blocks of Geronimo

I've oversimplified a bit but I think that is logical grouping you are 
proposing .... Is that correct?

If it is, then I think your 1st idea above is to include the knowledge 
of containment in the child rather the parent.  So would I be correct 
in assuming that you are proposing adding the knowledge of profile 
membership into the plugins (rather than modules/artifacts)?  I don't 
have an opinion on that yet ... just trying to understand.

I missed that and don't see how it would work.  If I come up with a new 
template or profile tomorrow, how will the current plugins know if they 
are in it or not?

As I tried to say before, hierarchy is good but we already have 
unlimited nesting in plugins.... we just need to fix a little bit of goo 
so they don't result in excess classloaders at runtime.


For example, right now, we provide five distinct 'flavors' of Geronimo:
minimal (framework), little-G (tomcat), little-G (jetty), JEE5 (tomcat),
and JEE5 (jetty).  Those would correspond to five 'provided' templates
for Geronimo.
As an (extremely oversimplified) example, here is what the little-G
template might look like:
<template id='little-g-tomcat'>
 <description>Geronimo Tomcat Little-G Server</description>
 <version>2.1.2</version>
 <includesProfile>
   <!-- full function profiles -->
   <profile id='geronimo framework' version='x.x' />
   <profile id='tomcat web container' version='x.x' />
 </includesProfile>
 <includesPlugin>
   <!-- individual plugins, either provided or 'customer specific' -->
 </includesPlugin>
 <includesArtifact>
   <!-- individual libraries -->
 </includesArtifact>
</template
A template like this would be relatively easy for either a program or
user to build and allows for almost unlimited customizability in
describing what should go into building a server.  Then, rather than
having our server assembly portlet actually do the work of making an
actual server - it could simply output the template.  Or, we could have
a new option that would allow for template creation and export (ie:
export current server as template).
Then we could either check the currently assembled server against the
template (and pull down whatever profiles/plugins are needed) or have a
'load template' function that would apply the template to a new
(presumably framework only) server.

This notion of a template is exactly in line with what we had 
discussed before and what I brought up again a few days ago.  It is 
basically a definition of a server in "thin" format.  I keep thinking 
of it in xml (like you referenced above) but it could be in some other 
format.  The idea was that you could use this template and some 
utility to enhance a framework server and turn it into whatever kind 
of server you want it to be.  The utility would be nothing more than 
something that could read the template, drill down to the appropriate 
set of plugins, and install them along with any other configuration 
necessary for the server.  The actual plugins that are installed (also 
groupings of plugins for simplification) could be in some local 
toolkit repo we ship with Geronimo or in remote repositories we make 
available when we release a server version (as we do today).

You can get a plugin to install just about anything you want already.  
The tomcat plugin installs the tomcat config under var, there are some 
instructions in the wiki on how to install application specific logging, 
etc etc.  There are also some ways to customize preexisting 
configuration (such as config.xml) that we use in the tck servers.... 
can't remember exactly how it works though.


I prefer this approach because it encourages portability and 
predictability.  You can generate a template and then use it to create 
many server images which should be identical.  Thought must be given 
when creating the template.  However, creating multiple server images 
is then a fairly simple process from the user perspective.  If we can 
come up with a version independent way to specify the Geronimo 
dependencies then this can be portable across server releases so that 
the user can potentially use the template they generated for a 
particular server image in 2.2.1 and generate a comparable server 
image for 2.2.2.  I wouldn't anticipate that we could make this 
portable across major releases but I would hope we could make it 
portable across minor releases.  We don't necessarily need to tackle 
the upgrade scenario now but I think the template approach sets the 
stage for some possibilities.

I really don't think we should encourage leaving out the versions.  it's 
a bit maven centric but maven can substitute versions when constructing 
a geronimo-plugin.xml.  I'm sure there are other ways.

I guess the other way is what you have mentioned before ... substitute 
the version when installing the plugin (via aliases).


I may not be understanding what you are saying.  It seems to me that 
what we're talking about is basically turning the lists of plugins now 
in the assembly poms into independent artifacts that can be used by the 
car-maven-plugin to assemble servers, and also from other spots such as 
gshell and maybe the console.  We're can do this today with the existing 
plugin mechanism although it has a few bad side effects which we should 
be able to eliminate pretty easily.

Right. I think the approach that you and Lin have been discussing to use 
the existing plugin mechanism and extend the function in the 
car-maven-plugin (and gshell/console) makes a lot of sense.  I think the 
concept of a plugin that is just an arbitrary container of other plugins 
(nested multiple levels) can be used in a variety of helpful ways:

1) It can be used to create these "profiles" of logical sets of plugins 
that make sense together for easy selection/installation by users.
2) A user can use the same feature to create their own profiles of 
functions base upon their own components.  For example, a user could 
create a shopping cart, bookstore, and electronics store profiles.  They 
could then combine/install these on servers to manage their book 
business, electronic business, or both on any combination of servers.
3) The profiles can be persisted, reused, and shared among users (just 
like regular plugins).
4) It can be used to construct entire collections of components that 
define a cohesive set of function for a server.  This can then be 
installed on a framework assembly to produce consistent/repeatable 
server images (such as our javaee5 server or some subset such as a web 
service only server image).
5) It might be able to be used to create a consistent server image 
across server released when modified (via automation or manually) to 
accommodate differences between the old and new server versions.  This 
can potentially provide an upgrade path for our users that have created 
custom assemblies.

I think this function is more general than just what is needed for 
"assemble a server".  However, in that specific context we could do the 
following:
- Let a user choose plugins, profile plugins (for lack of a better term) 
or whatever from which they would like to create a server image.
- Produce another profile plugin that includes the total selection 
instead of (or in addition to) creating a server image.
- The user could then save/publish/whatever the profile plugin.
- The profile plugin could be installed into a framework server image to 
convert it into the specific server that was desired.
- Of course, the real function (what we know as plugins now) would be 
installed from a repository - either local or remote.


This kinda gets at one of the things that has been nagging me about the 
"assemble a server" discussion ... that being that a user must select 
some core set of plugins to create a viable server image.  This always 
seemed a bit foreign to me because all along I've been thinking that 
assembling a server would always start with a core framework assembly 
which included the necessary core function (such as the ability to 
install plugins).  That was originally the purpose of the framework 
assembly and I ensured that you could install plugins into it way back 
when.  I think (but I might be wrong) that starting with a framework 
assembly when creating a new server assembly should make it easier to 
make a pure template/profile plugin because you don't need to worry 
about any core server capability.  All of the core stuff should already 
be included in server image used as the starting point.  A user would 
only need to install particular features on top of that to create the 
specific flavor of server they desire.  And, these features could be 
combined to create a server image that has the nature of multiple 
profiles.  From a user perspective it's all the same.  He can build up a 
server by installing plugins - some very small in granularity and other 
quite large (perhaps large enough to pull in everything they want in the 
server image with one selection).

I've rambled enough for now ... and I hope it isn't all lunacy.
thanks,
Joe




thanks
david jencks




Thoughts?
Jay
Lin Sun wrote:
Here is what I am thinking.   Let me take the Web profile as an 
example:

So we want to allow users to check/select the Web profile to select
all the necessary geronimo plugins for little G.   Users would only
see Web profile, instead the 10+ geronimo plugins.

-------------------------------------------------
Select from the following Profiles/Plugin Groups:
__ Web (when this selected, we'll install the 10+ geronimo plugins for
the user to get little G env.)
__ Web Service
...
-------------------------------------------------

In order to do this, we'll need to know which geronimo plugins can get
the users to the Web profile and store this relatonship somewhere that
is avail for both admin console and command line custom assembly.   I
come to the conclusion that we need some sort of group of plugins
function and David reminded me about the geronimo-plugin.xml that has
no module-id can work as group of plugins.   Here is the wording from
the schema:

If no module-id is provided, that means this is a plugin group, which
is just a list of other plugins to install.

With that, I can just build a geronimo plugin group for web profile
and have the 10+ geronimo plugins listed as dependencies.   This
geronimo plugin group can be available as part of the assmebly, along
with the other geronimo plugin groups.

The idea is that if a user selects Web profile in either admin console
or command line, we can just select the corresponding geronimo plugin
group behind the scene, which would install all its dependencies.

Now back to the web services sample, we 'll have 2 web service 
plugin groups:

web service CXF - cxf and cxf-deployer
web service Axis2 - axis2 and axis2-deployer

The web service Jetty plugin group will be included in the jetty
javaee5 assembly and web service tomcat plugin group will be included
in the tomcat javaee5 assembly.   Initially, I plan to only support
custom server assembly from the current local server, so when user has
jetty assembly, he will see web service CXF.   When user has tomcat
assembly, he'll see web service Axis2.   In the long run, we could
present both to the users and they can just pick either one.

I hope above addressed your questions.   Please feel free to let me
know any other comments you may have.

Lin

On Thu, Aug 21, 2008 at 11:57 PM, Jarek Gawor <[EMAIL PROTECTED]> wrote:
Hmm.. I'm not sure how this profile idea fits in with what the user
have to select in the "assemble a server" portlet. Would there be a
profile for axis2 that only has two plugins axis2 and axis2-deployer
defined? And there would be a similar profile with two plugins for
cxf? And the user would either pick the axis2 or cxf profile and
combine it with the jetty or tomcat profile? I'm just not sure how
this relates to the steps the user would have to go through in the
portlet to create the desired server.

Jarek

On Tue, Aug 19, 2008 at 4:20 PM, Lin Sun <[EMAIL PROTECTED]> wrote:
I have been thinking a bit more on how we achieve this.   Here is my
idea and I welcome your input -

So we have a need to allow users to install groups of 
plugins(function
profile), instead of individual plugins.   Install individual plugins
are nice for standalone apps, but for system modules, I think it 
would
be better to allow users to install groups of plugins as functional
profiles(unless the user is an expert user).    What we need is to
expose the groups of plugins for certain functions available to our
users and allow them to select the ones of their interest to build 
the
customer server.

I am proposing in addition to store plugin metadata of each plugin in
the plugin catalog, we could also host installable groups of plugins
information there (or in a separate catalog file).   For example, for
a function such as Web (same as little G) that has been discussed in
above posts, we could have the following plugin metadata -

<geronimo-plugin 
xmlns="http://geronimo.apache.org/xml/ns/plugins-1.3";
xmlns:ns2="http://geronimo.apache.org/xml/ns/attributes-1.2";>
  <name>Geronimo Assemblies :: Minimal + Tomcat</name>
  <category>WEB Profile</category>
  <profile>true</profile>
  <description>A minimal Geronimo server (Little-G) assembly using
the Tomcat web-container.</description>
  <url>http://www.apache.org/</url>
  <author>Apache Software Foundation</author>
  <license osi-approved="true">The Apache Software License, Version
2.0</license>
  <plugin-artifact>
      <module-id>
          <groupId>org.apache.geronimo.assemblies</groupId>
          <artifactId>geronimo-tomcat6-minimal</artifactId>
          <version>2.2-SNAPSHOT</version>
          <type>car</type>
      </module-id>
      <geronimo-version>2.2-SNAPSHOT</geronimo-version>
      <jvm-version>1.5</jvm-version>
      <jvm-version>1.6</jvm-version>

      <dependency>
          <groupId>org.apache.geronimo.assemblies</groupId>
          <artifactId>geronimo-boilderplate-minimal</artifactId>
          <version>2.2-SNAPSHOT</version>
          <type>jar</type>
      </dependency>

      <dependency start="false">
          <groupId>org.apache.geronimo.framework</groupId>
          <artifactId>upgrade-cli</artifactId>
          <version>2.2-SNAPSHOT</version>
          <type>car</type>
      </dependency>

      <dependency start="true">
          <groupId>org.apache.geronimo.framework</groupId>
          <artifactId>rmi-naming</artifactId>
          <version>2.2-SNAPSHOT</version>
          <type>car</type>
      </dependency>

      <dependency start="true">
          <groupId>org.apache.geronimo.framework</groupId>
          <artifactId>j2ee-security</artifactId>
          <version>2.2-SNAPSHOT</version>
          <type>car</type>
      </dependency>

      <dependency start="true">
          <groupId>org.apache.geronimo.configs</groupId>
          <artifactId>tomcat6</artifactId>
          <version>2.2-SNAPSHOT</version>
          <type>car</type>
      </dependency>
...

When a plugin is a profile, it means it just contains a group of
geronimo plugin dependencies that are installable and can perform
certain functions.  By installing it, it will simply install the
dependency plugins.

Questions -

How do we build this profile type of plugin?   We could build them
manually initially to try things but maybe c-m-p could be used here.
How do we install this profile type of plugin?  I think we could
leverage the pluginInstallerGBean to install it...when profile is
true, we just download the dependencies.
How/Where should we make this file avail?   We could make this file
avail in geronimo-plugins.xml (or another catalog file in repo) and
with our server assembly (one assembly contains the plugin 
profiles it
have).  When building customer server, when load all the plugins that
are profile and ask users to pick which ones they want.   If we 
have a
framework that can install geronimo plugins, a user can just download
the framework and pick from our apache repo on which plugin profiles
they want to build their geronimo server.
How are we handle the upgrade scenarios Joe mentioned?   No idea
yet... I think this is a rather complicated scenario.

Thanks,

Lin