Re: [DISCUSS] enhance the assemble server portlet usability

[David Jencks]

On Aug 25, 2008, at 3:25 PM, Joe Bohn wrote:
Jay D. McHugh wrote:
Hey all,
I have been trying to get my thought straight on profiles/templates.
And, I think I just might have done it (we'll see).  Warning, there  
is
very little 'implementation' here - mostly food for thought.
First of all, I think that it would be useful to have several ways of
thinking about groups of modules.  Right now, we have maven artifacts
and plugins (that are groups of artifacts).  In this discussion, we  
are
trying to figure out how to divide/build up a server.  And, the  
idea of
profiles came up to group plugins that are necessary for a particular
function.
So far, I like everything about the direction that the discussion is
going.  But, I have two ideas that I think might improve the
managability of server building/configuration.
The first involves adding a list of profiles that the different
Gernonimo modules/artifacts would 'satisfy' into the pom's.  That  
would
enable us to stay away from manually building/rebuilding the list of
default included ('provided by Geronimo') profiles.
The second would be to add one more level of grouping artifacts -
templates.  The idea would be that profiles group modules that  
provide a
particular function and templates would group profiles that (when
combined) provide a particular server.


I think I'm getting a little lost in the terms here.  IIUC you are  
using the following groupings (from the most comprehensive to the  
least):

Templates (server definitions) contain Profiles
Profiles (groupings of plugins) contain Plugins
Plugins contain Geronimo modules/artifacts
modules/artifacts - core building blocks of Geronimo

I've oversimplified a bit but I think that is logical grouping you  
are proposing .... Is that correct?

If it is, then I think your 1st idea above is to include the  
knowledge of containment in the child rather the parent.  So would I  
be correct in assuming that you are proposing adding the knowledge  
of profile membership into the plugins (rather than modules/ 
artifacts)?  I don't have an opinion on that yet ... just trying to  
understand.

I missed that and don't see how it would work.  If I come up with a  
new template or profile tomorrow, how will the current plugins know if  
they are in it or not?

As I tried to say before, hierarchy is good but we already have  
unlimited nesting in plugins.... we just need to fix a little bit of  
goo so they don't result in excess classloaders at runtime.


For example, right now, we provide five distinct 'flavors' of  
Geronimo:
minimal (framework), little-G (tomcat), little-G (jetty), JEE5  
(tomcat),
and JEE5 (jetty).  Those would correspond to five 'provided'  
templates
for Geronimo.
As an (extremely oversimplified) example, here is what the little-G
template might look like:
<template id='little-g-tomcat'>
 <description>Geronimo Tomcat Little-G Server</description>
 <version>2.1.2</version>
 <includesProfile>
   <!-- full function profiles -->
   <profile id='geronimo framework' version='x.x' />
   <profile id='tomcat web container' version='x.x' />
 </includesProfile>
 <includesPlugin>
   <!-- individual plugins, either provided or 'customer specific'  
-->
 </includesPlugin>
 <includesArtifact>
   <!-- individual libraries -->
 </includesArtifact>
</template
A template like this would be relatively easy for either a program or
user to build and allows for almost unlimited customizability in
describing what should go into building a server.  Then, rather than
having our server assembly portlet actually do the work of making an
actual server - it could simply output the template.  Or, we could  
have
a new option that would allow for template creation and export (ie:
export current server as template).
Then we could either check the currently assembled server against the
template (and pull down whatever profiles/plugins are needed) or  
have a
'load template' function that would apply the template to a new
(presumably framework only) server.

This notion of a template is exactly in line with what we had  
discussed before and what I brought up again a few days ago.  It is  
basically a definition of a server in "thin" format.  I keep  
thinking of it in xml (like you referenced above) but it could be in  
some other format.  The idea was that you could use this template  
and some utility to enhance a framework server and turn it into  
whatever kind of server you want it to be.  The utility would be  
nothing more than something that could read the template, drill down  
to the appropriate set of plugins, and install them along with any  
other configuration necessary for the server.  The actual plugins  
that are installed (also groupings of plugins for simplification)  
could be in some local toolkit repo we ship with Geronimo or in  
remote repositories we make available when we release a server  
version (as we do today).

You can get a plugin to install just about anything you want already.   
The tomcat plugin installs the tomcat config under var, there are some  
instructions in the wiki on how to install application specific  
logging, etc etc.  There are also some ways to customize preexisting  
configuration (such as config.xml) that we use in the tck servers....  
can't remember exactly how it works though.


I prefer this approach because it encourages portability and  
predictability.  You can generate a template and then use it to  
create many server images which should be identical.  Thought must  
be given when creating the template.  However, creating multiple  
server images is then a fairly simple process from the user  
perspective.  If we can come up with a version independent way to  
specify the Geronimo dependencies then this can be portable across  
server releases so that the user can potentially use the template  
they generated for a particular server image in 2.2.1 and generate a  
comparable server image for 2.2.2.  I wouldn't anticipate that we  
could make this portable across major releases but I would hope we  
could make it portable across minor releases.  We don't necessarily  
need to tackle the upgrade scenario now but I think the template  
approach sets the stage for some possibilities.

I really don't think we should encourage leaving out the versions.   
it's a bit maven centric but maven can substitute versions when  
constructing a geronimo-plugin.xml.  I'm sure there are other ways.

I may not be understanding what you are saying.  It seems to me that  
what we're talking about is basically turning the lists of plugins now  
in the assembly poms into independent artifacts that can be used by  
the car-maven-plugin to assemble servers, and also from other spots  
such as gshell and maybe the console.  We're can do this today with  
the existing plugin mechanism although it has a few bad side effects  
which we should be able to eliminate pretty easily.

thanks
david jencks




Thoughts?
Jay
Lin Sun wrote:
Here is what I am thinking.   Let me take the Web profile as an  
example:

So we want to allow users to check/select the Web profile to select
all the necessary geronimo plugins for little G.   Users would only
see Web profile, instead the 10+ geronimo plugins.

-------------------------------------------------
Select from the following Profiles/Plugin Groups:
__ Web (when this selected, we'll install the 10+ geronimo plugins  
for
the user to get little G env.)
__ Web Service
...
-------------------------------------------------

In order to do this, we'll need to know which geronimo plugins can  
get
the users to the Web profile and store this relatonship somewhere  
that
is avail for both admin console and command line custom  
assembly.   I
come to the conclusion that we need some sort of group of plugins
function and David reminded me about the geronimo-plugin.xml that  
has
no module-id can work as group of plugins.   Here is the wording  
from
the schema:

If no module-id is provided, that means this is a plugin group,  
which
is just a list of other plugins to install.

With that, I can just build a geronimo plugin group for web profile
and have the 10+ geronimo plugins listed as dependencies.   This
geronimo plugin group can be available as part of the assmebly,  
along
with the other geronimo plugin groups.

The idea is that if a user selects Web profile in either admin  
console
or command line, we can just select the corresponding geronimo  
plugin
group behind the scene, which would install all its dependencies.

Now back to the web services sample, we 'll have 2 web service  
plugin groups:

web service CXF - cxf and cxf-deployer
web service Axis2 - axis2 and axis2-deployer

The web service Jetty plugin group will be included in the jetty
javaee5 assembly and web service tomcat plugin group will be  
included
in the tomcat javaee5 assembly.   Initially, I plan to only support
custom server assembly from the current local server, so when user  
has
jetty assembly, he will see web service CXF.   When user has tomcat
assembly, he'll see web service Axis2.   In the long run, we could
present both to the users and they can just pick either one.

I hope above addressed your questions.   Please feel free to let me
know any other comments you may have.

Lin

On Thu, Aug 21, 2008 at 11:57 PM, Jarek Gawor <[EMAIL PROTECTED]>  
wrote:
Hmm.. I'm not sure how this profile idea fits in with what the user
have to select in the "assemble a server" portlet. Would there be a
profile for axis2 that only has two plugins axis2 and axis2- 
deployer
defined? And there would be a similar profile with two plugins for
cxf? And the user would either pick the axis2 or cxf profile and
combine it with the jetty or tomcat profile? I'm just not sure how
this relates to the steps the user would have to go through in the
portlet to create the desired server.

Jarek

On Tue, Aug 19, 2008 at 4:20 PM, Lin Sun <[EMAIL PROTECTED]>  
wrote:
I have been thinking a bit more on how we achieve this.   Here  
is my
idea and I welcome your input -

So we have a need to allow users to install groups of  
plugins(function
profile), instead of individual plugins.   Install individual  
plugins
are nice for standalone apps, but for system modules, I think it  
would
be better to allow users to install groups of plugins as  
functional
profiles(unless the user is an expert user).    What we need is to
expose the groups of plugins for certain functions available to  
our
users and allow them to select the ones of their interest to  
build the
customer server.

I am proposing in addition to store plugin metadata of each  
plugin in
the plugin catalog, we could also host installable groups of  
plugins
information there (or in a separate catalog file).   For  
example, for
a function such as Web (same as little G) that has been  
discussed in
above posts, we could have the following plugin metadata -

<geronimo-plugin xmlns="http://geronimo.apache.org/xml/ns/plugins-1.3 
"
xmlns:ns2="http://geronimo.apache.org/xml/ns/attributes-1.2";>
  <name>Geronimo Assemblies :: Minimal + Tomcat</name>
  <category>WEB Profile</category>
  <profile>true</profile>
  <description>A minimal Geronimo server (Little-G) assembly using
the Tomcat web-container.</description>
  <url>http://www.apache.org/</url>
  <author>Apache Software Foundation</author>
  <license osi-approved="true">The Apache Software License,  
Version
2.0</license>
  <plugin-artifact>
      <module-id>
          <groupId>org.apache.geronimo.assemblies</groupId>
          <artifactId>geronimo-tomcat6-minimal</artifactId>
          <version>2.2-SNAPSHOT</version>
          <type>car</type>
      </module-id>
      <geronimo-version>2.2-SNAPSHOT</geronimo-version>
      <jvm-version>1.5</jvm-version>
      <jvm-version>1.6</jvm-version>

      <dependency>
          <groupId>org.apache.geronimo.assemblies</groupId>
          <artifactId>geronimo-boilderplate-minimal</artifactId>
          <version>2.2-SNAPSHOT</version>
          <type>jar</type>
      </dependency>

      <dependency start="false">
          <groupId>org.apache.geronimo.framework</groupId>
          <artifactId>upgrade-cli</artifactId>
          <version>2.2-SNAPSHOT</version>
          <type>car</type>
      </dependency>

      <dependency start="true">
          <groupId>org.apache.geronimo.framework</groupId>
          <artifactId>rmi-naming</artifactId>
          <version>2.2-SNAPSHOT</version>
          <type>car</type>
      </dependency>

      <dependency start="true">
          <groupId>org.apache.geronimo.framework</groupId>
          <artifactId>j2ee-security</artifactId>
          <version>2.2-SNAPSHOT</version>
          <type>car</type>
      </dependency>

      <dependency start="true">
          <groupId>org.apache.geronimo.configs</groupId>
          <artifactId>tomcat6</artifactId>
          <version>2.2-SNAPSHOT</version>
          <type>car</type>
      </dependency>
...

When a plugin is a profile, it means it just contains a group of
geronimo plugin dependencies that are installable and can perform
certain functions.  By installing it, it will simply install the
dependency plugins.

Questions -

How do we build this profile type of plugin?   We could build them
manually initially to try things but maybe c-m-p could be used  
here.
How do we install this profile type of plugin?  I think we could
leverage the pluginInstallerGBean to install it...when profile is
true, we just download the dependencies.
How/Where should we make this file avail?   We could make this  
file
avail in geronimo-plugins.xml (or another catalog file in repo)  
and
with our server assembly (one assembly contains the plugin  
profiles it
have).  When building customer server, when load all the plugins  
that
are profile and ask users to pick which ones they want.   If we  
have a
framework that can install geronimo plugins, a user can just  
download
the framework and pick from our apache repo on which plugin  
profiles
they want to build their geronimo server.
How are we handle the upgrade scenarios Joe mentioned?   No idea
yet... I think this is a rather complicated scenario.

Thanks,

Lin