Hi Berin,
> -----Original Message-----
> From: Berin Loritsch [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, January 09, 2002 5:10 PM
>
> Paulo Gaspar wrote:
>
>
> > ...
> >
> > The SingleThreaded components I have that are not pooled are very light
> > anyway. In the whole picture of the many objects created and destroyed
> > to process a request they are not a representative extra load for the
> > garbage collector.
>
>
> Modern JVM's are very efficient when all you are doing is creating a new
> instance of a small final class (as close as Java will ever get to a C
> 'struct'). The problem occurs when the cost of creating a new Component
> also involves running it through a lifecycle, or it is an expensive
> resource like a Database Connection object.
That was my understanding too.
> > If it is a "ComponentHandler", I keep having those. My names
> are different
> > but you could call them:
> > - ThreadSafeComponentHandler
> > - SingleThreadedComponentHandler
>
> Yup. That's what I have as well.
I copied from Avalon, remember?
=;o)
I just changed on thing in that part of the structure: I have both
an AbstractComponentHandler and a ComponentHandlerFactory.
When I was trying to understand Avalon I found confusing having together
in the AbstractComponentHandlre both:
1) The code that creates the right ComponentHandler for a Component
depending on its LifeStyle;
2) The common code (mostly life cycle management) for all
ComponentHandler implementations.
I think that 1) belongs elsewhere. I really think you should change that
bit as I did, so that:
- AbstractComponentHandler has common lifecycle code (initialization,
disposal, etc.) to all ComponentHandlers;
- ComponentHandlerFactory implements the strategy to create the right
ComponentHandler implementation.
This also makes this code more reusable in case some one comes up with
some new (and probably weird) component life style.
> > Poolable objects have their pool wrapped by a factory and are handled by
> > the SingleThreadedComponentHandler.
>
> A pool wrapped by a factory? How does that work? I am used to seeing
> a Pool use a Factory to generate instances. If you are referring to the
> Factory pattern (Design Patterns, GoF), then I can see the Pool being
> part of the "factory".
>
> I think this may be the cause of my confusion.
It is as you say.
Since I force having a ComponentFactory for each role/type registered in
the ComponentManager, for poolable objects I end up having:
- A ComponentFactory (facade?) that creates, configures and privately
holds some pool implementation, and uses that pool to act as a factory
of "SingleThreaded" (pooled) components;
- That (ComponentFactory) private Pool implementation, which usually
must be parameterized with a...
- ...pool object factory.
> > ...
> >
> > That is too complex for me to consider at the moment but I am
> very curious
> > about those new dynamic developments, like the with run-time management/
> > reconfiguration and saving the current configuration back to disk.
> >
> > Probably I will end up finding some use for those ideas too.
> > =:o)
>
>
> I am still formulating the plans in my head.
Having already considered that kind of functionality, I sure have a
(maybe still vague) idea of how tricky that is.
> >>Perhaps this goes beyond what you are thinking of. However,
> such a managed
> >>system is able to make expensive decisions about the most
> efficient way to
> >>allocate resources asynchronously from the rest of the system. That way
> >>latencies involved in lookup and releasing of components are
> minimized to
> >>simple get() and put() calls on a Map.
> >>
> >
> > This paragraph is not clear to me.
> > Why will that improve latencies so much?
>
>
> Because the management is no longer performed synchronously.
> Instead of managing
> pool sizes as objects are requested and returned to the pool,
> they are managed by
> an asynchronous process. That asynchronous process will have the
> intelligence to
> predict the optimal pool size so that all get() requests pass
> without wasting resources.
Ah! The kind of adaptive caching I recently discussed with Stefano
on the Cocoon list!!! I love that!
> Also, for a cache implementation example, the asynchronous
> process will check the
> validity of the entries in the Cache, and automatically purge
> stale entries. That
> way the Cache does not have to perform that check synchronously.
But any cache usually has a "janitor" (using a Cocoon term) to do
that.
> When management of resources are moved outside of the critical
> path of execution,
> the critical path is shorter and simpler.
> Shorter and simpler == faster.
Ok, of course.
This kind of things I have already been wondering about and you
can do most without changing the ComponentManager.
I thought you had some new black magic!
=;o)
The interesting new bit at Avalon is already the ability to
serialize back to disk the new/adapted configuration of such
adaptive components.
=:o)
The most interesting bit of dynamic management, for me, is
interaction trough some user interface tool. Looking at metrics,
changing parameters on the run and saving the configuration
with beautiful and easy to use tools... what a nice dream!
=:oD
> >>> My "ComponentHandler" still takes care of lifecycle management
> >>> operations and strategies, like:
> >>> - Initializing, starting, stopping, disposing, etc.
> >>> - Using only one instance for thread safe components and
> >>> always a new instance for the others.
> >>>
> >>
> >>That is the implementer's perogative on how they want to manage
> >>their system
> >>or define Components. However, if your system does not work with
> >>any Avalon
> >>Component, then you have lost the ability to use a number of well tested
> >>Components. Hopefully you have not lost that ability.
> >>
> >
> > Most of my components are not from the Avalon universe, hence many
> > of my choices.
>
> I see.
And I think that Avalon should be a bit more open ( = make it easier)
to those external components. But I already told you this.
> > Even with components that exist at Avalon, sometimes I needed
> > something different. I ended up building my own pool an my own
> > connection pool in order to have better control over its behavior
> > and to better control its resources.
>
> Hmmm. What additional control do you need?
I looked at several implementations and I am not 100% sure of what I
specifically missed on Avalon. The features I was missing on most
implementations I investigated were:
- Controlling the pace the pool expands and then shrinks back;
- Resource management.
By resource management, I mean closing all open ResultSets/Statements
for a Connection being returned to the pool, and I am sure that Avalon
pool implementation was missing this one.
At least the Statements must be closed. The JDBC standard demands that
when a Statement is closed its ResultSets are closed too. In my pool I
just wrap the ResultSet too to be extra sure I do not get into trouble
if a get a driver that does not conform to that. (I want to make it
optional and parameterized this ResultSet wrapping bit.)
I still miss having a client-side DataSource, but that is a feature
that many people will not miss. It works like this:
- The component you get from the ComponentManager can be a DataSource
and not a Connection;
- You can get all the Connections from the pool trough this client
DataSource;
- This client DataSource has a (non standard) close() method that
can be used to return all the Connections obtained trough it to
the pool and to terminate its use.
> >>It seems like alot of work for a Component writer to create a
> new factory
> >>for each Component. I beleive that slows development time
> unnecessarily.
> >>By having one generic ComponentFactory, you can focus on more important
> >>issues like Component interaction, and how it should handle when
> >>Components are not available.
> >>
> >
> > Actually not. The component writer just ends up moving the
> configuration
> > code from the component to its factory, and thanks to the
> auto-configuration
> > functionality, that code is much simpler.
> >
> > Since that code works based on introspection (like in Ant, as I already
> > mentioned) I even use a separate bean (or just an object with fields) to
> > be introspected and receive the configuration information.
> >
> > I end up with 3 classes:
> > - The component;
> > - The factory;
> > - The configuration bean which is usually an inner class of
> the factory.
> >
> > But overall the code is much simpler/shorter than typical Avalon code.
> >
> > And I can use 3rd party / isolated components without modifying them.
>
>
> Can you post a code example. That might help me understand where you are
> coming from.
It is at the end of this mail.
> ...
> >
> > I think that just implementing the Component interface makes no much
> > difference. It is just a marker.
>
> Probably, and Peter surprised me when he mentioned he was in favor of
> removing Component.
He sure surprised me too!
=:o)
I think the Component interface is one of those ideas that look great
and useful before you start using them. Sometimes you have to experiment
a lot before you find out what really works.
I know I am repeating something that everybody here is aware of. This
comes up in many threads I have followed.
=:o)
> > Maybe we are talking about different things?
> >
> > However, I have a role interface which ends up being validated, of
> > course. Maybe that is what you mean.
>
> When I talk about a Component's interface, this is precisely what I
> am referring to.
AH! Cool! Then we agree.
But, again, that was something I learned from Avalon.
> > My roles configuration xml format is quite different but it includes
> > both the factory class and the role class, this last one being the
> > interface of the object built by the factory.
>
> That is only a configuration issue.
Yes.
This is another point where it shoes that I am still learning your rank
of values - the things you consider core and those you consider
accessory. It all makes sense for me until now.
> > ...
> >
> > However, I still see the Selectors as a way of having more than one
> > implementation for the same role, but I have a different problem: I
> > have different implementations and different instances.
>
>
> The ComponentSelector (in Excalibur) works exactly like the
> ComponentManager.
> In effect, you have different instances of the same implementations (even
> with different configurations per instance). It works like what you need.
> You just don't like the API :).
I got lost several times with Excalibur. I think I got confused several
times also because of the configuration samples I found.
One thing that is becoming obvious to me is that ALSO with the Component
Manager you should move configuration reading to a separate class. The Gof4
Strategy pattern, me thinks.
Then the ComponentManager/RolesManager implementations should expose an API
to add new Components/Roles to that configuration strategy pattern.
But I am still thinking about this one...
If both Avalon/Excalibur and my understanding of it will evolve to the point
I can use it, I will. I want to reduce the code I take care of (75 Kloc!). I
would just place its ComponentManager inside mine, to keep the convenient
facade I have.
> > VERY interesting idea.
> >
> > The query object can even use a task/request Context for the locale/user
> > data.
>
> Precisely :)
>
> This is what makes the concept so powerful.
I like very much that Query parameter idea.
=:o)
> > ...
> >
> > The other big difference is the "token" based resource management. The
> > one that allows me to ensure that all components requested for a task
> > are released at its end:
> >
> > // start processing request
> > obj1 = cm.get(token, role1, name1);
> > obj1 = cm.get(token, role2, name2);
> > obj1 = cm.get(token, role3, name3);
> > ....
> >
> > // finish processing request
> > cm.release(token);
>
>
> What are your perceived advantages to using the Token? I say perceived,
> because in order for them to be considered real they have to be proven.
> Many times, perceived advantage is just as important as real advantage.
>
> However, we may find that token is a step in the right direction, or
> maybe not. I just want to know why you happen to like it.
This token idea probably makes no sense for many other applications, but
in my case it is the most practical way of ensuring that no resources are
lost.
Most of the objects I expose to the scripted bits and to the templates
perform some housekeeping tasks. The idea is that you never have to close
anything and never have to do any cleanup in a script.
Typical situation:
- You get a SqlQuery object, you put some parameters and you place it in
a Velocity template context;
- When the template is processed, Velocity finds out that can get an
Iterator from this SqlQuery object in some "#foreach" directive and
that (getting the Iterator) automatically opens the necessary
Connection, Statement and ResultSet;
- When processing the "#foreach" goes over the last record in the
ResultSet, all the database objects are closed (ResultSet, Statement)
or returned to the pool (Connection).
But if there is some exception, maybe something stays open.
I do not want to force the scripter to know much. I do not even want
him to care about exceptions for most cases. The system will log them,
clean up and display an as-nice-as-possible error message.
The scripter does not even know that there is a token. That much is
hidden from him/her.
And it is possible to do so because the typical resource opens very
few resources, and this way I can postpone taking care of the cleanup
to the end of the request/task process.
Of course that the programmer that uses this ComponentManager has
other options. For complex parts Java will be used instead of script
and the traditional "try" blocks can and should be used.
For me the most convenient spot to track these resources is the
ComponentManager.
For each "SingleThreaded" component (which, as you know, includes the
poolable objects) requested, the ComponentManager creates an entry for
in a ComponentTracker, which is a "per-token" list of component
entries (there is obviously a Map of lists, but there is also a Map
of tokens per Component).
It is not necessary to return "ThreadSafe" components and I just
ignore those in order to make component tracking lighter. Since most
components I use are "ThreadSafe", this thing is still fast.
It the object is returned to the ComponentManager before the
"cm.release(token);" step at the end of the request/task processing,
its entry immediately removed from the ComponentTracker.
When the you get to that "cm.release(token);" call at the end of the
request/task processing, all components still not returned are
(returned) and the token entry (the List in the Map) is removed.
Of course that I can have a SafeComponentManager with a
ComponentTracker and an inner ComponentManager. This serves both use
models.
> >>>Notice that I followed a different approach than Cocoon, where the
> >>>sitemap has almost nothing to do with the ComponentManager.
> >>>
> >>>In my implementation, the sitemap is stored in the ComponentManager.
> >>>Overall, my implementation is much, much, much simpler.
> >>>
> >>Cocoon's Sitemap is stored in a ComponentManager as well. It just needs
> >>access to a large number of other Components.
> >>
> >
> > Ops! Where?
> > Is it the new tree walking stuff?
>
> No.
Where then? Do you happen to know?
=:o)
> > It was compiled, right? (I only looked at Cocoon2.)
>
> Yes. The Sitemap is a Component that happens to be Compiled and loaded
> dynamically by the ClassLoader. While it is being used, it behaves
> exactly like any other component.
But is/has the Sitemap a ComponentManager to store its structure?
AND NOW... the sample:
// Here goes a simple DB component factory.
// This one is missing a load of parameterization but makes a better
// example. I added some comments just for this mail!
// =:o)
import java.sql.Connection;
import javax.sql.DataSource;
import krankikom.sql.pool.IDbConnFactory;
import krankikom.sql.pool.dbcfactories.DriverManagerConnectionFactory;
import krankikom.sql.pool.PoolingDataSource;
import krankikom.sql.pool.DbConnectionPool;
import krankikom.activity.AbstractLoggableActivity;
import krankikom.log.ILogger;
import krankikom.framework.component.IComponentFactory;
import krankikom.framework.configuration.configurator.*;
public class DbFactory
extends AbstractLoggableActivity
implements IAutoConfigurable, IComponentFactory
// IAutoConfigurable exposes the autoConfigXXX() methods
// that the Configurator will use to "auto-configure" this
// factory.
{
// The IFieldBasedConfig marker tells the Configurator
// to use field introspection instead of the standard
// JavaBean introspection.
// (I am too lazy to find writting setXXX() methods
// interesting for this case.)
public class CfgBean implements IFieldBasedConfig
{
public String name;
public String driver;
public String url;
public String user;
public String password;
// Silly debug helper
public String toString()
{ return getClass().getName()
+ "{ " + fmtCell("*name", name)
+ fmtCell("*driver", driver)
+ fmtCell("*url", url)
+ fmtCell("*user", user)
+ fmtCell("*pwd", "*****")
+ " }";
}
}
public CfgBean cfg = new CfgBean();
private DbConnectionPool m_dbConnPool;
public DbFactory()
{ super();
}
//*** IComponentFactory stuff: ***
public Class createdClass()
{ return java.sql.Connection.class;
}
public boolean isComponentThreadSafe()
{ return false;
}
public Object newInstance()
throws Exception
{ return m_dbConnPool.newDataSource();
}
public void release(Object i_object)
throws Exception
{
if (null != i_object)
((PoolingDataSource)i_object).close();
}
//*** IAutoConfigurable stuff: ***
public Object autoConfigBean()
{ return cfg;
}
public void autoConfigInitialize()
{}
public void autoConfigPropertiesFinalize()
{
// I still want to see a sign that it is working!!! =;o)
System.out.println("CfgBean => " + cfg);
}
public void autoConfigFinalize()
{
}
//*** Life cycle ***
// This customXXX() are called by the lifecycle methods
// at AbstractLoggableActivity which implements the
// Initializable, Disposable, Startable and Logabble
// interfaces with a bunch of checks to make sure I am
// doing things right and also logs on "INFO" level when
// he performs the life cycle methods. UFFFfffff!
public void customInitialize()
throws Exception
{ IDbConnFactory dbConnFactory =
new DriverManagerConnectionFactory(cfg.driver, cfg.url, cfg.user,
cfg.password);
m_dbConnPool = new DbConnectionPool(logger(), dbConnFactory);
m_dbConnPool.setUseBorrowValidation(false);
m_dbConnPool.setUseIdleValidation(false);
m_dbConnPool.setUseReturnValidation(false);
m_dbConnPool.initialize();
}
public void customStart()
throws Exception
{ m_dbConnPool.start();
}
public void customStop()
throws Exception
{ m_dbConnPool.stop();
}
public void customDispose()
throws Exception
{ m_dbConnPool.dispose();
}
// Silly debug helpers
protected static String fmtCell(final String i_name, final String i_value)
{ return "[" + i_name + "=" + String.valueOf(i_value) + "]";
}
public String toString()
{ return cfg.toString();
}
}
--
To unsubscribe, e-mail: <mailto:[EMAIL PROTECTED]>
For additional commands, e-mail: <mailto:[EMAIL PROTECTED]>
