-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Eric Day wrote:
> Hi everyone!
>
> After looking through much of the drizzled config parsing and plugin
> code, I decided I would throw out some ideas of ways we might improve
> them to be more modular and maintainable in the future. I've included
> both config and plugin modifications since they are closely related
> (as I'll describe below). I'll be using the word module to describe
> a shared object that can be loaded vi dlopen(), and plugin will be
> an interface hook (for example, a single module may register multiple
> plugins). There are some fairly simple changes we can make that would
> give us the ability to support the following features;
>
> * Config option loading for dynamic modules (allowing us to get rid of
> sql_builtin).
Excellent.
> * The ability for a single module file to register multiple plugin types.
Yep.
> * Easy to upgrade/modify the plugin APIs.
Cool, but I'd like to hear more details about this...
> * Plugin/module dependency tracking.
Sure, but I consider this a "phase 2" type thing...
> * Dynamic code upgrades while keeping state. For example, InnoDB could
> keep the buffer pool in memory while the module was reloaded,
> keeping the cache warm.
Again, phase 2 type of thing. The reason I say this is because with the
current code base (or any large project really) it's incredibly easy to
go down a rathole and try to bite off more than what's feasible. My
best advice would be to start small and slowly improve on the interfaces
instead of a big-bang approach. Just some advice from what I've learned
over the past 9 months... :)
> * Easier to stack, skip, or replace core modules within drizzled.
Could you be more specific on what you mean by "stack"? Do you mean a
module which builds on top of another, or do you mean a "module
registry" which allows you to push and pop modules in a stack-like way?
> Config Options
>
>>From what I can see, config options are only available to the
> drizzled core and builtin plugins that register system vars. What I
> propose is to have the drizzled core continue parsing the options,
> but instead it would create an empty module placeholder and a list
> of given config options it found. These will be used later on when
> the module is actually loaded (or possibly never if the module is
> never loaded). This would allow us to be able to remove the built-in
> plugin list, and provide ways to dynamically add module vars through
> config file reload or queries. If there are any errors with processing
> config options during a module load, it will just report an error
> and not load.
Hmmm, ok. Sounds like what you want is a callback which allows the core
to ask the module to provide the core with a vector of configuration
variables when loaded. To do this, really we need to rework the way
that "system variables" are constructed and queried in the core.
Right now, system variables are stored in a map<>, and each system
variable is a derived type from sys_var (see drizzled/set_var.cc).
Plugins can register a system variable with the DRIZZLE_SYSVAR_{TYPE}
macros in drizzled/plugin.h but the system is quite difficult to
understand internally. I like your idea of having module configuration
variables loaded when the module is actually loaded (and not before),
but I'm not sure what you mean by "able to remove the built-in plugin
list". Could you expand on that?
I think it would be useful for a module to be able to tell the drizzled
core where it might find configuration files for the plugin, and even
have the drizzled core by default look in a section named for the plugin
in the main drizzled configuration files, too.
> Module Loader
>
> What we could do is move away from the traditional plugin system
> and make a more generic dynamic module loading system. This means
> making Drizzle look more like a the Linux kernel as far as loading
> modules goes. A module, unlike a traditional plugin, will no longer
> be responsible for a single plugin type, but instead be able to
> register plugin hooks wherever it likes during the init() function
> (or perhaps any other time).
Yes! And technically, there's no reason to have "plugin types" in my
opinion. The only thing that is necessary is to have a *well-defined*
and documented list of events which a module (or plugin, or some class,
or my WidgetyGadgetyDoodah) can register as a listener for.
Basically, instead of having eleventy-billion types of plugins, we can
simply have a published set of events which anything can listen to, and
if it needs to, modify the state of a supplied state parameter.
I'm, of course, talking about an Observer/Observable design pattern,
where instances of any class may "observe" the state of an "Observable"
class. An "event" would be the observable class, and a plugin (or
function, or some class) would be the observer. When the event "fired",
all observers would be notified of a state change in the event and could
respond accordingly (or ignore it if they wanted).
Assume this simple interface:
class EventStack;
class Event
{
public:
enum EventId
{
ON_CREATE_NEW_SESSION
, ON_DESTROY_SESSION
, ON_BEFORE_PARSE_STATEMENT
, ON_AFTER_PARSE_STATEMENT
, ON_BEFORE_OPTIMIZE_STATEMENT
, ON_AFTER_OPTIMIZE_STATEMENT
//, etc, etc..
};
protected:
/**
* Stores state about the event -- allows multiple listeners to
* "chain together", modifying the passed state of the event as
* each listener receives notification.
*/
void *_state;
/** Array of listening observers */
std::vector<Observer *> _observers;
EventId _id;
public:
/**
* Construct an event with a pointer to the state which the event
* maintains between event notifications.
*/
Event(void *in_state);
/** Adds an observing class as a listener to the event */
void addListener(Observer *in_observer);
/** Notifies listeners of a state change in the event */
void notify();
friend class EventStack;
};
class EventStack
{
private:
std::vector<Event *> _events;
public:
notify(Event::EventId id, void *state);
push_back(Event *e);
}
class Observer
{
public:
virtual void respond(void *in_state)= 0;
};
So, basically an observer class is any class which implements the above
pure virtual respond() method...
In a simple implementation, one could have:
Event::Event(void *in_state) :_state(in_state) {};
void Event::addListener(Observer *in_observer)
{
observers.push_back(in_observer);
}
void Event::notify()
{
std::vector<Observer *>::iterator p= _observers.begin();
while (p != _observers.end())
(*p)->respond(_state);
}
EventStack::notify(Event::EventId id, void *state)
{
std::vector<Event *>::iterator p= _events.begin();
while (p != _events.end())
{
if ((*p)->_id == id)
(*p)->notify(state);
}
}
EventStack::push_back(Event *e)
{
_events.push_back(e);
}
Of course, the above is incredibly simple and just meant to demonstrate
the concept...
And here is how it might be called...
Let's say we have the following:
class OnCreateNewSession :public Event
{
private:
const Event::EventId _id= ON_CREATE_NEW_SESSION;
public:
OnCreateNewSession(SessionList *s) {_state= (void *) s;}
}
class OnBeforeParseStatement :public Event
{
private:
const Event::EventId _id= ON_BEFORE_PARSE_STATEMENT;
public:
OnBeforeParseStatement(Statement *s) {_state= (void *) s;}
}
In our global namespace, the drizzled core has a set of
globally-applicable events, called global_events:
EventStack global_events;
which, during drizzled startup, we populate with a number of events,
like so:
global_events.push_back(
new OnCreateNewSession(global_session_list)
);
global_events.push_back(
new OnDestroySession(global_session_list)
);
and we have the following new member of the Session class:
EventStack session_events;
which is populated in the Session constructor and at certain points in
the session's lifetime with events, like so:
Session::Session()
{
session_events.push_back(new OnBeforeParseStatement(this));
}
and we have a couple classes defined in Toru's pluggable_parser module:
class PreParseInterceptor :public Observer
{
// lots of other stuff...
respond(void *event_state)
{
Statement *s= (Statement *) event_state;
// Do something with the statement...
}
}
class NewSessionInterceptor :public Observer
{
// lots of other stuff...
respond(void *event_state)
{
SessionList *s= (SessionList *) event_state;
Session *sess= s->findCurrentSession();
pre_parse_interceptor= new PreParseInterceptor();
sess->session_events.addListener(pre_parse_interceptor);
}
}
static NewSessionInterceptor new_session_interceptor;
int parser_module_init()
{
new_session_interceptor= new NewSessionInterceptor();
global_events.addListener(
ON_CREATE_NEW_SESSION
, new_session_interceptor
);
We could then have the following in the Session:
Session::readAndSaveQuery(const char *query, size_t query_len)
{
session_events.notify(ON_BEFORE_PARSE_STATEMENT, this);
// The normal stuff...
session_events.notify(ON_AFTER_PARSE_STATEMENT, this);
}
In this way, we detach the notion of a plugin having a specific "type"
and simply have the plugins "respond" to various events which are going
on...
We also detach ourselves from the plugin_foreach() stuff, since the
*event* now handles *pushing* it's notifications out to the plugins
which registered as listeners for it.
Some events which occur on a global level would be responded to by a
plugin's global listeners (like the new_session_interceptor static
variable in the parse plugin above) and others, such as the
pre_parse_interceptor variable, would be localized listeners which could
die at the end of a session's lifetime...
In this way, we no longer have to keep track of a FULLTEXT plugin type,
or a SCHEDULER plugin type, or an ERRMSG plugin type.
We simply have a list of events which can occur in the system (the
Event::EventId enum) and modules, classes, plugins, or whatever, simply
listens and responds to those events when they get fired.
> <snip>
>
> Or perhaps I just had an unhealthy obsession with Legos as a kid and
> I want to turn everything into building blocks. :)
Heh, I had the same obsession... :)
Cheers,
Jay
> -Eric
>
> _______________________________________________
> Mailing list: https://launchpad.net/~drizzle-discuss
> Post to : [email protected]
> Unsubscribe : https://launchpad.net/~drizzle-discuss
> More help : https://help.launchpad.net/ListHelp
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iEYEARECAAYFAknFBa8ACgkQ2upbWsB4UtEKPQCdEWumTftHLpi+0Gjf6/tfdxY1
6MMAn02+3aHzYS77cxvhuU4HIxBw2JJd
=7wcS
-----END PGP SIGNATURE-----
_______________________________________________
Mailing list: https://launchpad.net/~drizzle-discuss
Post to : [email protected]
Unsubscribe : https://launchpad.net/~drizzle-discuss
More help : https://help.launchpad.net/ListHelp
