Hi, everyone,
The Module Loader idea is very attractive. With Module Loader, it may
be easy to implement server provisioning APIs, so that Drizzle database
resources could be dynamically allocated/provisioned and
deallocated/de-provisioned from various cloud administration tools.
Thanks,
Alex Esterkin
Eric Day wrote:
Hi everyone!
After looking through much of the drizzled config parsing and plugin
code, I decided I would throw out some ideas of ways we might improve
them to be more modular and maintainable in the future. I've included
both config and plugin modifications since they are closely related
(as I'll describe below). I'll be using the word module to describe
a shared object that can be loaded vi dlopen(), and plugin will be
an interface hook (for example, a single module may register multiple
plugins). There are some fairly simple changes we can make that would
give us the ability to support the following features;
* Config option loading for dynamic modules (allowing us to get rid of
sql_builtin).
* The ability for a single module file to register multiple plugin types.
* Easy to upgrade/modify the plugin APIs.
* Plugin/module dependency tracking.
* Dynamic code upgrades while keeping state. For example, InnoDB could
keep the buffer pool in memory while the module was reloaded,
keeping the cache warm.
* Easier to stack, skip, or replace core modules within drizzled.
Config Options
>From what I can see, config options are only available to the
drizzled core and builtin plugins that register system vars. What I
propose is to have the drizzled core continue parsing the options,
but instead it would create an empty module placeholder and a list
of given config options it found. These will be used later on when
the module is actually loaded (or possibly never if the module is
never loaded). This would allow us to be able to remove the built-in
plugin list, and provide ways to dynamically add module vars through
config file reload or queries. If there are any errors with processing
config options during a module load, it will just report an error
and not load.
Module Loader
What we could do is move away from the traditional plugin system
and make a more generic dynamic module loading system. This means
making Drizzle look more like a the Linux kernel as far as loading
modules goes. A module, unlike a traditional plugin, will no longer
be responsible for a single plugin type, but instead be able to
register plugin hooks wherever it likes during the init() function
(or perhaps any other time). This would allow us to keep the plugin
APIs out of the structs that are mapped in via dlsym, and can use a
class interface making it easier to maintain in the future. A module
would only need to provide a struct with:
* Name
* List of other module dependencies
* List of all possible config options (like system_vars in old plugin
struct)
* Init function, something like: drizzled_return_t init(void *state);
* Deinit function, something like: const void *deinit(bool reload);
When the drizzle core loads a module, it will dlopen(), dlsym() the
above struct, and then verify any config options it has loaded against
the list provided. If all options are ok, it will then call init() with
state=NULL, at which time the module will use the various plugin APIs
to register one or more plugins where needed (UDFs, scheduler, ...).
To reload a module, the module loader would unload all child modules,
then the module requesting the reload. It would call deinit(true)
for each, which tells the module it can return a void * which will be
passed back into init at the end of the reload. This allows the modules
to keep state between code reloads (for example, InnoDB buffer pool).
Having the dependency tracking also helps during shutdown, since we can
unload modules in a sane order. The modules list looks like a tree at
this point (again, think Linux kernel modules) rather than a flat list.
Taking this one step further, it would be great to work towards having
the drizzle core itself be a module (loaded first of course). We would
the chip away at this, breaking it up into logical pieces until the
entire server is just a series of modules. At this point it makes
it very easy for any module to be replaced or tuned, or allowing
drizzled to be used in a variety of embedded ways. This also makes
the code base even more approachable for new developers, and gets
back to making Drizzle a true micro-kernel DBMS.
Or perhaps I just had an unhealthy obsession with Legos as a kid and
I want to turn everything into building blocks. :)
-Eric
_______________________________________________
Mailing list: https://launchpad.net/~drizzle-discuss
Post to : [email protected]
Unsubscribe : https://launchpad.net/~drizzle-discuss
More help : https://help.launchpad.net/ListHelp
_______________________________________________
Mailing list: https://launchpad.net/~drizzle-discuss
Post to : [email protected]
Unsubscribe : https://launchpad.net/~drizzle-discuss
More help : https://help.launchpad.net/ListHelp
