Hi everyone! After looking through much of the drizzled config parsing and plugin code, I decided I would throw out some ideas of ways we might improve them to be more modular and maintainable in the future. I've included both config and plugin modifications since they are closely related (as I'll describe below). I'll be using the word module to describe a shared object that can be loaded vi dlopen(), and plugin will be an interface hook (for example, a single module may register multiple plugins). There are some fairly simple changes we can make that would give us the ability to support the following features;
* Config option loading for dynamic modules (allowing us to get rid of sql_builtin). * The ability for a single module file to register multiple plugin types. * Easy to upgrade/modify the plugin APIs. * Plugin/module dependency tracking. * Dynamic code upgrades while keeping state. For example, InnoDB could keep the buffer pool in memory while the module was reloaded, keeping the cache warm. * Easier to stack, skip, or replace core modules within drizzled. Config Options >From what I can see, config options are only available to the drizzled core and builtin plugins that register system vars. What I propose is to have the drizzled core continue parsing the options, but instead it would create an empty module placeholder and a list of given config options it found. These will be used later on when the module is actually loaded (or possibly never if the module is never loaded). This would allow us to be able to remove the built-in plugin list, and provide ways to dynamically add module vars through config file reload or queries. If there are any errors with processing config options during a module load, it will just report an error and not load. Module Loader What we could do is move away from the traditional plugin system and make a more generic dynamic module loading system. This means making Drizzle look more like a the Linux kernel as far as loading modules goes. A module, unlike a traditional plugin, will no longer be responsible for a single plugin type, but instead be able to register plugin hooks wherever it likes during the init() function (or perhaps any other time). This would allow us to keep the plugin APIs out of the structs that are mapped in via dlsym, and can use a class interface making it easier to maintain in the future. A module would only need to provide a struct with: * Name * List of other module dependencies * List of all possible config options (like system_vars in old plugin struct) * Init function, something like: drizzled_return_t init(void *state); * Deinit function, something like: const void *deinit(bool reload); When the drizzle core loads a module, it will dlopen(), dlsym() the above struct, and then verify any config options it has loaded against the list provided. If all options are ok, it will then call init() with state=NULL, at which time the module will use the various plugin APIs to register one or more plugins where needed (UDFs, scheduler, ...). To reload a module, the module loader would unload all child modules, then the module requesting the reload. It would call deinit(true) for each, which tells the module it can return a void * which will be passed back into init at the end of the reload. This allows the modules to keep state between code reloads (for example, InnoDB buffer pool). Having the dependency tracking also helps during shutdown, since we can unload modules in a sane order. The modules list looks like a tree at this point (again, think Linux kernel modules) rather than a flat list. Taking this one step further, it would be great to work towards having the drizzle core itself be a module (loaded first of course). We would the chip away at this, breaking it up into logical pieces until the entire server is just a series of modules. At this point it makes it very easy for any module to be replaced or tuned, or allowing drizzled to be used in a variety of embedded ways. This also makes the code base even more approachable for new developers, and gets back to making Drizzle a true micro-kernel DBMS. Or perhaps I just had an unhealthy obsession with Legos as a kid and I want to turn everything into building blocks. :) -Eric _______________________________________________ Mailing list: https://launchpad.net/~drizzle-discuss Post to : [email protected] Unsubscribe : https://launchpad.net/~drizzle-discuss More help : https://help.launchpad.net/ListHelp
