On Dec 15, 2009, at 4:11 PM, Tom Davis wrote:
>> we aren't really using them for anything
>
> Well, they *should* be used to validate a driver, as was the original
> intent. If this isn't implemented it would be trivial to add tests for this
> purpose. It also allows us to automate discovery / validation of drivers
> rather than manually list them out in some module that needs to be edited
> each time one is added or removed (which was also an initial intention
> though meant to be added later on).
Since providers are added every now and then instead of every day, I think
manually specifying which providers are included is not that big of a deal.
As for autodiscovery of drivers within each provider, that is what subclasses
and introspection are for -- introducing an entire abstraction layer with
Zope is not the best solution for this task.
If I were designing such a system from scratch, it'd look something like:
base.py:
class Driver(object):
pass
providerA.py:
import base
class ProviderADriver1(base.Driver):
def hello(self):
print "Hello from driver 1 in provider A!"
class ProviderADriver2(base.Driver):
def hello(self):
print "Hello from driver 2 in provider A!"
class UnrelatedObject(object):
def hello(self):
print "Oh noes :("
providers.py:
import providerA
loading_crap.py:
import base
import providers
def toplevel(mod):
for i in dir(mod):
if i[:2] != "__":
yield i
def interested(mod, obj):
try:
item = eval("providers.%s.%s" % (mod, obj))
return item if issubclass(item, base.Driver) else None
except: return None
def main():
drivers = []
for prov in toplevel(providers):
possible = toplevel(eval("providers.%s" % prov))
drivers.extend(interested(prov, i) for i in possible)
for driver in drivers:
if not driver:
continue
else:
d = driver()
d.hello()
if __name__ == "__main__":
main()
11:40 jsm...@upsidedown% python loading_crap.py
Hello from driver 1 in provider A!
Hello from driver 2 in provider A!
Very rough, but I've taken this approach with my software in the past. This
is the real lazy approach, even -- more maintainable and less prone to
programmer error is to have a specific member that the loader code looks for,
which enumerates the drivers the plugin expects the loader to load.
Python's introspection capabilities make it shine in this regard, allowing
really easy stuff which took me weeks in .NET -- I just wrote that example
above in about 20 minutes. You don't need a heavyweight import like Zope to
get that done, and the arguments I'm reading in this thread for documentation
and testing just aren't selling me on it.
Plus, look how easy providerA's interface was to write.
> I can't argue that now isn't a good time, but I still feel they are (and can
> be made more) useful. For one, they can assist in getting rid of the *
> providers.py* file entirely. A simple metaclass which functions as a mount
> coupled with interface verification (we don't want to provide a driver that
> doesn't yet implement the API) fixes the need for manually listing /
> commenting drivers, not to mention the ugly static paths / names that are
> going on there.
That would introduce a hurt locker full of complexity for little to no gain.
Static things in code, particularly just a list of providers that the library
supports, are nothing to be afraid of and are far from ugly.
How satisfying is it to a committer to see their work's name added to "the
list"? That's almost a signature of we-accept-this-module. Since libcloud
is a product which is, for the most part, centrally-controlled, a lot of the
automated discovery stuff is for no benefit whatsoever as /we/ are in control
of what goes out in the repository. Developers making a module can add their
own, and that's not terribly difficult -- having libcloud automagically say
"hey! I see your new provider and plugins within!" is a novelty, at best,
for an extraordinarily small use case.
Honestly, and perhaps a little off-topic, I haven't liked Zope since the
beginning of time because some of the approaches they take with Python really
go against, in my mind, what Python is all about. Python was designed with a
lack of interfaces on purpose, and munging them isn't helping things.
(However, I am biased toward Django -- which performs unspeakable horrors...)
> I think it was useful in the beginning just to vet
> plugins as they were all very new and disparate in their implementation. If
> everybody is happy manually verifying and exposing drivers (despite it being
> a far larger pain in the ass, imo) then by all means...
I disagree that it's a larger pain in the ass. Pulling such a heavyweight
dependency in via setuptools or forcing people to install it just to use the
product is the pain in the ass to me; if it did my dishes I might consider it,
but the amount of gain awarded is disproportionate to the effort put in.
JS
