I know I'm arriving late to this thread, but I want to chime in with a resounding "yes!" to this approach.
We've held to a fairly strict policy around maintaining compatibility within the driver API since early on, and we'll continue to do that as we add new interfaces (see the new ManagementInterface) or enhance existing ones (like the new @clean_step decorator). I believe that, as a hardware abstraction layer, we need to enable authors of vendor-specific plugins/drivers to innovate out of tree -- whether that is through a driver + library, such as stackforge/proliantutils, or in a completely out of tree driver (I'm not aware of any open source examples of this yet, but I have heard of private ones). As far as the two (dis)advantages you list: 1) vendor drivers reviewed by the community On the one hand, I believe there is some benefit from the broader OpenStack community reviewing vendor code. Some... but not much. Architecture and general principles (is it scalable? are you doing something that operators will hate?) is valuable, but that's where it stops. As you pointed out, it is impractical to expect open source developers (or developers at other hardware companies) to have the knowledge and expertise necessary (or hardware available to them) to determine whether or not vendor-specific code will work with specific vendor hardware. As an interesting data point, we now have support for 8 different types of hardware drivers in tree (10 if you count the ssh and vbox drivers that we use in testing). 2) distros (not) packaging dependencies Because we can not actually test every driver's dependencies in the upstream gate (there's no hardware to exercise against), we do not install those packages in devstack-gate, and so we do not list driver's python dependencies in requirements.txt. We mock those libraries and test the in-tree driver code, and rely on the library and driver authors to ensure their fitness for use against real hardware. So, yes, third-party-CI is essential to this -- and I wish more vendors would step up and run CI on their drivers. But I digress... After talking with Thomas Goirand about debian packaging of Ironic, we realized that there wasn't a clear list of our driver's dependencies. So to provide distros with a single location for this information, we are now maintaining a "driver-requirements.txt" file. It's fairly new -- I added it about a month ago. See https://github.com/openstack/ironic/blob/master/driver-requirements.txt Thanks for writing up your experiences with this, Ramakrish. I believe it is a great example to other driver authors -- and it would be great to have a wiki or doc page describing the how's and why's of this approach. Cheers, Devananda On Fri, Feb 27, 2015 at 10:28 PM, Ramakrishnan G <rameshg87.openst...@gmail.com> wrote: > > Hello All, > > This is about adding vendor drivers in Ironic. > > In Kilo, we have many vendor drivers getting added in Ironic which is a very > good thing. But something I noticed is that, most of these reviews have > lots of hardware-specific code in them. This is something most of the other > Ironic folks cannot understand unless they go and read the hardware manuals > of the vendor hardware about what is being done. Otherwise we just need to > blindly mark the file as reviewed. > > Now let me pitch in with our story about this. We added a vendor driver for > HP Proliant hardware (the *ilo drivers in Ironic). Initially we proposed > this same thing in Ironic that we will add all the hardware specific code in > Ironic itself under the directory drivers/modules/ilo. But few of the > Ironic folks didn't agree on this (Devananda especially who is from my > company :)). So we created a new module proliantutils, hosted in our own > github and recently moved it to stackforge. We gave a limited set of APIs > for Ironic to use - like get_host_power_status(), set_host_power(), > get_one_time_boot(), set_one_time_boot(), etc. (Entire list is here > https://github.com/stackforge/proliantutils/blob/master/proliantutils/ilo/operations.py). > > We have only seen benefits in doing it. Let me bring in some examples: > > 1) We tried to add support for some lower version of servers. We could do > this without making any changes in Ironic (Review in proliantutils > https://review.openstack.org/#/c/153945/) > 2) We are adding support for newer models of servers (earlier we use to talk > to servers in protocol called RIBCL, newer servers we will use a protocol > called RIS) - We could do this with just 14 lines of actual code change in > Ironic (this was needed mainly because we didn't think we will have to use a > new protocol itself when we started) - > https://review.openstack.org/#/c/154403/ > > Now talking about the advantages of putting hardware-specific code in > Ironic: > > 1) It's reviewed by Openstack community and tested: > No. I doubt if I throw in 600 lines of new iLO specific code that is here > (https://github.com/stackforge/proliantutils/blob/master/proliantutils/ilo/ris.py) > for Ironic folks, they will hardly take a look at it. And regarding > testing, it's not tested in the gate unless we have a 3rd party CI for it. > [We (iLO drivers) also don't have 3rd party CI right now, but we are working > on it.] > > 2) Everything gets packaged into distributions automatically: > Now the hardware-specific code that we add in Ironic under > drivers/modules/<vendor>/ will get packaged into distributions, but this > code in turn will have dependencies which needs to be installed manually by > the operator (I assume vendor specific dependencies are not considered by > Linux distributions while packaging Openstack Ironic). Anyone installing > Ironic and wanting to manage my company's servers will again need to install > these dependencies manually. Why not install the wrapper if there is one > too. > > I assume we only get these advantages by moving all of hardware-specific > code to a wrapper module in stackforge and just exposing some APIs for > Ironic to use: > * Ironic code would be much cleaner and easier to maintain > * Any changes related to your hardware - support for newer hardware, bug > fixes in particular models of hardware, would be very easy. You don't need > to change Ironic code for that. You could just fix the bug in your module, > release a new version and ask your users to install a newer version of the > module. > * python-fooclient could be used outside Ironic to easily manage foo > servers. > * Openstack CI for free if you are in stackforge - unit tests, flake tests, > doc generation, merge, pypi release everything handled automatically. > > I don't see any disadvantages. > > Now regarding the time taken to do this, if you have all the code ready now > in Ironic (which assume you will already have), perhaps it will take a day > to do this - half a day for putting into a separate module in python/github > and half a day for stackforge. The request to add stackforge should get > approved in the same day (if everything is all-right). > > Let me know all of your thoughts on this. If we agree, I feel we should > have some documentation on it in our Ironic docs directory. > > Thanks for reading :) > > Regards, > Ramesh > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
