So regarding to the 3rd-party CI. There is FuelCI that runs tests for some stackforge projects. Attaching it to Ironic to run required tests that detect problems in FuelAgent driver is not a big deal at all and will be done as soon as it's necessary.
On Tue, Dec 9, 2014 at 3:45 PM, Vladimir Kozhukalov < vkozhuka...@mirantis.com> wrote: > s/though/throw/g > > Vladimir Kozhukalov > > On Tue, Dec 9, 2014 at 5:40 PM, Vladimir Kozhukalov < > vkozhuka...@mirantis.com> wrote: > >> >> >> Vladimir Kozhukalov >> >> On Tue, Dec 9, 2014 at 3:51 PM, Dmitry Tantsur <dtant...@redhat.com> >> wrote: >> >>> Hi folks, >>> >>> Thank you for additional explanation, it does clarify things a bit. I'd >>> like to note, however, that you talk a lot about how _different_ Fuel Agent >>> is from what Ironic does now. I'd like actually to know how well it's going >>> to fit into what Ironic does (in additional to your specific use cases). >>> Hence my comments inline: >> >> >>> >>> On 12/09/2014 01:01 PM, Vladimir Kozhukalov wrote: >>> >>>> Just a short explanation of Fuel use case. >>>> >>>> Fuel use case is not a cloud. Fuel is a deployment tool. We install OS >>>> on bare metal servers and on VMs >>>> and then configure this OS using Puppet. We have been using Cobbler as >>>> our OS provisioning tool since the beginning of Fuel. >>>> However, Cobbler assumes using native OS installers (Anaconda and >>>> Debian-installer). For some reasons we decided to >>>> switch to image based approach for installing OS. >>>> >>>> One of Fuel features is the ability to provide advanced partitioning >>>> schemes (including software RAIDs, LVM). >>>> Native installers are quite difficult to customize in the field of >>>> partitioning >>>> (that was one of the reasons to switch to image based approach). >>>> Moreover, we'd like to implement even more >>>> flexible user experience. We'd like to allow user to choose which hard >>>> drives to use for root FS, for >>>> allocating DB. We'd like user to be able to put root FS over LV or MD >>>> device (including stripe, mirror, multipath). >>>> We'd like user to be able to choose which hard drives are bootable (if >>>> any), which options to use for mounting file systems. >>>> Many many various cases are possible. If you ask why we'd like to >>>> support all those cases, the answer is simple: >>>> because our users want us to support all those cases. >>>> Obviously, many of those cases can not be implemented as image >>>> internals, some cases can not be also implemented on >>>> configuration stage (placing root fs on lvm device). >>>> >>>> As far as those use cases were rejected to be implemented in term of >>>> IPA, we implemented so called Fuel Agent. >>>> Important Fuel Agent features are: >>>> >>>> * It does not have REST API >>>> >>> I would not call it a feature :-P >>> >>> Speaking seriously, if you agent is a long-running thing and it gets >>> it's configuration from e.g. JSON file, how can Ironic notify it of any >>> changes? >>> >>> Fuel Agent is not long-running service. Currently there is no need to >> have REST API. If we deal with kind of keep alive stuff of >> inventory/discovery then we probably add API. Frankly, IPA REST API is not >> REST at all. However that is not a reason to not to call it a feature and >> through it away. It is a reason to work on it and improve. That is how I >> try to look at things (pragmatically). >> >> Fuel Agent has executable entry point[s] like /usr/bin/provision. You can >> run this entry point with options (oslo.config) and point out where to find >> input json data. It is supposed Ironic will use ssh (currently in Fuel we >> use mcollective) connection and run this waiting for exit code. If exit >> code is equal to 0, provisioning is done. Extremely simple. >> >> >>> * it has executable entry point[s] >>>> * It uses local json file as it's input >>>> * It is planned to implement ability to download input data via HTTP >>>> (kind of metadata service) >>>> * It is designed to be agnostic to input data format, not only Fuel >>>> format (data drivers) >>>> * It is designed to be agnostic to image format (tar images, file system >>>> images, disk images, currently fs images) >>>> * It is designed to be agnostic to image compression algorithm >>>> (currently gzip) >>>> * It is designed to be agnostic to image downloading protocol (currently >>>> local file and HTTP link) >>>> >>> Does it support Glance? I understand it's HTTP, but it requires >>> authentication. >>> >>> >>>> So, it is clear that being motivated by Fuel, Fuel Agent is quite >>>> independent and generic. And we are open for >>>> new use cases. >>>> >>> My favorite use case is hardware introspection (aka getting data >>> required for scheduling from a node automatically). Any ideas on this? >>> (It's not a priority for this discussion, just curious). >> >> >> That is exactly what we do in Fuel. Currently we use so called 'Default' >> pxelinux config and all nodes being powered on are supposed to boot with so >> called 'Bootstrap' ramdisk where Ohai based agent (not Fuel Agent) runs >> periodically and sends hardware report to Fuel master node. >> User then is able to look at CPU, hard drive and network info and choose >> which nodes to use for controllers, which for computes, etc. That is what >> nova scheduler is supposed to do (look at hardware info and choose a >> suitable node). >> >> Talking about future, we are planning to re-implement inventory/discovery >> stuff in terms of Fuel Agent (currently, this stuff is implemented as Ohai >> based independent script). Estimation for that is March 2015. >> >>> >>> >>> >>>> According Fuel itself, our nearest plan is to get rid of Cobbler because >>>> in the case of image based approach it is huge overhead. The question is >>>> which tool we can use instead of Cobbler. We need power management, >>>> we need TFTP management, we need DHCP management. That is >>>> exactly what Ironic is able to do. Frankly, we can implement >>>> power/TFTP/DHCP >>>> management tool independently, but as Devananda said, we're all working >>>> on the same problems, >>>> so let's do it together. Power/TFTP/DHCP management is where we are >>>> working on the same problems, >>>> but IPA and Fuel Agent are about different use cases. This case is not >>>> just Fuel, any mature >>>> deployment case require advanced partition/fs management. >>>> >>> Taking into consideration that you're doing a generic OS installation >>> tool... yeah, it starts to make some sense. For cloud advanced partition is >>> definitely a "pet" case. >> >> >> Generic image based OS installation tool. >> >> >>> >>> However, for >>> >>>> me it is OK, if it is easily possible >>>> to use Ironic with external drivers (not merged to Ironic and not tested >>>> on Ironic CI). >>>> >>>> AFAIU, this spec https://review.openstack.org/#/c/138115/ does not >>>> assume changing Ironic API and core. >>>> Jim asked about how Fuel Agent will know about advanced disk >>>> partitioning scheme if API is not >>>> changed. The answer is simple: Ironic is supposed to send a link to >>>> metadata service (http or local file) >>>> where Fuel Agent can download input json data. >>>> >>> That's not about not changing Ironic. Changing Ironic is ok for >>> reasonable use cases - we do a huge change right now to accommodate >>> zapping, hardware introspection and RAID configuration. >>> >>> Minimal changes because we don't want to break anything. It is clear how >> difficult to convince people to do even minimal changes. Again it is just a >> pragmatic approach. We want to do things iteratively so as not to break >> Ironic as well as Fuel. We just can not change all at once. >> >> >>> I actually have problems with this particular statement. It does not >>> sound like Fuel Agent will integrate enough with Ironic. This JSON file: >>> who is going to generate it? In the most popular use case we're driven by >>> Nova. Will Nova generate this file? >>> >>> If the answer is "generate it manually for every node", it's too much a >>> "pet" case for me personally. >>> >>> That is how this provision data look like right now >> https://github.com/stackforge/fuel-web/blob/master/fuel_agent_ci/samples/provision.json >> Do you still think it is written manually? Currently Fuel Agent works as a >> part of Fuel ecosystem. We have a service which serializes provision data >> for us into json. Fuel Agent is agnostic to data format (data drivers). If >> someone wants to use another format, they are welcome to implement a >> driver. >> >> We assume next step will be to put provision data (disk partition scheme, >> maybe other data) into driver_info and make Fuel Agent driver able to >> serialize those data (special format) and implement a corresponding data >> driver in Fuel Agent for this format. Again very simple. Maybe it is time >> to think of having Ironic metadata service (just maybe). >> >> Another point is that currently Fuel stores hardware info in its own >> database but when it is possible to get those data from Ironic (when >> inventory stuff is implemented) we will be glad to use Ironic API for that. >> That is what I mean when I say 'to make Fuel stuff closer to Ironic >> abstractions' >> >> >> >>> >>>> As Roman said, we try to be pragmatic and suggest something which does >>>> not break anything. All changes >>>> are supposed to be encapsulated into a driver. No API and core changes. >>>> We have resources to support, test >>>> and improve this driver. This spec is just a zero step. Further steps >>>> are supposed to improve driver >>>> so as to make it closer to Ironic abstractions. >>>> >>> Honestly I think you should at least write a roadmap for it - see my >>> comments above. >>> >> >> Honestly, I think writing roadmap right now is not very rational as far >> as I am not even sure people are interested in widening Ironic use cases. >> Some of the comments were not even constructive like "I don't understand >> what your use case is, please use IPA". >> >> >>> >>> About testing and support: are you providing a 3rdparty CI for it? It >>> would be a big plus as to me: we already have troubles with drivers broken >>> accidentally. >> >> >> We are flexible here but I'm not ready to answer this question right now. >> We will try to fit Ironic requirements wherever it is possible. >> >> >>> >>> >>>> For Ironic that means widening use cases and user community. But, as I >>>> already said, >>>> we are OK if Ironic does not need this feature. >>>> >>> I don't think we should through away your hardware provision use case, >>> but I personally would like to see how well Fuel Agent is going to play >>> with how Ironic and Nova operate. >>> >> >> Nova is not our case. Fuel is totally about deployment. There is some in >> common >> >> As I already explained, currently we need power/tftp/dhcp management >> Ironic capabilities. Again, it is not a problem to implement this stuff >> independently like it happened with Fuel Agent (because this use case was >> rejected several months ago). Our suggestion is not about "let's compete >> with IPA" it is totally about "let's work on the same problems together". >> >> >> >>> >>>> Vladimir Kozhukalov >>>> >>>> On Tue, Dec 9, 2014 at 1:09 PM, Roman Prykhodchenko >>>> <rprikhodche...@mirantis.com <mailto:rprikhodche...@mirantis.com>> >>>> wrote: >>>> >>>> It is true that IPA and FuelAgent share a lot of functionality in >>>> common. However there is a major difference between them which is >>>> that they are intended to be used to solve a different problem. >>>> >>>> IPA is a solution for provision-use-destroy-use_by_different_user >>>> use-case and is really great for using it for providing BM nodes for >>>> other OS services or in services like Rackspace OnMetal. FuelAgent >>>> itself serves for provision-use-use-…-use use-case like Fuel or >>>> TripleO have. >>>> >>>> Those two use-cases require concentration on different details in >>>> first place. For instance for IPA proper decommissioning is more >>>> important than advanced disk management, but for FuelAgent >>>> priorities are opposite because of obvious reasons. >>>> >>>> Putting all functionality to a single driver and a single agent may >>>> cause conflicts in priorities and make a lot of mess inside both the >>>> driver and the agent. Actually previously changes to IPA were >>>> blocked right because of this conflict of priorities. Therefore >>>> replacing FuelAgent by IPA in where FuelAgent is used currently does >>>> not seem like a good option because come people (and I’m not talking >>>> about Mirantis) might loose required features because of different >>>> priorities. >>>> >>>> Having two separate drivers along with two separate agents for those >>>> different use-cases will allow to have two independent teams that >>>> are concentrated on what’s really important for a specific use-case. >>>> I don’t see any problem in overlapping functionality if it’s used >>>> differently. >>>> >>>> >>>> P. S. >>>> I realise that people may be also confused by the fact that >>>> FuelAgent is actually called like that and is used only in Fuel atm. >>>> Our point is to make it a simple, powerful and what’s more important >>>> a generic tool for provisioning. It is not bound to Fuel or Mirantis >>>> and if it will cause confusion in the future we will even be happy >>>> to give it a different and less confusing name. >>>> >>>> P. P. S. >>>> Some of the points of this integration do not look generic enough or >>>> nice enough. We look pragmatic on the stuff and are trying to >>>> implement what’s possible to implement as the first step. For sure >>>> this is going to have a lot more steps to make it better and more >>>> generic. >>>> >>>> >>>> On 09 Dec 2014, at 01:46, Jim Rollenhagen <j...@jimrollenhagen.com >>>>> <mailto:j...@jimrollenhagen.com>> wrote: >>>>> >>>>> >>>>> >>>>> On December 8, 2014 2:23:58 PM PST, Devananda van der Veen >>>>> <devananda....@gmail.com <mailto:devananda....@gmail.com>> wrote: >>>>> >>>>>> I'd like to raise this topic for a wider discussion outside of the >>>>>> hallway >>>>>> track and code reviews, where it has thus far mostly remained. >>>>>> >>>>>> In previous discussions, my understanding has been that the Fuel >>>>>> team >>>>>> sought to use Ironic to manage "pets" rather than "cattle" - and >>>>>> doing >>>>>> so >>>>>> required extending the API and the project's functionality in >>>>>> ways that >>>>>> no >>>>>> one else on the core team agreed with. Perhaps that understanding >>>>>> was >>>>>> wrong >>>>>> (or perhaps not), but in any case, there is now a proposal to add >>>>>> a >>>>>> FuelAgent driver to Ironic. The proposal claims this would meet >>>>>> that >>>>>> teams' >>>>>> needs without requiring changes to the core of Ironic. >>>>>> >>>>>> https://review.openstack.org/#/c/138115/ >>>>>> >>>>> >>>>> I think it's clear from the review that I share the opinions >>>>> expressed in this email. >>>>> >>>>> That said (and hopefully without derailing the thread too much), >>>>> I'm curious how this driver could do software RAID or LVM without >>>>> modifying Ironic's API or data model. How would the agent know how >>>>> these should be built? How would an operator or user tell Ironic >>>>> what the disk/partition/volume layout would look like? >>>>> >>>>> And before it's said - no, I don't think vendor passthru API calls >>>>> are an appropriate answer here. >>>>> >>>>> // jim >>>>> >>>>> >>>>>> The Problem Description section calls out four things, which have >>>>>> all >>>>>> been >>>>>> discussed previously (some are here [0]). I would like to address >>>>>> each >>>>>> one, >>>>>> invite discussion on whether or not these are, in fact, problems >>>>>> facing >>>>>> Ironic (not whether they are problems for someone, somewhere), >>>>>> and then >>>>>> ask >>>>>> why these necessitate a new driver be added to the project. >>>>>> >>>>>> >>>>>> They are, for reference: >>>>>> >>>>>> 1. limited partition support >>>>>> >>>>>> 2. no software RAID support >>>>>> >>>>>> 3. no LVM support >>>>>> >>>>>> 4. no support for hardware that lacks a BMC >>>>>> >>>>>> #1. >>>>>> >>>>>> When deploying a partition image (eg, QCOW format), Ironic's PXE >>>>>> deploy >>>>>> driver performs only the minimal partitioning necessary to >>>>>> fulfill its >>>>>> mission as an OpenStack service: respect the user's request for >>>>>> root, >>>>>> swap, >>>>>> and ephemeral partition sizes. When deploying a whole-disk image, >>>>>> Ironic >>>>>> does not perform any partitioning -- such is left up to the >>>>>> operator >>>>>> who >>>>>> created the disk image. >>>>>> >>>>>> Support for arbitrarily complex partition layouts is not required >>>>>> by, >>>>>> nor >>>>>> does it facilitate, the goal of provisioning physical servers via >>>>>> a >>>>>> common >>>>>> cloud API. Additionally, as with #3 below, nothing prevents a >>>>>> user from >>>>>> creating more partitions in unallocated disk space once they have >>>>>> access to >>>>>> their instance. Therefor, I don't see how Ironic's minimal >>>>>> support for >>>>>> partitioning is a problem for the project. >>>>>> >>>>>> #2. >>>>>> >>>>>> There is no support for defining a RAID in Ironic today, at all, >>>>>> whether >>>>>> software or hardware. Several proposals were floated last cycle; >>>>>> one is >>>>>> under review right now for DRAC support [1], and there are >>>>>> multiple >>>>>> call >>>>>> outs for RAID building in the state machine mega-spec [2]. Any >>>>>> such >>>>>> support >>>>>> for hardware RAID will necessarily be abstract enough to support >>>>>> multiple >>>>>> hardware vendor's driver implementations and both in-band >>>>>> creation (via >>>>>> IPA) and out-of-band creation (via vendor tools). >>>>>> >>>>>> Given the above, it may become possible to add software RAID >>>>>> support to >>>>>> IPA >>>>>> in the future, under the same abstraction. This would closely tie >>>>>> the >>>>>> deploy agent to the images it deploys (the latter image's kernel >>>>>> would >>>>>> be >>>>>> dependent upon a software RAID built by the former), but this >>>>>> would >>>>>> necessarily be true for the proposed FuelAgent as well. >>>>>> >>>>>> I don't see this as a compelling reason to add a new driver to the >>>>>> project. >>>>>> Instead, we should (plan to) add support for software RAID to the >>>>>> deploy >>>>>> agent which is already part of the project. >>>>>> >>>>>> #3. >>>>>> >>>>>> LVM volumes can easily be added by a user (after provisioning) >>>>>> within >>>>>> unallocated disk space for non-root partitions. I have not yet >>>>>> seen a >>>>>> compelling argument for doing this within the provisioning phase. >>>>>> >>>>>> #4. >>>>>> >>>>>> There are already in-tree drivers [3] [4] [5] which do not >>>>>> require a >>>>>> BMC. >>>>>> One of these uses SSH to connect and run pre-determined commands. >>>>>> Like >>>>>> the >>>>>> spec proposal, which states at line 122, "Control via SSH access >>>>>> feature >>>>>> intended only for experiments in non-production environment," the >>>>>> current >>>>>> SSHPowerDriver is only meant for testing environments. We could >>>>>> probably >>>>>> extend this driver to do what the FuelAgent spec proposes, as far >>>>>> as >>>>>> remote >>>>>> power control for cheap always-on hardware in testing >>>>>> environments with >>>>>> a >>>>>> pre-shared key. >>>>>> >>>>>> (And if anyone wonders about a use case for Ironic without >>>>>> external >>>>>> power >>>>>> control ... I can only think of one situation where I would >>>>>> rationally >>>>>> ever >>>>>> want to have a control-plane agent running inside a >>>>>> user-instance: I am >>>>>> both the operator and the only user of the cloud.) >>>>>> >>>>>> >>>>>> ---------------- >>>>>> >>>>>> In summary, as far as I can tell, all of the problem statements >>>>>> upon >>>>>> which >>>>>> the FuelAgent proposal are based are solvable through incremental >>>>>> changes >>>>>> in existing drivers, or out of scope for the project entirely. As >>>>>> another >>>>>> software-based deploy agent, FuelAgent would duplicate the >>>>>> majority of >>>>>> the >>>>>> functionality which ironic-python-agent has today. >>>>>> >>>>>> Ironic's driver ecosystem benefits from a diversity of >>>>>> hardware-enablement >>>>>> drivers. Today, we have two divergent software deployment drivers >>>>>> which >>>>>> approach image deployment differently: "agent" drivers use a local >>>>>> agent to >>>>>> prepare a system and download the image; "pxe" drivers use a >>>>>> remote >>>>>> agent >>>>>> and copy the image over iSCSI. I don't understand how a second >>>>>> driver >>>>>> which >>>>>> duplicates the functionality we already have, and shares the same >>>>>> goals >>>>>> as >>>>>> the drivers we already have, is beneficial to the project. >>>>>> >>>>>> Doing the same thing twice just increases the burden on the team; >>>>>> we're >>>>>> all >>>>>> working on the same problems, so let's do it together. >>>>>> >>>>>> -Devananda >>>>>> >>>>>> >>>>>> [0] >>>>>> https://blueprints.launchpad.net/ironic/+spec/ironic- >>>>>> python-agent-partition >>>>>> >>>>>> [1] https://review.openstack.org/#/c/107981/ >>>>>> >>>>>> [2] >>>>>> https://review.openstack.org/#/c/133828/11/specs/kilo/new- >>>>>> ironic-state-machine.rst >>>>>> >>>>>> >>>>>> [3] >>>>>> http://git.openstack.org/cgit/openstack/ironic/tree/ironic/ >>>>>> drivers/modules/snmp.py >>>>>> >>>>>> [4] >>>>>> http://git.openstack.org/cgit/openstack/ironic/tree/ironic/ >>>>>> drivers/modules/iboot.py >>>>>> >>>>>> [5] >>>>>> http://git.openstack.org/cgit/openstack/ironic/tree/ironic/ >>>>>> drivers/modules/ssh.py >>>>>> >>>>>> >>>>>> ------------------------------------------------------------ >>>>>> ------------ >>>>>> >>>>>> _______________________________________________ >>>>>> OpenStack-dev mailing list >>>>>> OpenStack-dev@lists.openstack.org >>>>>> <mailto:OpenStack-dev@lists.openstack.org> >>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> OpenStack-dev mailing list >>>>> OpenStack-dev@lists.openstack.org >>>>> <mailto:OpenStack-dev@lists.openstack.org> >>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>> >>>> >>>> >>>> _______________________________________________ >>>> OpenStack-dev mailing list >>>> OpenStack-dev@lists.openstack.org >>>> <mailto:OpenStack-dev@lists.openstack.org> >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> OpenStack-dev mailing list >>>> OpenStack-dev@lists.openstack.org >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >>>> >>> >>> _______________________________________________ >>> OpenStack-dev mailing list >>> OpenStack-dev@lists.openstack.org >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >> >> > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >
_______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
