Re: [openstack-dev] [tripleo][heat][ironic] Heat Ironic resources and ready state orchestration

2014-09-17 Thread Dmitry Tantsur
On Tue, 2014-09-16 at 15:42 -0400, Zane Bitter wrote:
 On 16/09/14 15:24, Devananda van der Veen wrote:
  On Tue, Sep 16, 2014 at 11:44 AM, Zane Bitter zbit...@redhat.com wrote:
  On 16/09/14 13:56, Devananda van der Veen wrote:
 
  On Mon, Sep 15, 2014 at 9:00 AM, Steven Hardy sha...@redhat.com wrote:
 
  For example, today, I've been looking at the steps required for driving
  autodiscovery:
 
  https://etherpad.openstack.org/p/Ironic-PoCDiscovery-Juno
 
  Driving this process looks a lot like application orchestration:
 
  1. Take some input (IPMI credentials and MAC addresses)
  2. Maybe build an image and ramdisk(could drop credentials in)
  3. Interact with the Ironic API to register nodes in maintenance mode
  4. Boot the nodes, monitor state, wait for a signal back containing some
   data obtained during discovery (same as WaitConditions or
   SoftwareDeployment resources in Heat..)
  5. Shutdown the nodes and mark them ready for use by nova
 
 
  My apologies if the following sounds snarky -- but I think there are a
  few misconceptions that need to be cleared up about how and when one
  might use Ironic. I also disagree that 1..5 looks like application
  orchestration. Step 4 is a workflow, which I'll go into in a bit, but
  this doesn't look at all like describing or launching an application
  to me.
 
 
  +1 (Although step 3 does sound to me like something that matches Heat's
  scope.)
 
  I think it's a simplistic use case, and Heat supports a lot more
  complexity than is necessary to enroll nodes with Ironic.
 
 
  Step 1 is just parse a text file.
 
  Step 2 should be a prerequisite to doing -anything- with Ironic. Those
  images need to be built and loaded in Glance, and the image UUID(s)
  need to be set on each Node in Ironic (or on the Nova flavor, if going
  that route) after enrollment. Sure, Heat can express this
  declaratively (ironic.node.driver_info must contain key:deploy_kernel
  with value:), but are you suggesting that Heat build the images,
  or just take the UUIDs as input?
 
  Step 3 is, again, just parse a text file
 
  I'm going to make an assumption here [*], because I think step 4 is
  misleading. You shouldn't boot a node using Ironic -- you do that
  through Nova. And you _dont_ get to specify which node you're booting.
  You ask Nova to provision an _instance_ on a _flavor_ and it picks an
  available node from the pool of nodes that match the request.
 
 
  I think your assumption is incorrect. Steve is well aware that provisioning
  a bare-metal Ironic server is done through the Nova API. What he's
  suggesting here is that the nodes would be booted - not Nova-booted, but
  booted in the sense of having power physically applied - while in
  maintenance mode in order to do autodiscovery of their capabilities,
 
  Except simply applying power doesn't, in itself, accomplish anything
  besides causing the machine to power on. Ironic will only prepare the
  PXE boot environment when initiating a _deploy_.
 
  From what I gather elsewhere in this thread, the autodiscovery stuff is 
 a proposal for the future, not something that exists in Ironic now, and 
 that may be the source of the confusion.
 
 In any case, the etherpad linked at the top of this email was written by 
 someone in the Ironic team and _clearly_ describes PXE booting a 
 discovery image in maintenance mode in order to obtain hardware 
 information about the box.
If was written by me and it seems to be my fault that I didn't state
there more clear that this work is not and probably will not be merged
into Ironic upstream. Sorry for the confusion.

That said, my experiments proved quite possible (though not without some
network-related hacks as of now) to follow these steps to collect (aka
discover) hardware information required for scheduling from a node,
knowing only it's IPMI credentials.

 
 cheers,
 Zane.
 
  which
  is presumably hard to do automatically when they're turned off.
 
  Vendors often have ways to do this while the power is turned off, eg.
  via the OOB management interface.
 
  He's also
  suggesting that Heat could drive this process, which I happen to disagree
  with because it is a workflow not an end state.
 
  +1
 
  However the main takeaway
  here is that you guys are talking completely past one another, and have 
  been
  for some time.
 
 
  Perhaps more detail in the expected interactions with Ironic would be
  helpful and avoid me making (perhaps incorrect) assumptions.
 
  -D
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org

Re: [openstack-dev] [tripleo][heat][ironic] Heat Ironic resources and ready state orchestration

2014-09-17 Thread Steven Hardy
On Tue, Sep 16, 2014 at 02:06:59PM -0700, Devananda van der Veen wrote:
 On Tue, Sep 16, 2014 at 12:42 PM, Zane Bitter zbit...@redhat.com wrote:
  On 16/09/14 15:24, Devananda van der Veen wrote:
 
  On Tue, Sep 16, 2014 at 11:44 AM, Zane Bitter zbit...@redhat.com wrote:
 
  On 16/09/14 13:56, Devananda van der Veen wrote:
 
 
  On Mon, Sep 15, 2014 at 9:00 AM, Steven Hardy sha...@redhat.com wrote:
 
 
  For example, today, I've been looking at the steps required for driving
  autodiscovery:
 
  https://etherpad.openstack.org/p/Ironic-PoCDiscovery-Juno
 
  Driving this process looks a lot like application orchestration:
 
  1. Take some input (IPMI credentials and MAC addresses)
  2. Maybe build an image and ramdisk(could drop credentials in)
  3. Interact with the Ironic API to register nodes in maintenance mode
  4. Boot the nodes, monitor state, wait for a signal back containing
  some
   data obtained during discovery (same as WaitConditions or
   SoftwareDeployment resources in Heat..)
  5. Shutdown the nodes and mark them ready for use by nova
 
 
  My apologies if the following sounds snarky -- but I think there are a
  few misconceptions that need to be cleared up about how and when one
  might use Ironic. I also disagree that 1..5 looks like application
  orchestration. Step 4 is a workflow, which I'll go into in a bit, but
  this doesn't look at all like describing or launching an application
  to me.
 
 
 
  +1 (Although step 3 does sound to me like something that matches Heat's
  scope.)
 
 
  I think it's a simplistic use case, and Heat supports a lot more
  complexity than is necessary to enroll nodes with Ironic.
 
 
  Step 1 is just parse a text file.
 
  Step 2 should be a prerequisite to doing -anything- with Ironic. Those
  images need to be built and loaded in Glance, and the image UUID(s)
  need to be set on each Node in Ironic (or on the Nova flavor, if going
  that route) after enrollment. Sure, Heat can express this
  declaratively (ironic.node.driver_info must contain key:deploy_kernel
  with value:), but are you suggesting that Heat build the images,
  or just take the UUIDs as input?
 
  Step 3 is, again, just parse a text file
 
  I'm going to make an assumption here [*], because I think step 4 is
  misleading. You shouldn't boot a node using Ironic -- you do that
  through Nova. And you _dont_ get to specify which node you're booting.
  You ask Nova to provision an _instance_ on a _flavor_ and it picks an
  available node from the pool of nodes that match the request.
 
 
 
  I think your assumption is incorrect. Steve is well aware that
  provisioning
  a bare-metal Ironic server is done through the Nova API. What he's
  suggesting here is that the nodes would be booted - not Nova-booted, but
  booted in the sense of having power physically applied - while in
  maintenance mode in order to do autodiscovery of their capabilities,
 
 
  Except simply applying power doesn't, in itself, accomplish anything
  besides causing the machine to power on. Ironic will only prepare the
  PXE boot environment when initiating a _deploy_.
 
 
  From what I gather elsewhere in this thread, the autodiscovery stuff is a
  proposal for the future, not something that exists in Ironic now, and that
  may be the source of the confusion.
 
  In any case, the etherpad linked at the top of this email was written by
  someone in the Ironic team and _clearly_ describes PXE booting a discovery
  image in maintenance mode in order to obtain hardware information about the
  box.
 
 
 Huh. I should have looked at that earlier in the discussion. It is
 referring to out-of-tree code whose spec was not approved during Juno.
 
 Apparently, and unfortunately, throughout much of this discussion,
 folks have been referring to potential features Ironic might someday
 have, whereas I have been focused on the features we actually support
 today. That is probably why it seems we are talking past each other.

FWIW I think a big part of the problem has been that you've been focussing
on the fact that my solution doesn't match your preconceived ideas of how
Ironic should interface with the world, while completely ignoring the
use-case, e.g the actual problem I'm trying to solve.

That is why I'm referring to features Ironic might someday have - because
Ironic currently does not solve my problem, so I'm looking for a workable
way to change that.

When I posted the draft Ironic resources, I did fail to provide detailed
use-case info, so my bad there, but since I've posted the spec I don't
really feel like the discussion has been much more productive - I've tried,
repeatedly, to get you to understand my use-case, and you've tried,
repeatedly, to tell me my implementation is wrong (without providing any
fully-formed alternative, I call this unqualified your-idea-sucks, a
common and destructive review anti-pattern IMO)

It wasn't until Jay Faulkner's message earlier in this thread that someone
actually proposed a 

Re: [openstack-dev] [tripleo][heat][ironic] Heat Ironic resources and ready state orchestration

2014-09-17 Thread Lucas Alvares Gomes
 1. Not everyone will have an enterprise CMDB, so there should be some way
 to input inventory without one (even if it is a text file fed into
 ironicclient). The bulk-loading format to do this is TBD.

 2. A way to generate that inventory in an automated way is desirable for
 some folks, but looks likely to be out-of-scope for Ironic.  Folks are -1
 on using heat to drive this process, so we'll probably end up with some
 scary shell scripts instead, or maybe a mistral workflow in future.

FWIW, I'm not against having an automatic way to discovery the
hardware properties in Ironic.

Lemme try to be clear, if Ironic needs to know the size of disk,
amount of memory, no of CPUs and CPU arch to be able to deploy a
machine, and BMCs like Drac, iLO, MM, etc... provides an OOB endpoint
which Ironic can use to get such informations, so I don't see why we
should not consume that from in Ironic. Note that I don't agree that
Ironic should store *all* informations about the hardware, but the
informations it's going to use to be able to *deploy* that machine.

So the reasons I think we should do that is:

1) First because Ironic is an abstraction layer for the hardware, so
if it's a feature that the hardware provides I don't see why not
abstracting and creating an API for it. We already have the
vendor_passthru endpoint where things like that can be used by the
vendor to expose their new shiny hardware capabiltities, and if other
drivers start implementing the same thing in their vendor_passthru we
can go then and promote that feature to a common API. So we already
even have a strategy for that.


2) To be less error prone, this is about quality. Why would I rely on
a human to input all this information to Ironic if I can go there and
interrogate the hardware directly and be sure that this is the real
amount of resources I have there. @Jim even said in this thread that
dealing with the incorrect data from vendor, DC team, etc is the hard
part of registering this things. So I don't wanna rely on them, if I
have a mean to talk to the hardware directly and get this informations
why not do it?


 3. Vendor-specific optimization of nodes for particular roles will be
 handled via Ironic drivers, which expose capabilities which can be selected
 via nova flavors. (is there a BP for this?)

 4. Stuff like RAID configuration will be handled via in-band config
 management tools, nobody has offered any solution for using management
 interfaces to do this, and drac-raid-mgmt is unlikely to land in Ironic
 (where would such an interface be appropriate then?)

IMO, this falls in the same argument I have about Ironic is an
abstraction layer for hardware, if the BMC supports it and configuring
RAID is something that makes sense to do prior to deploying a node, I
don't see why we should not abstract it. Again, we have a strategy for
that, let vendors expose it first in their vendor_passthru interface,
and if more see it as a nice feature to have and implement in their
vendor_passthru as well we can go and promote it to a common API
interface


 5. Nobody has offered any solution for management and convergence of BIOS
 and firmware levels (would this be part of the Ironic driver mentioned in
 (3), or are we punting the entire problem to in-band provision-time tooling?)

 If anyone can help by providing existing BP's related to the above (which I
 can follow and/or contribute to) that would be great - I'm happy to drop
 the whole Heat resource thing, but only if there's a clear path to solving
 the problems in some other/better way.

 Thanks,

 Steve

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo][heat][ironic] Heat Ironic resources and ready state orchestration

2014-09-17 Thread Dmitry Tantsur
On Wed, 2014-09-17 at 10:36 +0100, Steven Hardy wrote:
 On Tue, Sep 16, 2014 at 02:06:59PM -0700, Devananda van der Veen wrote:
  On Tue, Sep 16, 2014 at 12:42 PM, Zane Bitter zbit...@redhat.com wrote:
   On 16/09/14 15:24, Devananda van der Veen wrote:
  
   On Tue, Sep 16, 2014 at 11:44 AM, Zane Bitter zbit...@redhat.com wrote:
  
   On 16/09/14 13:56, Devananda van der Veen wrote:
  
  
   On Mon, Sep 15, 2014 at 9:00 AM, Steven Hardy sha...@redhat.com 
   wrote:
  
  
   For example, today, I've been looking at the steps required for 
   driving
   autodiscovery:
  
   https://etherpad.openstack.org/p/Ironic-PoCDiscovery-Juno
  
   Driving this process looks a lot like application orchestration:
  
   1. Take some input (IPMI credentials and MAC addresses)
   2. Maybe build an image and ramdisk(could drop credentials in)
   3. Interact with the Ironic API to register nodes in maintenance mode
   4. Boot the nodes, monitor state, wait for a signal back containing
   some
data obtained during discovery (same as WaitConditions or
SoftwareDeployment resources in Heat..)
   5. Shutdown the nodes and mark them ready for use by nova
  
  
   My apologies if the following sounds snarky -- but I think there are a
   few misconceptions that need to be cleared up about how and when one
   might use Ironic. I also disagree that 1..5 looks like application
   orchestration. Step 4 is a workflow, which I'll go into in a bit, but
   this doesn't look at all like describing or launching an application
   to me.
  
  
  
   +1 (Although step 3 does sound to me like something that matches Heat's
   scope.)
  
  
   I think it's a simplistic use case, and Heat supports a lot more
   complexity than is necessary to enroll nodes with Ironic.
  
  
   Step 1 is just parse a text file.
  
   Step 2 should be a prerequisite to doing -anything- with Ironic. Those
   images need to be built and loaded in Glance, and the image UUID(s)
   need to be set on each Node in Ironic (or on the Nova flavor, if going
   that route) after enrollment. Sure, Heat can express this
   declaratively (ironic.node.driver_info must contain key:deploy_kernel
   with value:), but are you suggesting that Heat build the images,
   or just take the UUIDs as input?
  
   Step 3 is, again, just parse a text file
  
   I'm going to make an assumption here [*], because I think step 4 is
   misleading. You shouldn't boot a node using Ironic -- you do that
   through Nova. And you _dont_ get to specify which node you're booting.
   You ask Nova to provision an _instance_ on a _flavor_ and it picks an
   available node from the pool of nodes that match the request.
  
  
  
   I think your assumption is incorrect. Steve is well aware that
   provisioning
   a bare-metal Ironic server is done through the Nova API. What he's
   suggesting here is that the nodes would be booted - not Nova-booted, but
   booted in the sense of having power physically applied - while in
   maintenance mode in order to do autodiscovery of their capabilities,
  
  
   Except simply applying power doesn't, in itself, accomplish anything
   besides causing the machine to power on. Ironic will only prepare the
   PXE boot environment when initiating a _deploy_.
  
  
   From what I gather elsewhere in this thread, the autodiscovery stuff is a
   proposal for the future, not something that exists in Ironic now, and that
   may be the source of the confusion.
  
   In any case, the etherpad linked at the top of this email was written by
   someone in the Ironic team and _clearly_ describes PXE booting a 
   discovery
   image in maintenance mode in order to obtain hardware information about 
   the
   box.
  
  
  Huh. I should have looked at that earlier in the discussion. It is
  referring to out-of-tree code whose spec was not approved during Juno.
  
  Apparently, and unfortunately, throughout much of this discussion,
  folks have been referring to potential features Ironic might someday
  have, whereas I have been focused on the features we actually support
  today. That is probably why it seems we are talking past each other.
 
 FWIW I think a big part of the problem has been that you've been focussing
 on the fact that my solution doesn't match your preconceived ideas of how
 Ironic should interface with the world, while completely ignoring the
 use-case, e.g the actual problem I'm trying to solve.
 
 That is why I'm referring to features Ironic might someday have - because
 Ironic currently does not solve my problem, so I'm looking for a workable
 way to change that.
 
 When I posted the draft Ironic resources, I did fail to provide detailed
 use-case info, so my bad there, but since I've posted the spec I don't
 really feel like the discussion has been much more productive - I've tried,
 repeatedly, to get you to understand my use-case, and you've tried,
 repeatedly, to tell me my implementation is wrong (without providing any
 fully-formed alternative, I 

Re: [openstack-dev] [tripleo][heat][ironic] Heat Ironic resources and ready state orchestration

2014-09-16 Thread Devananda van der Veen
On Sep 15, 2014 8:20 AM, James Slagle james.sla...@gmail.com wrote:

 On Mon, Sep 15, 2014 at 7:44 AM, Steven Hardy sha...@redhat.com wrote:
  All,
 
  Starting this thread as a follow-up to a strongly negative reaction by the
  Ironic PTL to my patches[1] adding initial Heat-Ironic integration, and
  subsequent very detailed justification and discussion of why they may be
  useful in this spec[2].
 
  Back in Atlanta, I had some discussions with folks interesting in making
  ready state[3] preparation of bare-metal resources possible when
  deploying bare-metal nodes via TripleO/Heat/Ironic.

 After a cursory reading of the references, it seems there's a couple of 
 issues:
 - are the features to move hardware to a ready-state even going to
 be in Ironic proper, whether that means in ironic at all or just in
 contrib.
 - assuming some of the features are there, should Heat have any Ironic
 resources given that Ironic's API is admin-only.

 
  The initial assumption is that there is some discovery step (either
  automatic or static generation of a manifest of nodes), that can be input
  to either Ironic or Heat.

 I think it makes a lot of sense to use Heat to do the bulk
 registration of nodes via Ironic. I understand the argument that the
 Ironic API should be admin-only a little bit for the non-TripleO
 case, but for TripleO, we only have admins interfacing with the
 Undercloud. The user of a TripleO undercloud is the deployer/operator
 and in some scenarios this may not be the undercloud admin. So,
 talking about TripleO, I don't really buy that the Ironic API is
 admin-only.


When I say the ironic API is admin only, I'm speaking to the required
permissions for accessing it. One must be authenticated with keystone
with the admin privilege. Borrowing from the ops guide:

 An administrative super user, which has full permissions across all
projects and should be used with great care.

I'm not sure how TripleO is dividing operator and admin in the
undercloud - so I just want to be clear on what you mean when you say
may not be the undercloud admin. Simply put, to use Ironic in the
undercloud, you must have admin privileges in the undercloud -- or
you need to disable Ironic's auth entirely.

 Therefore, why not have some declarative Heat resources for things
 like Ironic nodes, that the deployer can make use of in a Heat
 template to do bulk node registration?

 The alternative listed in the spec:

 Don’t implement the resources and rely on scripts which directly
 interact with the Ironic API, prior to any orchestration via Heat.

 would just be a bit silly IMO. That goes against one of the main
 drivers of TripleO, which is to use OpenStack wherever possible. Why
 go off and write some other thing that is going to parse a
 json/yaml/csv of nodes and orchestrate a bunch of Ironic api calls?
 Why would it be ok for that other thing to use Ironic's admin-only
 API yet claim it's not ok for Heat on the undercloud to do so?


Heat has a mission. It's not just a hammer with which to parse
json/yaml/etc into a for loop and throw text at an API. From the wiki:

... to create a human- and machine-accessible service for managing
the entire lifecycle of infrastructure and applications within
OpenStack clouds.

The resources ironic exposes are not within OpenStack clouds. They
are _underneath_ the cloud. Actually, they're underneath the
undercloud.

Configuring a node in Ironic is akin to configuring the SAN from which
Cinder provisions volumes. That is clearly a thing which an operator
needs to do -- but are you suggesting that, if my SAN has a REST API,
I should use Heat to configure it?

This is the crux of my objection. I would be surprised if your answer
is yes, heat should be used to configure my SAN. If you're wondering
why I used that example, it's because folks already asked me if they
can use Ironic to deploy their SAN's management software, perform
firmware upgrades on it, and so on. (I said no, not today but it's
an interesting scope discussion for Ironic).

  Following discovery, but before an undercloud deploying OpenStack onto the
  nodes, there are a few steps which may be desired, to get the hardware into
  a state where it's ready and fully optimized for the subsequent deployment:
 

As others have said already, based on discussions during the Juno
cycle, discovery has not landed, and most of us agree that it is out
of scope.

  - Updating and aligning firmware to meet requirements of qualification or
site policy
  - Optimization of BIOS configuration to match workloads the node is
expected to run
  - Management of machine-local storage, e.g configuring local RAID for
optimal resilience or performance.
 

These steps are desirable not just the first time a node is added to
ironic, but often subsequently, either between every deployment, or
when the operator changes the role/function that hardware fulfills, or
if the hardware components are replaced.

I suspect the workflow to apply the 

Re: [openstack-dev] [tripleo][heat][ironic] Heat Ironic resources and ready state orchestration

2014-09-16 Thread Devananda van der Veen
On Mon, Sep 15, 2014 at 10:51 AM, Jay Faulkner j...@jvf.cc wrote:
 Steven,

 It's important to note that two of the blueprints you reference:

 https://blueprints.launchpad.net/ironic/+spec/drac-raid-mgmt
 https://blueprints.launchpad.net/ironic/+spec/drac-hw-discovery

 are both very unlikely to land in Ironic -- these are configuration and 
 discovery pieces that best fit inside a operator-deployed CMDB, rather than 
 Ironic trying to extend its scope significantly to include these type of 
 functions. I expect the scoping or Ironic with regards to hardware 
 discovery/interrogation as well as configuration of hardware (like I will 
 outline below) to be hot topics in Ironic design summit sessions at Paris.

 A good way of looking at it is that Ironic is responsible for hardware *at 
 provision time*. Registering the nodes in Ironic, as well as hardware 
 settings/maintenance/etc while a workload is provisioned is left to the 
 operators' CMDB.

 This means what Ironic *can* do is modify the configuration of a node at 
 provision time based on information passed down the provisioning pipeline. 
 For instance, if you wanted to configure certain firmware pieces at provision 
 time, you could do something like this:

 Nova flavor sets capability:vm_hypervisor in the flavor that maps to the 
 Ironic node. This would map to an Ironic driver that exposes vm_hypervisor as 
 a capability, and upon seeing capability:vm_hypervisor has been requested, 
 could then configure the firmware/BIOS of the machine to 'hypervisor 
 friendly' settings, such as VT bit on and Turbo mode off. You could map 
 multiple different combinations of capabilities as different Ironic flavors, 
 and have them all represent different configurations of the same pool of 
 nodes. So, you end up with two categories of abilities: inherent abilities of 
 the node (such as amount of RAM or CPU installed), and configurable abilities 
 (i.e. things than can be turned on/off at provision time on demand) -- or 
 perhaps, in the future, even things like RAM and CPU will be dynamically 
 provisioned into nodes at provision time.



Thanks for the explanation, Jay.

Steven, in response to your question [what would] just do that
optimization before the deployment? -- see Jay's example above. This
path has grown out of several discussions we've had over the last two
years, and is closer aligned to what I *thought* TripleO wanted back
when I was more involved in that project.

To paraphrase: Ironic exposes capabilities to Nova, and the Nova
scheduler can pick a node based on which capability is requested in
the flavor definition. We don't yet, but are planning to, support
on-demand customization of nodes based on the requested capabilities.
Toggling the VT bit is a canonical example of this -- we should be
able to dynamically update a node's firmware configuration to satisfy
both virtualization and non-virtualization workloads. That's going to
be expressed via Nova flavors and communicated at provision time by
Nova to Ironic. Eventually, I'd like to see everything in that space
(except perhaps RAID topology, since that actually takes a lot of time
to change).

-Devananad

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo][heat][ironic] Heat Ironic resources and ready state orchestration

2014-09-16 Thread Devananda van der Veen
On Mon, Sep 15, 2014 at 9:00 AM, Steven Hardy sha...@redhat.com wrote:
 For example, today, I've been looking at the steps required for driving
 autodiscovery:

 https://etherpad.openstack.org/p/Ironic-PoCDiscovery-Juno

 Driving this process looks a lot like application orchestration:

 1. Take some input (IPMI credentials and MAC addresses)
 2. Maybe build an image and ramdisk(could drop credentials in)
 3. Interact with the Ironic API to register nodes in maintenance mode
 4. Boot the nodes, monitor state, wait for a signal back containing some
data obtained during discovery (same as WaitConditions or
SoftwareDeployment resources in Heat..)
 5. Shutdown the nodes and mark them ready for use by nova


My apologies if the following sounds snarky -- but I think there are a
few misconceptions that need to be cleared up about how and when one
might use Ironic. I also disagree that 1..5 looks like application
orchestration. Step 4 is a workflow, which I'll go into in a bit, but
this doesn't look at all like describing or launching an application
to me.


Step 1 is just parse a text file.

Step 2 should be a prerequisite to doing -anything- with Ironic. Those
images need to be built and loaded in Glance, and the image UUID(s)
need to be set on each Node in Ironic (or on the Nova flavor, if going
that route) after enrollment. Sure, Heat can express this
declaratively (ironic.node.driver_info must contain key:deploy_kernel
with value:), but are you suggesting that Heat build the images,
or just take the UUIDs as input?

Step 3 is, again, just parse a text file

I'm going to make an assumption here [*], because I think step 4 is
misleading. You shouldn't boot a node using Ironic -- you do that
through Nova. And you _dont_ get to specify which node you're booting.
You ask Nova to provision an _instance_ on a _flavor_ and it picks an
available node from the pool of nodes that match the request.

Step 5 is a prerequisite for step 4 -- you can't boot a node that is
in maintenance mode, and if the node is not in maintenance mode, Nova
exposes it to clients. That is in fact how you'd boot it in step 4.

[*] I'm assuming you are not planning to re-implement the Nova
ironic driver in Heat. Booting a node with Ironic is not just a
matter of making one or two API calls. It's a declarative
transformation involving multiple changes in Ironic's API, and
presumably also some calls to Neutron if you want network access
to/from your node, and polling the resource to see when its state
converges on the requested state. Actually that sounds like exactly
the sort of thing that Heat could drive. But all this is already
implemented in Nova.


-Devananda

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo][heat][ironic] Heat Ironic resources and ready state orchestration

2014-09-16 Thread Devananda van der Veen
On Mon, Sep 15, 2014 at 1:08 PM, Steven Hardy sha...@redhat.com wrote:
 On Mon, Sep 15, 2014 at 05:51:43PM +, Jay Faulkner wrote:
 Steven,

 It's important to note that two of the blueprints you reference:

 https://blueprints.launchpad.net/ironic/+spec/drac-raid-mgmt
 https://blueprints.launchpad.net/ironic/+spec/drac-hw-discovery

 are both very unlikely to land in Ironic -- these are configuration and 
 discovery pieces that best fit inside a operator-deployed CMDB, rather than 
 Ironic trying to extend its scope significantly to include these type of 
 functions. I expect the scoping or Ironic with regards to hardware 
 discovery/interrogation as well as configuration of hardware (like I will 
 outline below) to be hot topics in Ironic design summit sessions at Paris.

 Hmm, okay - not sure I really get how a CMDB is going to help you configure
 your RAID arrays in an automated way?

 Or are you subscribing to the legacy datacentre model where a sysadmin
 configures a bunch of boxes via whatever method, puts their details into
 the CMDB, then feeds those details into Ironic?

 A good way of looking at it is that Ironic is responsible for hardware *at 
 provision time*. Registering the nodes in Ironic, as well as hardware 
 settings/maintenance/etc while a workload is provisioned is left to the 
 operators' CMDB.

 This means what Ironic *can* do is modify the configuration of a node at 
 provision time based on information passed down the provisioning pipeline. 
 For instance, if you wanted to configure certain firmware pieces at 
 provision time, you could do something like this:

 Nova flavor sets capability:vm_hypervisor in the flavor that maps to the 
 Ironic node. This would map to an Ironic driver that exposes vm_hypervisor 
 as a capability, and upon seeing capability:vm_hypervisor has been 
 requested, could then configure the firmware/BIOS of the machine to 
 'hypervisor friendly' settings, such as VT bit on and Turbo mode off. You 
 could map multiple different combinations of capabilities as different 
 Ironic flavors, and have them all represent different configurations of the 
 same pool of nodes. So, you end up with two categories of abilities: 
 inherent abilities of the node (such as amount of RAM or CPU installed), and 
 configurable abilities (i.e. things than can be turned on/off at provision 
 time on demand) -- or perhaps, in the future, even things like RAM and CPU 
 will be dynamically provisioned into nodes at provision time.

 So you advocate pushing all the vendor-specific stuff down into various
 Ironic drivers,

... and providing a common abstraction / representation for it. Yes.
That is, after all, what OpenStack has done for compute, storage, and
networking, and what Ironic has set out to do for hardware
provisioning from the beginning.

  is any of what you describe above possible today?

No. We had other priorities in Juno. It's probably one of the things
we'll prioritize in Kilo.

If you can't wait for Ironic to implement a common abstraction layer
for such functionality, then by all means, implement vendor-native
templates in Heat, but keep in mind that our goal is to move any
functionality which multiple vendors provide into the common API over
time. Vendor passthru is there as an early proving ground for vendors
to add their unique capabilities while we work towards cross-vendor
standards.

-D

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo][heat][ironic] Heat Ironic resources and ready state orchestration

2014-09-16 Thread Zane Bitter

On 16/09/14 13:56, Devananda van der Veen wrote:

On Mon, Sep 15, 2014 at 9:00 AM, Steven Hardy sha...@redhat.com wrote:

For example, today, I've been looking at the steps required for driving
autodiscovery:

https://etherpad.openstack.org/p/Ironic-PoCDiscovery-Juno

Driving this process looks a lot like application orchestration:

1. Take some input (IPMI credentials and MAC addresses)
2. Maybe build an image and ramdisk(could drop credentials in)
3. Interact with the Ironic API to register nodes in maintenance mode
4. Boot the nodes, monitor state, wait for a signal back containing some
data obtained during discovery (same as WaitConditions or
SoftwareDeployment resources in Heat..)
5. Shutdown the nodes and mark them ready for use by nova



My apologies if the following sounds snarky -- but I think there are a
few misconceptions that need to be cleared up about how and when one
might use Ironic. I also disagree that 1..5 looks like application
orchestration. Step 4 is a workflow, which I'll go into in a bit, but
this doesn't look at all like describing or launching an application
to me.


+1 (Although step 3 does sound to me like something that matches Heat's 
scope.)



Step 1 is just parse a text file.

Step 2 should be a prerequisite to doing -anything- with Ironic. Those
images need to be built and loaded in Glance, and the image UUID(s)
need to be set on each Node in Ironic (or on the Nova flavor, if going
that route) after enrollment. Sure, Heat can express this
declaratively (ironic.node.driver_info must contain key:deploy_kernel
with value:), but are you suggesting that Heat build the images,
or just take the UUIDs as input?

Step 3 is, again, just parse a text file

I'm going to make an assumption here [*], because I think step 4 is
misleading. You shouldn't boot a node using Ironic -- you do that
through Nova. And you _dont_ get to specify which node you're booting.
You ask Nova to provision an _instance_ on a _flavor_ and it picks an
available node from the pool of nodes that match the request.


I think your assumption is incorrect. Steve is well aware that 
provisioning a bare-metal Ironic server is done through the Nova API. 
What he's suggesting here is that the nodes would be booted - not 
Nova-booted, but booted in the sense of having power physically applied 
- while in maintenance mode in order to do autodiscovery of their 
capabilities, which is presumably hard to do automatically when they're 
turned off. He's also suggesting that Heat could drive this process, 
which I happen to disagree with because it is a workflow not an end 
state. However the main takeaway here is that you guys are talking 
completely past one another, and have been for some time.


cheers,
Zane.


Step 5 is a prerequisite for step 4 -- you can't boot a node that is
in maintenance mode, and if the node is not in maintenance mode, Nova
exposes it to clients. That is in fact how you'd boot it in step 4.

[*] I'm assuming you are not planning to re-implement the Nova
ironic driver in Heat. Booting a node with Ironic is not just a
matter of making one or two API calls. It's a declarative
transformation involving multiple changes in Ironic's API, and
presumably also some calls to Neutron if you want network access
to/from your node, and polling the resource to see when its state
converges on the requested state. Actually that sounds like exactly
the sort of thing that Heat could drive. But all this is already
implemented in Nova.


-Devananda

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo][heat][ironic] Heat Ironic resources and ready state orchestration

2014-09-16 Thread Zane Bitter

On 16/09/14 13:54, Devananda van der Veen wrote:

On Sep 15, 2014 8:20 AM, James Slaglejames.sla...@gmail.com  wrote:


On Mon, Sep 15, 2014 at 7:44 AM, Steven Hardysha...@redhat.com  wrote:

 
 The initial assumption is that there is some discovery step (either
 automatic or static generation of a manifest of nodes), that can be input
 to either Ironic or Heat.


I think it makes a lot of sense to use Heat to do the bulk
registration of nodes via Ironic. I understand the argument that the
Ironic API should be admin-only a little bit for the non-TripleO
case, but for TripleO, we only have admins interfacing with the
Undercloud. The user of a TripleO undercloud is the deployer/operator
and in some scenarios this may not be the undercloud admin. So,
talking about TripleO, I don't really buy that the Ironic API is
admin-only.


When I say the ironic API is admin only, I'm speaking to the required
permissions for accessing it. One must be authenticated with keystone
with the admin privilege. Borrowing from the ops guide:


In most contexts admin only means that the default policy.json file 
requires the admin role in the current project in order to access a 
particular API endpoint. If you are relying on the is_admin flag in the 
user context and not policy.json then it's likely you are Doing Keystone 
Wrong(TM).



 An administrative super user, which has full permissions across all
projects and should be used with great care.

I'm not sure how TripleO is dividing operator and admin in the
undercloud - so I just want to be clear on what you mean when you say
may not be the undercloud admin. Simply put, to use Ironic in the
undercloud, you must have admin privileges in the undercloud -- or
you need to disable Ironic's auth entirely.


TripleO can presumably deploy any policy.json file they like in the 
undercloud. It's not entirely surprising that some operations that might 
are admin-only in an overcloud might need to be available in the 
undercloud to ordinary users - who, after all, have permissions to 
create entire overclouds - despite them not being admins of the 
undercloud itself.


cheers,
Zane.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo][heat][ironic] Heat Ironic resources and ready state orchestration

2014-09-16 Thread Devananda van der Veen
On Tue, Sep 16, 2014 at 11:57 AM, Zane Bitter zbit...@redhat.com wrote:
 On 16/09/14 13:54, Devananda van der Veen wrote:

 On Sep 15, 2014 8:20 AM, James Slaglejames.sla...@gmail.com  wrote:

 
 On Mon, Sep 15, 2014 at 7:44 AM, Steven Hardysha...@redhat.com  wrote:

  
  The initial assumption is that there is some discovery step (either
  automatic or static generation of a manifest of nodes), that can be
   input
  to either Ironic or Heat.

 
 I think it makes a lot of sense to use Heat to do the bulk
 registration of nodes via Ironic. I understand the argument that the
 Ironic API should be admin-only a little bit for the non-TripleO
 case, but for TripleO, we only have admins interfacing with the
 Undercloud. The user of a TripleO undercloud is the deployer/operator
 and in some scenarios this may not be the undercloud admin. So,
 talking about TripleO, I don't really buy that the Ironic API is
 admin-only.
 

 When I say the ironic API is admin only, I'm speaking to the required
 permissions for accessing it. One must be authenticated with keystone
 with the admin privilege. Borrowing from the ops guide:


 In most contexts admin only means that the default policy.json file
 requires the admin role in the current project in order to access a
 particular API endpoint. If you are relying on the is_admin flag in the user
 context and not policy.json then it's likely you are Doing Keystone
 Wrong(TM).

http://git.openstack.org/cgit/openstack/ironic/tree/etc/ironic/policy.json

There are no per-endpoint policies implemented in Ironic. This was
intentional when we started the project.

Also, there have been recent requests to begin providing read-only
access to certain resources to less-privileged users, so we may, in
Kilo, begin implementing more tunable policies.


  An administrative super user, which has full permissions across all
 projects and should be used with great care.

 I'm not sure how TripleO is dividing operator and admin in the
 undercloud - so I just want to be clear on what you mean when you say
 may not be the undercloud admin. Simply put, to use Ironic in the
 undercloud, you must have admin privileges in the undercloud -- or
 you need to disable Ironic's auth entirely.


 TripleO can presumably deploy any policy.json file they like in the
 undercloud. It's not entirely surprising that some operations that might are
 admin-only in an overcloud

Ironic isn't in the overcloud, so I'm not sure how this comparison is
appropriate.

 might need to be available in the undercloud to
 ordinary users - who, after all, have permissions to create entire
 overclouds - despite them not being admins of the undercloud itself.

Sure. Such a user, who has access to create an overcloud, and would
be an admin in the overcloud they deployed, would be a regular user of
the undercloud, and have access to the undercloud Nova to provision
their workload. They would not be an admin in the undercloud, nor
would they have any need to talk directly to Ironic.

-Devananda

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo][heat][ironic] Heat Ironic resources and ready state orchestration

2014-09-16 Thread Devananda van der Veen
On Tue, Sep 16, 2014 at 11:44 AM, Zane Bitter zbit...@redhat.com wrote:
 On 16/09/14 13:56, Devananda van der Veen wrote:

 On Mon, Sep 15, 2014 at 9:00 AM, Steven Hardy sha...@redhat.com wrote:

 For example, today, I've been looking at the steps required for driving
 autodiscovery:

 https://etherpad.openstack.org/p/Ironic-PoCDiscovery-Juno

 Driving this process looks a lot like application orchestration:

 1. Take some input (IPMI credentials and MAC addresses)
 2. Maybe build an image and ramdisk(could drop credentials in)
 3. Interact with the Ironic API to register nodes in maintenance mode
 4. Boot the nodes, monitor state, wait for a signal back containing some
 data obtained during discovery (same as WaitConditions or
 SoftwareDeployment resources in Heat..)
 5. Shutdown the nodes and mark them ready for use by nova


 My apologies if the following sounds snarky -- but I think there are a
 few misconceptions that need to be cleared up about how and when one
 might use Ironic. I also disagree that 1..5 looks like application
 orchestration. Step 4 is a workflow, which I'll go into in a bit, but
 this doesn't look at all like describing or launching an application
 to me.


 +1 (Although step 3 does sound to me like something that matches Heat's
 scope.)

I think it's a simplistic use case, and Heat supports a lot more
complexity than is necessary to enroll nodes with Ironic.


 Step 1 is just parse a text file.

 Step 2 should be a prerequisite to doing -anything- with Ironic. Those
 images need to be built and loaded in Glance, and the image UUID(s)
 need to be set on each Node in Ironic (or on the Nova flavor, if going
 that route) after enrollment. Sure, Heat can express this
 declaratively (ironic.node.driver_info must contain key:deploy_kernel
 with value:), but are you suggesting that Heat build the images,
 or just take the UUIDs as input?

 Step 3 is, again, just parse a text file

 I'm going to make an assumption here [*], because I think step 4 is
 misleading. You shouldn't boot a node using Ironic -- you do that
 through Nova. And you _dont_ get to specify which node you're booting.
 You ask Nova to provision an _instance_ on a _flavor_ and it picks an
 available node from the pool of nodes that match the request.


 I think your assumption is incorrect. Steve is well aware that provisioning
 a bare-metal Ironic server is done through the Nova API. What he's
 suggesting here is that the nodes would be booted - not Nova-booted, but
 booted in the sense of having power physically applied - while in
 maintenance mode in order to do autodiscovery of their capabilities,

Except simply applying power doesn't, in itself, accomplish anything
besides causing the machine to power on. Ironic will only prepare the
PXE boot environment when initiating a _deploy_.

 which
 is presumably hard to do automatically when they're turned off.

Vendors often have ways to do this while the power is turned off, eg.
via the OOB management interface.

 He's also
 suggesting that Heat could drive this process, which I happen to disagree
 with because it is a workflow not an end state.

+1

 However the main takeaway
 here is that you guys are talking completely past one another, and have been
 for some time.


Perhaps more detail in the expected interactions with Ironic would be
helpful and avoid me making (perhaps incorrect) assumptions.

-D

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo][heat][ironic] Heat Ironic resources and ready state orchestration

2014-09-16 Thread Zane Bitter

On 16/09/14 15:24, Devananda van der Veen wrote:

On Tue, Sep 16, 2014 at 11:44 AM, Zane Bitter zbit...@redhat.com wrote:

On 16/09/14 13:56, Devananda van der Veen wrote:


On Mon, Sep 15, 2014 at 9:00 AM, Steven Hardy sha...@redhat.com wrote:


For example, today, I've been looking at the steps required for driving
autodiscovery:

https://etherpad.openstack.org/p/Ironic-PoCDiscovery-Juno

Driving this process looks a lot like application orchestration:

1. Take some input (IPMI credentials and MAC addresses)
2. Maybe build an image and ramdisk(could drop credentials in)
3. Interact with the Ironic API to register nodes in maintenance mode
4. Boot the nodes, monitor state, wait for a signal back containing some
 data obtained during discovery (same as WaitConditions or
 SoftwareDeployment resources in Heat..)
5. Shutdown the nodes and mark them ready for use by nova



My apologies if the following sounds snarky -- but I think there are a
few misconceptions that need to be cleared up about how and when one
might use Ironic. I also disagree that 1..5 looks like application
orchestration. Step 4 is a workflow, which I'll go into in a bit, but
this doesn't look at all like describing or launching an application
to me.



+1 (Although step 3 does sound to me like something that matches Heat's
scope.)


I think it's a simplistic use case, and Heat supports a lot more
complexity than is necessary to enroll nodes with Ironic.




Step 1 is just parse a text file.

Step 2 should be a prerequisite to doing -anything- with Ironic. Those
images need to be built and loaded in Glance, and the image UUID(s)
need to be set on each Node in Ironic (or on the Nova flavor, if going
that route) after enrollment. Sure, Heat can express this
declaratively (ironic.node.driver_info must contain key:deploy_kernel
with value:), but are you suggesting that Heat build the images,
or just take the UUIDs as input?

Step 3 is, again, just parse a text file

I'm going to make an assumption here [*], because I think step 4 is
misleading. You shouldn't boot a node using Ironic -- you do that
through Nova. And you _dont_ get to specify which node you're booting.
You ask Nova to provision an _instance_ on a _flavor_ and it picks an
available node from the pool of nodes that match the request.



I think your assumption is incorrect. Steve is well aware that provisioning
a bare-metal Ironic server is done through the Nova API. What he's
suggesting here is that the nodes would be booted - not Nova-booted, but
booted in the sense of having power physically applied - while in
maintenance mode in order to do autodiscovery of their capabilities,


Except simply applying power doesn't, in itself, accomplish anything
besides causing the machine to power on. Ironic will only prepare the
PXE boot environment when initiating a _deploy_.


From what I gather elsewhere in this thread, the autodiscovery stuff is 
a proposal for the future, not something that exists in Ironic now, and 
that may be the source of the confusion.


In any case, the etherpad linked at the top of this email was written by 
someone in the Ironic team and _clearly_ describes PXE booting a 
discovery image in maintenance mode in order to obtain hardware 
information about the box.


cheers,
Zane.


which
is presumably hard to do automatically when they're turned off.


Vendors often have ways to do this while the power is turned off, eg.
via the OOB management interface.


He's also
suggesting that Heat could drive this process, which I happen to disagree
with because it is a workflow not an end state.


+1


However the main takeaway
here is that you guys are talking completely past one another, and have been
for some time.



Perhaps more detail in the expected interactions with Ironic would be
helpful and avoid me making (perhaps incorrect) assumptions.

-D

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo][heat][ironic] Heat Ironic resources and ready state orchestration

2014-09-15 Thread James Slagle
On Mon, Sep 15, 2014 at 7:44 AM, Steven Hardy sha...@redhat.com wrote:
 All,

 Starting this thread as a follow-up to a strongly negative reaction by the
 Ironic PTL to my patches[1] adding initial Heat-Ironic integration, and
 subsequent very detailed justification and discussion of why they may be
 useful in this spec[2].

 Back in Atlanta, I had some discussions with folks interesting in making
 ready state[3] preparation of bare-metal resources possible when
 deploying bare-metal nodes via TripleO/Heat/Ironic.

After a cursory reading of the references, it seems there's a couple of issues:
- are the features to move hardware to a ready-state even going to
be in Ironic proper, whether that means in ironic at all or just in
contrib.
- assuming some of the features are there, should Heat have any Ironic
resources given that Ironic's API is admin-only.


 The initial assumption is that there is some discovery step (either
 automatic or static generation of a manifest of nodes), that can be input
 to either Ironic or Heat.

I think it makes a lot of sense to use Heat to do the bulk
registration of nodes via Ironic. I understand the argument that the
Ironic API should be admin-only a little bit for the non-TripleO
case, but for TripleO, we only have admins interfacing with the
Undercloud. The user of a TripleO undercloud is the deployer/operator
and in some scenarios this may not be the undercloud admin. So,
talking about TripleO, I don't really buy that the Ironic API is
admin-only.

Therefore, why not have some declarative Heat resources for things
like Ironic nodes, that the deployer can make use of in a Heat
template to do bulk node registration?

The alternative listed in the spec:

Don’t implement the resources and rely on scripts which directly
interact with the Ironic API, prior to any orchestration via Heat.

would just be a bit silly IMO. That goes against one of the main
drivers of TripleO, which is to use OpenStack wherever possible. Why
go off and write some other thing that is going to parse a
json/yaml/csv of nodes and orchestrate a bunch of Ironic api calls?
Why would it be ok for that other thing to use Ironic's admin-only
API yet claim it's not ok for Heat on the undercloud to do so?


 Following discovery, but before an undercloud deploying OpenStack onto the
 nodes, there are a few steps which may be desired, to get the hardware into
 a state where it's ready and fully optimized for the subsequent deployment:

 - Updating and aligning firmware to meet requirements of qualification or
   site policy
 - Optimization of BIOS configuration to match workloads the node is
   expected to run
 - Management of machine-local storage, e.g configuring local RAID for
   optimal resilience or performance.

 Interfaces to Ironic are landing (or have landed)[4][5][6] which make many
 of these steps possible, but there's no easy way to either encapsulate the
 (currently mostly vendor specific) data associated with each step, or to
 coordinate sequencing of the steps.

 What is required is some tool to take a text definition of the required
 configuration, turn it into a correctly sequenced series of API calls to
 Ironic, expose any data associated with those API calls, and declare
 success or failure on completion.  This is what Heat does.

 So the idea is to create some basic (contrib, disabled by default) Ironic
 heat resources, then explore the idea of orchestrating ready-state
 configuration via Heat.

 Given that Devananda and I have been banging heads over this for some time
 now, I'd like to get broader feedback of the idea, my interpretation of
 ready state applied to the tripleo undercloud, and any alternative
 implementation ideas.

My opinion is that if the features are in Ironic, they should be
exposed via Heat resources for orchestration. If the TripleO case is
too much of a one-off (which I don't really think it is), then sure,
keep it all in contrib so that no one gets confused about why the
resources are there.

-- 
-- James Slagle
--

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo][heat][ironic] Heat Ironic resources and ready state orchestration

2014-09-15 Thread Steven Hardy
On Mon, Sep 15, 2014 at 11:15:21AM -0400, James Slagle wrote:
 On Mon, Sep 15, 2014 at 7:44 AM, Steven Hardy sha...@redhat.com wrote:
  All,
 
  Starting this thread as a follow-up to a strongly negative reaction by the
  Ironic PTL to my patches[1] adding initial Heat-Ironic integration, and
  subsequent very detailed justification and discussion of why they may be
  useful in this spec[2].
 
  Back in Atlanta, I had some discussions with folks interesting in making
  ready state[3] preparation of bare-metal resources possible when
  deploying bare-metal nodes via TripleO/Heat/Ironic.
 
 After a cursory reading of the references, it seems there's a couple of 
 issues:
 - are the features to move hardware to a ready-state even going to
 be in Ironic proper, whether that means in ironic at all or just in
 contrib.

Intially at least, it sounds like much of what ready-state entails would
be via Ironic vendor-specific extensions.

But that's not necessaily an insurmountable problem IMO - encapsulating the
vendor-specific stuff inside some provider resource templates would
actually enable easier substitution of different vendors stuff based on
deployment time choices (user input or data from autodiscovery).

 - assuming some of the features are there, should Heat have any Ironic
 resources given that Ironic's API is admin-only.

Yes, we have some admin-only resources already, but they're restricted to
contrib, as generally we'd rather keep the main tree (default enabled)
resources accessible to all normal users.

So, given that all actors in the undercloud will be admins, as noted in the
spec, I don't think this is a problem given that the primary use-case for
this stuff is TripleO.

  The initial assumption is that there is some discovery step (either
  automatic or static generation of a manifest of nodes), that can be input
  to either Ironic or Heat.
 
 I think it makes a lot of sense to use Heat to do the bulk
 registration of nodes via Ironic. I understand the argument that the
 Ironic API should be admin-only a little bit for the non-TripleO
 case, but for TripleO, we only have admins interfacing with the
 Undercloud. The user of a TripleO undercloud is the deployer/operator
 and in some scenarios this may not be the undercloud admin. So,
 talking about TripleO, I don't really buy that the Ironic API is
 admin-only.

\o/

 Therefore, why not have some declarative Heat resources for things
 like Ironic nodes, that the deployer can make use of in a Heat
 template to do bulk node registration?
 
 The alternative listed in the spec:
 
 Don’t implement the resources and rely on scripts which directly
 interact with the Ironic API, prior to any orchestration via Heat.
 
 would just be a bit silly IMO. That goes against one of the main
 drivers of TripleO, which is to use OpenStack wherever possible. Why
 go off and write some other thing that is going to parse a
 json/yaml/csv of nodes and orchestrate a bunch of Ironic api calls?
 Why would it be ok for that other thing to use Ironic's admin-only
 API yet claim it's not ok for Heat on the undercloud to do so?

Yeah this mirrors my thoughts, I suppose the alternative is somewhat
flippant, but I was trying to illustrate that exposing the Ironic API via
Heat provides some interesting opportunities for reuse of existing Heat
capabilities.

For example, today, I've been looking at the steps required for driving
autodiscovery:

https://etherpad.openstack.org/p/Ironic-PoCDiscovery-Juno

Driving this process looks a lot like application orchestration:

1. Take some input (IPMI credentials and MAC addresses)
2. Maybe build an image and ramdisk(could drop credentials in)
3. Interact with the Ironic API to register nodes in maintenance mode
4. Boot the nodes, monitor state, wait for a signal back containing some
   data obtained during discovery (same as WaitConditions or
   SoftwareDeployment resources in Heat..)
5. Shutdown the nodes and mark them ready for use by nova

At some point near the end of this sequence, you could optionally insert
the ready state workflow described in the spec.

So I guess my question then becomes, regardless of ready state, what is
expected to drive the steps above if it's not Heat?

I'm not really clear what the plan is but it certainly seems like it'd be
a win from a TripleO perspective if the exiting tooling (e.g Heat) could be
reused?

Steve

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo][heat][ironic] Heat Ironic resources and ready state orchestration

2014-09-15 Thread Keith Basil
On Sep 15, 2014, at 12:00 PM, Steven Hardy wrote:

 On Mon, Sep 15, 2014 at 11:15:21AM -0400, James Slagle wrote:
 On Mon, Sep 15, 2014 at 7:44 AM, Steven Hardy sha...@redhat.com wrote:
 All,
 
 Starting this thread as a follow-up to a strongly negative reaction by the
 Ironic PTL to my patches[1] adding initial Heat-Ironic integration, and
 subsequent very detailed justification and discussion of why they may be
 useful in this spec[2].
 
 Back in Atlanta, I had some discussions with folks interesting in making
 ready state[3] preparation of bare-metal resources possible when
 deploying bare-metal nodes via TripleO/Heat/Ironic.

Additional thinking and background behind Ready State.

http://robhirschfeld.com/2014/04/25/ready-state-infrastructure/

-k


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo][heat][ironic] Heat Ironic resources and ready state orchestration

2014-09-15 Thread Clint Byrum
Excerpts from James Slagle's message of 2014-09-15 08:15:21 -0700:
 On Mon, Sep 15, 2014 at 7:44 AM, Steven Hardy sha...@redhat.com wrote:
  All,
 
  Starting this thread as a follow-up to a strongly negative reaction by the
  Ironic PTL to my patches[1] adding initial Heat-Ironic integration, and
  subsequent very detailed justification and discussion of why they may be
  useful in this spec[2].
 
  Back in Atlanta, I had some discussions with folks interesting in making
  ready state[3] preparation of bare-metal resources possible when
  deploying bare-metal nodes via TripleO/Heat/Ironic.
 
 After a cursory reading of the references, it seems there's a couple of 
 issues:
 - are the features to move hardware to a ready-state even going to
 be in Ironic proper, whether that means in ironic at all or just in
 contrib.
 - assuming some of the features are there, should Heat have any Ironic
 resources given that Ironic's API is admin-only.
 
 
  The initial assumption is that there is some discovery step (either
  automatic or static generation of a manifest of nodes), that can be input
  to either Ironic or Heat.
 
 I think it makes a lot of sense to use Heat to do the bulk
 registration of nodes via Ironic. I understand the argument that the
 Ironic API should be admin-only a little bit for the non-TripleO
 case, but for TripleO, we only have admins interfacing with the
 Undercloud. The user of a TripleO undercloud is the deployer/operator
 and in some scenarios this may not be the undercloud admin. So,
 talking about TripleO, I don't really buy that the Ironic API is
 admin-only.
 
 Therefore, why not have some declarative Heat resources for things
 like Ironic nodes, that the deployer can make use of in a Heat
 template to do bulk node registration?
 
 The alternative listed in the spec:
 
 Don’t implement the resources and rely on scripts which directly
 interact with the Ironic API, prior to any orchestration via Heat.
 
 would just be a bit silly IMO. That goes against one of the main
 drivers of TripleO, which is to use OpenStack wherever possible. Why
 go off and write some other thing that is going to parse a
 json/yaml/csv of nodes and orchestrate a bunch of Ironic api calls?
 Why would it be ok for that other thing to use Ironic's admin-only
 API yet claim it's not ok for Heat on the undercloud to do so?
 

An alternative that is missed, is to just define a bulk loading format
for hardware, or adopt an existing one (I find it hard to believe there
isn't already an open format for this), and make use of it in Ironic.

The analogy I'd use is shipping dry goods in a refrigerated truck.
It's heavier, has a bit less capacity, and unnecessary features.  If all
you have is the refrigerated truck, ok. But we're talking about _building_
a special dry-goods add-on to our refrigerated truck (Heat) to avoid
building the same thing into the regular trucks we already have (Ironic).

  Following discovery, but before an undercloud deploying OpenStack onto the
  nodes, there are a few steps which may be desired, to get the hardware into
  a state where it's ready and fully optimized for the subsequent deployment:
 
  - Updating and aligning firmware to meet requirements of qualification or
site policy
  - Optimization of BIOS configuration to match workloads the node is
expected to run
  - Management of machine-local storage, e.g configuring local RAID for
optimal resilience or performance.
 
  Interfaces to Ironic are landing (or have landed)[4][5][6] which make many
  of these steps possible, but there's no easy way to either encapsulate the
  (currently mostly vendor specific) data associated with each step, or to
  coordinate sequencing of the steps.
 
  What is required is some tool to take a text definition of the required
  configuration, turn it into a correctly sequenced series of API calls to
  Ironic, expose any data associated with those API calls, and declare
  success or failure on completion.  This is what Heat does.
 
  So the idea is to create some basic (contrib, disabled by default) Ironic
  heat resources, then explore the idea of orchestrating ready-state
  configuration via Heat.
 
  Given that Devananda and I have been banging heads over this for some time
  now, I'd like to get broader feedback of the idea, my interpretation of
  ready state applied to the tripleo undercloud, and any alternative
  implementation ideas.
 
 My opinion is that if the features are in Ironic, they should be
 exposed via Heat resources for orchestration. If the TripleO case is
 too much of a one-off (which I don't really think it is), then sure,
 keep it all in contrib so that no one gets confused about why the
 resources are there.
 

And I think if this is a common thing that Ironic users need to do,
then Ironic should do it, not Heat.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org

Re: [openstack-dev] [tripleo][heat][ironic] Heat Ironic resources and ready state orchestration

2014-09-15 Thread Steven Hardy
On Mon, Sep 15, 2014 at 09:50:24AM -0700, Clint Byrum wrote:
 Excerpts from Steven Hardy's message of 2014-09-15 04:44:24 -0700:
  All,
  
  Starting this thread as a follow-up to a strongly negative reaction by the
  Ironic PTL to my patches[1] adding initial Heat-Ironic integration, and
  subsequent very detailed justification and discussion of why they may be
  useful in this spec[2].
  
  Back in Atlanta, I had some discussions with folks interesting in making
  ready state[3] preparation of bare-metal resources possible when
  deploying bare-metal nodes via TripleO/Heat/Ironic.
  
  The initial assumption is that there is some discovery step (either
  automatic or static generation of a manifest of nodes), that can be input
  to either Ironic or Heat.
  
  Following discovery, but before an undercloud deploying OpenStack onto the
  nodes, there are a few steps which may be desired, to get the hardware into
  a state where it's ready and fully optimized for the subsequent deployment:
  
  - Updating and aligning firmware to meet requirements of qualification or
site policy
  - Optimization of BIOS configuration to match workloads the node is
expected to run
  - Management of machine-local storage, e.g configuring local RAID for
optimal resilience or performance.
  
  Interfaces to Ironic are landing (or have landed)[4][5][6] which make many
  of these steps possible, but there's no easy way to either encapsulate the
  (currently mostly vendor specific) data associated with each step, or to
  coordinate sequencing of the steps.
  
 
 First, Ironic is hidden under Nova as far as TripleO is concerned. So
 mucking with the servers underneath Nova during deployment is a difficult
 proposition. Would I look up the Ironic node ID of the nova server,
 and then optimize it for the workload after the workload arrived? Why
 wouldn't I just do that optimization before the deployment?

That's exactly what I'm proposing - a series of preparatory steps performed
before the node is visible to nova, before the deployment.

The whole point is that Ironic is hidden under nova, and provides no way to
perform these pre-deploy steps via interaction with nova.

 
  What is required is some tool to take a text definition of the required
  configuration, turn it into a correctly sequenced series of API calls to
  Ironic, expose any data associated with those API calls, and declare
  success or failure on completion.  This is what Heat does.
  
 
 I'd rather see Ironic define or adopt a narrow scope document format
 that it can consume for bulk loading. Heat is extremely generic, and thus
 carries a ton of complexity for what is probably doable with a CSV file.

Perhaps you can read the spec - it's not really about the bulk-load part,
it's about orchestrating the steps to prepare the node, after it's
registered with Ironic, but before it's ready to have the stuff deployed to
it.

What tool do you think will just do that optimization before the
deployment? (snark not intended, I genuinely want to know, is it scripts
in TripleO, some sysadmin pre-deploy steps, magic in Ironic?)

  So the idea is to create some basic (contrib, disabled by default) Ironic
  heat resources, then explore the idea of orchestrating ready-state
  configuration via Heat.
  
  Given that Devananda and I have been banging heads over this for some time
  now, I'd like to get broader feedback of the idea, my interpretation of
  ready state applied to the tripleo undercloud, and any alternative
  implementation ideas.
  
 
 I think there may be value in being able to tie Ironic calls to other
 OpenStack API calls. I'm dubious that this is an important idea, but
 I think if somebody wants to step forward with their use case for it,
 then the resources might make sense. However, I realy don't see the
 _enrollment_ phase as capturing any value that isn't entirely offset by
 added complexity.

I've stepped forward with a use-case for it, in the spec, and it's not
about the enrollment phase (although Heat could do that, if for example it
was used to manage the series of steps required for autodiscovery).

If there's a better way to do this, I'm very happy to hear about it, but I
am getting a little tired of folks telling me the idea is dubious without
offering any alternative solution to the requirement :(

Steve

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo][heat][ironic] Heat Ironic resources and ready state orchestration

2014-09-15 Thread Jay Faulkner
Steven,

It's important to note that two of the blueprints you reference: 

https://blueprints.launchpad.net/ironic/+spec/drac-raid-mgmt
https://blueprints.launchpad.net/ironic/+spec/drac-hw-discovery

are both very unlikely to land in Ironic -- these are configuration and 
discovery pieces that best fit inside a operator-deployed CMDB, rather than 
Ironic trying to extend its scope significantly to include these type of 
functions. I expect the scoping or Ironic with regards to hardware 
discovery/interrogation as well as configuration of hardware (like I will 
outline below) to be hot topics in Ironic design summit sessions at Paris.

A good way of looking at it is that Ironic is responsible for hardware *at 
provision time*. Registering the nodes in Ironic, as well as hardware 
settings/maintenance/etc while a workload is provisioned is left to the 
operators' CMDB. 

This means what Ironic *can* do is modify the configuration of a node at 
provision time based on information passed down the provisioning pipeline. For 
instance, if you wanted to configure certain firmware pieces at provision time, 
you could do something like this:

Nova flavor sets capability:vm_hypervisor in the flavor that maps to the Ironic 
node. This would map to an Ironic driver that exposes vm_hypervisor as a 
capability, and upon seeing capability:vm_hypervisor has been requested, could 
then configure the firmware/BIOS of the machine to 'hypervisor friendly' 
settings, such as VT bit on and Turbo mode off. You could map multiple 
different combinations of capabilities as different Ironic flavors, and have 
them all represent different configurations of the same pool of nodes. So, you 
end up with two categories of abilities: inherent abilities of the node (such 
as amount of RAM or CPU installed), and configurable abilities (i.e. things 
than can be turned on/off at provision time on demand) -- or perhaps, in the 
future, even things like RAM and CPU will be dynamically provisioned into nodes 
at provision time. 

-Jay Faulkner


From: Steven Hardy sha...@redhat.com
Sent: Monday, September 15, 2014 4:44 AM
To: openstack-dev@lists.openstack.org
Subject: [openstack-dev] [tripleo][heat][ironic] Heat Ironic resources and 
ready state orchestration

All,

Starting this thread as a follow-up to a strongly negative reaction by the
Ironic PTL to my patches[1] adding initial Heat-Ironic integration, and
subsequent very detailed justification and discussion of why they may be
useful in this spec[2].

Back in Atlanta, I had some discussions with folks interesting in making
ready state[3] preparation of bare-metal resources possible when
deploying bare-metal nodes via TripleO/Heat/Ironic.

The initial assumption is that there is some discovery step (either
automatic or static generation of a manifest of nodes), that can be input
to either Ironic or Heat.

Following discovery, but before an undercloud deploying OpenStack onto the
nodes, there are a few steps which may be desired, to get the hardware into
a state where it's ready and fully optimized for the subsequent deployment:

- Updating and aligning firmware to meet requirements of qualification or
  site policy
- Optimization of BIOS configuration to match workloads the node is
  expected to run
- Management of machine-local storage, e.g configuring local RAID for
  optimal resilience or performance.

Interfaces to Ironic are landing (or have landed)[4][5][6] which make many
of these steps possible, but there's no easy way to either encapsulate the
(currently mostly vendor specific) data associated with each step, or to
coordinate sequencing of the steps.

What is required is some tool to take a text definition of the required
configuration, turn it into a correctly sequenced series of API calls to
Ironic, expose any data associated with those API calls, and declare
success or failure on completion.  This is what Heat does.

So the idea is to create some basic (contrib, disabled by default) Ironic
heat resources, then explore the idea of orchestrating ready-state
configuration via Heat.

Given that Devananda and I have been banging heads over this for some time
now, I'd like to get broader feedback of the idea, my interpretation of
ready state applied to the tripleo undercloud, and any alternative
implementation ideas.

Thanks!

Steve

[1] https://review.openstack.org/#/c/104222/
[2] https://review.openstack.org/#/c/120778/
[3] http://robhirschfeld.com/2014/04/25/ready-state-infrastructure/
[4] https://blueprints.launchpad.net/ironic/+spec/drac-management-driver
[5] https://blueprints.launchpad.net/ironic/+spec/drac-raid-mgmt
[6] https://blueprints.launchpad.net/ironic/+spec/drac-hw-discovery

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list

Re: [openstack-dev] [tripleo][heat][ironic] Heat Ironic resources and ready state orchestration

2014-09-15 Thread James Slagle
On Mon, Sep 15, 2014 at 12:59 PM, Clint Byrum cl...@fewbar.com wrote:
 Excerpts from James Slagle's message of 2014-09-15 08:15:21 -0700:
 On Mon, Sep 15, 2014 at 7:44 AM, Steven Hardy sha...@redhat.com wrote:
  Following discovery, but before an undercloud deploying OpenStack onto the
  nodes, there are a few steps which may be desired, to get the hardware into
  a state where it's ready and fully optimized for the subsequent deployment:
 
  - Updating and aligning firmware to meet requirements of qualification or
site policy
  - Optimization of BIOS configuration to match workloads the node is
expected to run
  - Management of machine-local storage, e.g configuring local RAID for
optimal resilience or performance.
 
  Interfaces to Ironic are landing (or have landed)[4][5][6] which make many
  of these steps possible, but there's no easy way to either encapsulate the
  (currently mostly vendor specific) data associated with each step, or to
  coordinate sequencing of the steps.
 
  What is required is some tool to take a text definition of the required
  configuration, turn it into a correctly sequenced series of API calls to
  Ironic, expose any data associated with those API calls, and declare
  success or failure on completion.  This is what Heat does.
 
  So the idea is to create some basic (contrib, disabled by default) Ironic
  heat resources, then explore the idea of orchestrating ready-state
  configuration via Heat.
 
  Given that Devananda and I have been banging heads over this for some time
  now, I'd like to get broader feedback of the idea, my interpretation of
  ready state applied to the tripleo undercloud, and any alternative
  implementation ideas.

 My opinion is that if the features are in Ironic, they should be
 exposed via Heat resources for orchestration. If the TripleO case is
 too much of a one-off (which I don't really think it is), then sure,
 keep it all in contrib so that no one gets confused about why the
 resources are there.


 And I think if this is a common thing that Ironic users need to do,
 then Ironic should do it, not Heat.

I would think Heat would be well suited for the case where you want to
orchestrate a workflow on top of existing Ironic API's for managing
the infrastructure lifecycle, of which attaining ready state is one
such use case.

It's a fair point that if these things are common enough to all users
that they should just be done in Ironic. To what extent such an API in
Ironic would just end up orchestrating other Ironic API's the same way
Heat might do it would be hard to tell. It seems like that's the added
complexity in my view vs. a set of simple Heat resources and taking
advantage of all the orchestration that Heat already offers.

I know this use case isn't just about enrolling nodes (apologies if I
implied that in my earlier response). That was just one such use that
jumped out at me in which it might be nice to use Heat. I think about
how os-cloud-config registers nodes today. It has to create the node,
then create the port (2 separate calls). And, it also needs the
ability to update registered nodes[1]. This logic is going to end up
living in os-cloud-config.

And perhaps the answer is no, but it seems to me Heat could do this
sort of thing easier already if it had the resources defined to do so.
It'd be neat to have a yaml file of all your defined nodes, and use
stack-create to register them in Ironic. When you need to add some new
ones, update the yaml, and then stack-update.

[1] http://lists.openstack.org/pipermail/openstack-dev/2014-August/043782.html
(thread crossed into Sept)

-- 
-- James Slagle
--

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo][heat][ironic] Heat Ironic resources and ready state orchestration

2014-09-15 Thread Jim Rollenhagen
On Mon, Sep 15, 2014 at 12:44:24PM +0100, Steven Hardy wrote:
 All,
 
 Starting this thread as a follow-up to a strongly negative reaction by the
 Ironic PTL to my patches[1] adding initial Heat-Ironic integration, and
 subsequent very detailed justification and discussion of why they may be
 useful in this spec[2].
 
 Back in Atlanta, I had some discussions with folks interesting in making
 ready state[3] preparation of bare-metal resources possible when
 deploying bare-metal nodes via TripleO/Heat/Ironic.
 
 The initial assumption is that there is some discovery step (either
 automatic or static generation of a manifest of nodes), that can be input
 to either Ironic or Heat.

We've discussed this a *lot* within Ironic, and have decided that
auto-discovery (with registration) is out of scope for Ironic. In my
opinion, this is straightforward enough for operators to write small
scripts to take a CSV/JSON/whatever file and register the nodes in that
file with Ironic. This is what we've done at Rackspace, and it's really
not that annoying; the hard part is dealing with incorrect data from
the (vendor|DC team|whatever).

That said, I like the thought of Ironic having a bulk-registration
feature with some sort of specified format (I imagine this would just be
a simple JSON list of node objects).

We are likely doing a session on discovery in general in Paris. It seems
like the main topic will be about how to interface with external
inventory management systems to coordinate node discovery. Maybe Heat is
a valid tool to integrate with here, maybe not.

 Following discovery, but before an undercloud deploying OpenStack onto the
 nodes, there are a few steps which may be desired, to get the hardware into
 a state where it's ready and fully optimized for the subsequent deployment:

These pieces are mostly being done downstream, and (IMO) in scope for
Ironic in the Kilo cycle. More below.

 - Updating and aligning firmware to meet requirements of qualification or
   site policy

Rackspace does this today as part of what we call decommissioning.
There are patches up for review for both ironic-python-agent (IPA) [1] and
Ironic [2] itself. We have support for 1) flashing a BIOS on a node, and
2) Writing a set of BIOS settings to a node (these are embedded in the agent
image as a set, not through an Ironic API). These are both implemented as
a hardware manager plugin, and so can easily be vendor-specific.

I expect this to land upstream in the Kilo release.

 - Optimization of BIOS configuration to match workloads the node is
   expected to run

The Ironic team has also discussed this, mostly at the last mid-cycle
meetup. We'll likely have a session on capabilities, which we think
might be the best way to handle this case. Essentially, a node can be
tagged with arbitrary capabilities, e.g. hypervisor, which Nova
(flavors?) could use for scheduling, and Ironic drivers could use to do
per-provisioning work, like setting BIOS settings. This may even tie in
with the next point.

Looks like Jay just ninja'd me a bit on this point. :)

 - Management of machine-local storage, e.g configuring local RAID for
   optimal resilience or performance.

I don't see why Ironic couldn't do something with this in Kilo. It's
dangerously close to the inventory management line, however I think
it's reasonable for a user to specify that his or her root partition
should be on a RAID or a specific disk out of many in the node.

 Interfaces to Ironic are landing (or have landed)[4][5][6] which make many
 of these steps possible, but there's no easy way to either encapsulate the
 (currently mostly vendor specific) data associated with each step, or to
 coordinate sequencing of the steps.

It's important to remember that just because a blueprint/spec exists,
does not mean it will be approved. :) I don't expect the DRAC
discovery blueprint to go through, and the DRAC RAID blueprint is
questionable, with regards to scope.

 What is required is some tool to take a text definition of the required
 configuration, turn it into a correctly sequenced series of API calls to
 Ironic, expose any data associated with those API calls, and declare
 success or failure on completion.  This is what Heat does.

This is a fair point, however none of these use cases have code landed
in mainline Ironic, and certainly don't have APIs exposed, with the
exception of node registration. Is it useful to start writing plumbing to
talk to APIs that don't exist?

All that said, I don't think it's unreasonable for Heat to talk directly
to Ironic, but only if there's a valid use case that Ironic can't (or
won't) provide a solution for.

// jim

[1] https://review.openstack.org/104379
[2] 
https://review.openstack.org/#/q/status:open+project:openstack/ironic+branch:master+topic:decom-nodes,n,z

 So the idea is to create some basic (contrib, disabled by default) Ironic
 heat resources, then explore the idea of orchestrating ready-state
 configuration via Heat.
 
 Given that 

Re: [openstack-dev] [tripleo][heat][ironic] Heat Ironic resources and ready state orchestration

2014-09-15 Thread Dmitry Tantsur
On Mon, 2014-09-15 at 11:04 -0700, Jim Rollenhagen wrote:
 On Mon, Sep 15, 2014 at 12:44:24PM +0100, Steven Hardy wrote:
  All,
  
  Starting this thread as a follow-up to a strongly negative reaction by the
  Ironic PTL to my patches[1] adding initial Heat-Ironic integration, and
  subsequent very detailed justification and discussion of why they may be
  useful in this spec[2].
  
  Back in Atlanta, I had some discussions with folks interesting in making
  ready state[3] preparation of bare-metal resources possible when
  deploying bare-metal nodes via TripleO/Heat/Ironic.
  
  The initial assumption is that there is some discovery step (either
  automatic or static generation of a manifest of nodes), that can be input
  to either Ironic or Heat.
 
 We've discussed this a *lot* within Ironic, and have decided that
 auto-discovery (with registration) is out of scope for Ironic. 
Even if there is such an agreement, it's the first time I hear about it.
All previous discussions _I'm aware of_ (e.g. midcycle) ended up with
we can discover only things that are required for scheduling. When did
it change?

 In my
 opinion, this is straightforward enough for operators to write small
 scripts to take a CSV/JSON/whatever file and register the nodes in that
 file with Ironic. This is what we've done at Rackspace, and it's really
 not that annoying; the hard part is dealing with incorrect data from
 the (vendor|DC team|whatever).
Provided this CSV contains all the required data, not only IPMI
credentials, which IIRC is often the case.

 
 That said, I like the thought of Ironic having a bulk-registration
 feature with some sort of specified format (I imagine this would just be
 a simple JSON list of node objects).
 
 We are likely doing a session on discovery in general in Paris. It seems
 like the main topic will be about how to interface with external
 inventory management systems to coordinate node discovery. Maybe Heat is
 a valid tool to integrate with here, maybe not.
 
  Following discovery, but before an undercloud deploying OpenStack onto the
  nodes, there are a few steps which may be desired, to get the hardware into
  a state where it's ready and fully optimized for the subsequent deployment:
 
 These pieces are mostly being done downstream, and (IMO) in scope for
 Ironic in the Kilo cycle. More below.
 
  - Updating and aligning firmware to meet requirements of qualification or
site policy
 
 Rackspace does this today as part of what we call decommissioning.
 There are patches up for review for both ironic-python-agent (IPA) [1] and
 Ironic [2] itself. We have support for 1) flashing a BIOS on a node, and
 2) Writing a set of BIOS settings to a node (these are embedded in the agent
 image as a set, not through an Ironic API). These are both implemented as
 a hardware manager plugin, and so can easily be vendor-specific.
 
 I expect this to land upstream in the Kilo release.
 
  - Optimization of BIOS configuration to match workloads the node is
expected to run
 
 The Ironic team has also discussed this, mostly at the last mid-cycle
 meetup. We'll likely have a session on capabilities, which we think
 might be the best way to handle this case. Essentially, a node can be
 tagged with arbitrary capabilities, e.g. hypervisor, which Nova
 (flavors?) could use for scheduling, and Ironic drivers could use to do
 per-provisioning work, like setting BIOS settings. This may even tie in
 with the next point.
 
 Looks like Jay just ninja'd me a bit on this point. :)
 
  - Management of machine-local storage, e.g configuring local RAID for
optimal resilience or performance.
 
 I don't see why Ironic couldn't do something with this in Kilo. It's
 dangerously close to the inventory management line, however I think
 it's reasonable for a user to specify that his or her root partition
 should be on a RAID or a specific disk out of many in the node.
 
  Interfaces to Ironic are landing (or have landed)[4][5][6] which make many
  of these steps possible, but there's no easy way to either encapsulate the
  (currently mostly vendor specific) data associated with each step, or to
  coordinate sequencing of the steps.
 
 It's important to remember that just because a blueprint/spec exists,
 does not mean it will be approved. :) I don't expect the DRAC
 discovery blueprint to go through, and the DRAC RAID blueprint is
 questionable, with regards to scope.
 
  What is required is some tool to take a text definition of the required
  configuration, turn it into a correctly sequenced series of API calls to
  Ironic, expose any data associated with those API calls, and declare
  success or failure on completion.  This is what Heat does.
 
 This is a fair point, however none of these use cases have code landed
 in mainline Ironic, and certainly don't have APIs exposed, with the
 exception of node registration. Is it useful to start writing plumbing to
 talk to APIs that don't exist?
 
 All that said, I don't think it's 

Re: [openstack-dev] [tripleo][heat][ironic] Heat Ironic resources and ready state orchestration

2014-09-15 Thread Zane Bitter

On 15/09/14 12:00, Steven Hardy wrote:

For example, today, I've been looking at the steps required for driving
autodiscovery:

https://etherpad.openstack.org/p/Ironic-PoCDiscovery-Juno

Driving this process looks a lot like application orchestration:

1. Take some input (IPMI credentials and MAC addresses)
2. Maybe build an image and ramdisk(could drop credentials in)
3. Interact with the Ironic API to register nodes in maintenance mode
4. Boot the nodes, monitor state, wait for a signal back containing some
data obtained during discovery (same as WaitConditions or
SoftwareDeployment resources in Heat..)
5. Shutdown the nodes and mark them ready for use by nova

At some point near the end of this sequence, you could optionally insert
the ready state workflow described in the spec.

So I guess my question then becomes, regardless of ready state, what is
expected to drive the steps above if it's not Heat?


Maybe you're not explaining it the way you intended to, because I was 
more or less on board until I read this, which doesn't sound like Heat's 
domain at all. It sounds like a workflow, of the kind that could in 
future be handled by something like Mistral.


Only step 3 sounds like a job for Heat (i.e. involves a declarative 
representation of some underlying resources). I think it certainly makes 
sense to use Heat for that if it can have some meaningful input into the 
lifecycle (i.e. you can usefully update to add and remove nodes, and 
unregister them all on delete). If not then it's hard to see what value 
Heat adds over and above a for-loop.


For the record, IMO the fact that the Ironic API is operator-facing 
should *not* be an obstacle here. We have an established policy for how 
to handle such APIs - write the plugins, maintain them in /contrib - and 
this proposal is entirely consistent with that.


cheers,
Zane.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo][heat][ironic] Heat Ironic resources and ready state orchestration

2014-09-15 Thread Clint Byrum
Excerpts from Steven Hardy's message of 2014-09-15 10:10:05 -0700:
 On Mon, Sep 15, 2014 at 09:50:24AM -0700, Clint Byrum wrote:
  Excerpts from Steven Hardy's message of 2014-09-15 04:44:24 -0700:
   All,
   
   Starting this thread as a follow-up to a strongly negative reaction by the
   Ironic PTL to my patches[1] adding initial Heat-Ironic integration, and
   subsequent very detailed justification and discussion of why they may be
   useful in this spec[2].
   
   Back in Atlanta, I had some discussions with folks interesting in making
   ready state[3] preparation of bare-metal resources possible when
   deploying bare-metal nodes via TripleO/Heat/Ironic.
   
   The initial assumption is that there is some discovery step (either
   automatic or static generation of a manifest of nodes), that can be input
   to either Ironic or Heat.
   
   Following discovery, but before an undercloud deploying OpenStack onto the
   nodes, there are a few steps which may be desired, to get the hardware 
   into
   a state where it's ready and fully optimized for the subsequent 
   deployment:
   
   - Updating and aligning firmware to meet requirements of qualification or
 site policy
   - Optimization of BIOS configuration to match workloads the node is
 expected to run
   - Management of machine-local storage, e.g configuring local RAID for
 optimal resilience or performance.
   
   Interfaces to Ironic are landing (or have landed)[4][5][6] which make many
   of these steps possible, but there's no easy way to either encapsulate the
   (currently mostly vendor specific) data associated with each step, or to
   coordinate sequencing of the steps.
   
  
  First, Ironic is hidden under Nova as far as TripleO is concerned. So
  mucking with the servers underneath Nova during deployment is a difficult
  proposition. Would I look up the Ironic node ID of the nova server,
  and then optimize it for the workload after the workload arrived? Why
  wouldn't I just do that optimization before the deployment?
 
 That's exactly what I'm proposing - a series of preparatory steps performed
 before the node is visible to nova, before the deployment.
 

Ok good, so I didn't misunderstand. I'm having trouble seeing where Heat
is a good fit there.

 The whole point is that Ironic is hidden under nova, and provides no way to
 perform these pre-deploy steps via interaction with nova.
 
  
   What is required is some tool to take a text definition of the required
   configuration, turn it into a correctly sequenced series of API calls to
   Ironic, expose any data associated with those API calls, and declare
   success or failure on completion.  This is what Heat does.
   
  
  I'd rather see Ironic define or adopt a narrow scope document format
  that it can consume for bulk loading. Heat is extremely generic, and thus
  carries a ton of complexity for what is probably doable with a CSV file.
 
 Perhaps you can read the spec - it's not really about the bulk-load part,
 it's about orchestrating the steps to prepare the node, after it's
 registered with Ironic, but before it's ready to have the stuff deployed to
 it.
 

Sounds like workflow to me. :-P

 What tool do you think will just do that optimization before the
 deployment? (snark not intended, I genuinely want to know, is it scripts
 in TripleO, some sysadmin pre-deploy steps, magic in Ironic?)


If it can all be done by calls to the ironic client with the node ID and
parameters from the user, I'd suggest that this is a simple workflow
and can be done in the step prior to 'heat stack-create'. I don't see
any reason to keep a bunch of records around in Heat to describe what
happened, identically, for Ironic nodes. It is an ephemeral step in the
evolution of the system, not something we need to edit on a regular basis.

My new bar for whether something is a good fit for Heat is what happens
to my workload when I update it. If I go into my Ironic pre-registration
stack and change things around, the likely case is that my box reboots
to re-apply BIOS updates with the new paramters. And there is a missing
dependency expression when using the orchestration tool to do the
workflow job. It may actually be necessary to always do these things to
the hardware in a certain sequence. But editting the Heat template and
updating has no way to express that.

To contrast this with developing it in a workflow control language
(like bash), it is imperative so I am consciously deciding to re-apply
those things by running it. If I only want to do one step, I just do
the one step.

Basically, the imperative model is rigid, sharp, and pointy, but the
declarative model is soft and maleable, and full of unexpected sharp
pointy things. I think users are more comfortable with knowing where
the sharp and pointy things are, than stumbling on them.

   So the idea is to create some basic (contrib, disabled by default) Ironic
   heat resources, then explore the idea of orchestrating ready-state
   

Re: [openstack-dev] [tripleo][heat][ironic] Heat Ironic resources and ready state orchestration

2014-09-15 Thread Steven Hardy
On Mon, Sep 15, 2014 at 05:51:43PM +, Jay Faulkner wrote:
 Steven,
 
 It's important to note that two of the blueprints you reference: 
 
 https://blueprints.launchpad.net/ironic/+spec/drac-raid-mgmt
 https://blueprints.launchpad.net/ironic/+spec/drac-hw-discovery
 
 are both very unlikely to land in Ironic -- these are configuration and 
 discovery pieces that best fit inside a operator-deployed CMDB, rather than 
 Ironic trying to extend its scope significantly to include these type of 
 functions. I expect the scoping or Ironic with regards to hardware 
 discovery/interrogation as well as configuration of hardware (like I will 
 outline below) to be hot topics in Ironic design summit sessions at Paris.

Hmm, okay - not sure I really get how a CMDB is going to help you configure
your RAID arrays in an automated way?

Or are you subscribing to the legacy datacentre model where a sysadmin
configures a bunch of boxes via whatever method, puts their details into
the CMDB, then feeds those details into Ironic?

 A good way of looking at it is that Ironic is responsible for hardware *at 
 provision time*. Registering the nodes in Ironic, as well as hardware 
 settings/maintenance/etc while a workload is provisioned is left to the 
 operators' CMDB. 
 
 This means what Ironic *can* do is modify the configuration of a node at 
 provision time based on information passed down the provisioning pipeline. 
 For instance, if you wanted to configure certain firmware pieces at provision 
 time, you could do something like this:
 
 Nova flavor sets capability:vm_hypervisor in the flavor that maps to the 
 Ironic node. This would map to an Ironic driver that exposes vm_hypervisor as 
 a capability, and upon seeing capability:vm_hypervisor has been requested, 
 could then configure the firmware/BIOS of the machine to 'hypervisor 
 friendly' settings, such as VT bit on and Turbo mode off. You could map 
 multiple different combinations of capabilities as different Ironic flavors, 
 and have them all represent different configurations of the same pool of 
 nodes. So, you end up with two categories of abilities: inherent abilities of 
 the node (such as amount of RAM or CPU installed), and configurable abilities 
 (i.e. things than can be turned on/off at provision time on demand) -- or 
 perhaps, in the future, even things like RAM and CPU will be dynamically 
 provisioned into nodes at provision time.

So you advocate pushing all the vendor-specific stuff down into various
Ironic drivers, interesting - is any of what you describe above possible
today?

Steve

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo][heat][ironic] Heat Ironic resources and ready state orchestration

2014-09-15 Thread Jay Pipes

On 09/15/2014 04:08 PM, Steven Hardy wrote:

On Mon, Sep 15, 2014 at 05:51:43PM +, Jay Faulkner wrote:

Steven,

It's important to note that two of the blueprints you reference:

https://blueprints.launchpad.net/ironic/+spec/drac-raid-mgmt
https://blueprints.launchpad.net/ironic/+spec/drac-hw-discovery

are both very unlikely to land in Ironic -- these are configuration
and discovery pieces that best fit inside a operator-deployed CMDB,
rather than Ironic trying to extend its scope significantly to
include these type of functions. I expect the scoping or Ironic
with regards to hardware discovery/interrogation as well as
configuration of hardware (like I will outline below) to be hot
topics in Ironic design summit sessions at Paris.


Hmm, okay - not sure I really get how a CMDB is going to help you
configure your RAID arrays in an automated way?


FWIW, we used Chef to configure all of our RAID stuff at ATT. It worked 
just fine for Dell and LSI controllers.


-jay


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev