Re: [openstack-dev] [ironic] [inspector] Auto discovery extension for Ironic Inspector

[Yuiko Takada]

Hi,

Using fake driver means we need a manual step to set it to something
> non-fake :) and the current introspection process already has 1 manual step
> (enrolling nodes), so I'd like autodiscovery to require 0 of them (at least
> for the majority of users).

Exactly. I recognize that our purpose of auto-discovery is eradicating
manual steps.

I prefer option 2, but as Sam said, it's not suitable for mixed driver
environment.
Then, is it impossible that detecting(judging?) driver from node info?
For example, if Vendor is XXX and Product Name is YYY, then driver is ZZZ.
What do you think?

Best Regards,
Yuiko Takada


2015-11-18 22:27 GMT+09:00 Dmitry Tantsur :

> I have to admin I forgot about this thread. Please find comments inline.
>
> On 11/06/2015 05:25 PM, Bruno Cornec wrote:
>
>> Hello,
>>
>> Pavlo Shchelokovskyy said on Tue, Nov 03, 2015 at 09:41:51PM +:
>>
>>> For auto-setting driver options on enrollment, I would vote for option 2
>>> with default being fake driver + optional CMDB integration. This would
>>> ease
>>> managing a homogeneous pool of BMs, but still (using fake driver or data
>>> from CMDB) work reasonably well in heterogeneous case.
>>>
>>
> Using fake driver means we need a manual step to set it to something
> non-fake :) and the current introspection process already has 1 manual step
> (enrolling nodes), so I'd like autodiscovery to require 0 of them (at least
> for the majority of users).
>
>
>>> As for setting a random password, CMDB integration is crucial IMO. Large
>>> deployments usually have some sort of it already, and it must serve as a
>>> single source of truth for the deployment. So if inspector is changing
>>> the
>>> ipmi password, it should not only notify/update Ironic's knowledge on
>>> that
>>> node, but also notify/update the CMDB on that change - at least there
>>> must
>>> be a possibility (a ready-to-use plug point) to do that before we roll
>>> out
>>> such feature.
>>>
>>
> Well, if we have a CMDB, we probably don't need to set credentials. Or at
> least we should rely on the CMDB as a primary source. This "setting random
> password" thing is more about people without CMDB (aka using ironic as a
> CMDB ;). I'm not sure it's a compelling enough use case.
>
> Anyway, it could be interesting to talk about some generic OpenStack-CMDB
> interface, which might something proposed below.
>
>
>> wrt interaction with CMDB, we have investigating around some ideas tha
>> we have gathered at https://github.com/uggla/alexandria/wiki
>>
>
> Oh, that's interesting. I see some potential overlap with ironic and
> ironic-inspector. Would be cool to chat on it the next summit.
>
>
>> Some code has been written to try to model some of these aspects, but
>> having more contributors and patches to enhance that integration would
>> be great ! Similarly available at https://github.com/uggla/alexandria
>>
>> We had planned to talk about these ideas at the previous OpenStack
>> summit but didn't get enough votes it seems. So now aiming at preenting
>> to the next one ;-)
>>
>
> +100, would love to hear.
>
>
>> HTH,
>> Bruno.
>>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ironic] [inspector] Auto discovery extension for Ironic Inspector

[Sam Betts (sambetts)]

What Yuiko has described makes a lot of sense, and from that perspective 
perhaps instead of us defining what driver a node should and shouldn't be using 
a config file, we should just provide a guide to using the inspector rules for 
this and maybe some prewritten rules that can set the driver and driver info 
etc fields for different cases?

Then the work flow would be, default to Fake driver because we don't need any 
special info for that, however if a rule detects that its IPMIable by making 
sure that the IPMI address is valid or something, then it can set the driver to 
an ipmitool one and then set the password and username based on either a 
retrieved field or values defined in the rule itself.

WDYT?

Sam
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ironic] [inspector] Auto discovery extension for Ironic Inspector

[Dmitry Tantsur]

On 11/19/2015 11:57 AM, Sam Betts (sambetts) wrote:

What Yuiko has described makes a lot of sense, and from that perspective
perhaps instead of us defining what driver a node should and shouldn’t
be using a config file, we should just provide a guide to using the
inspector rules for this and maybe some prewritten rules that can set
the driver and driver info etc fields for different cases?

Then the work flow would be, default to Fake driver because we don’t
need any special info for that, however if a rule detects that its
IPMIable by making sure that the IPMI address is valid or something,
then it can set the driver to an ipmitool one and then set the password
and username based on either a retrieved field or values defined in the
rule itself.


Using introspection rules, wow! That's a really nice idea, I would why I 
didn't think about it.




WDYT?


That sounds really promising, and simplifies our life a lot. I would 
love to see all these ideas written. Do you folks think it's time for 
ironic-inspector-specs repo? :D




Sam


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ironic] [inspector] Auto discovery extension for Ironic Inspector

[Dmitry Tantsur]

I have to admin I forgot about this thread. Please find comments inline.

On 11/06/2015 05:25 PM, Bruno Cornec wrote:

Hello,

Pavlo Shchelokovskyy said on Tue, Nov 03, 2015 at 09:41:51PM +:

For auto-setting driver options on enrollment, I would vote for option 2
with default being fake driver + optional CMDB integration. This would
ease
managing a homogeneous pool of BMs, but still (using fake driver or data
from CMDB) work reasonably well in heterogeneous case.


Using fake driver means we need a manual step to set it to something 
non-fake :) and the current introspection process already has 1 manual 
step (enrolling nodes), so I'd like autodiscovery to require 0 of them 
(at least for the majority of users).




As for setting a random password, CMDB integration is crucial IMO. Large
deployments usually have some sort of it already, and it must serve as a
single source of truth for the deployment. So if inspector is changing
the
ipmi password, it should not only notify/update Ironic's knowledge on
that
node, but also notify/update the CMDB on that change - at least there
must
be a possibility (a ready-to-use plug point) to do that before we roll
out
such feature.


Well, if we have a CMDB, we probably don't need to set credentials. Or 
at least we should rely on the CMDB as a primary source. This "setting 
random password" thing is more about people without CMDB (aka using 
ironic as a CMDB ;). I'm not sure it's a compelling enough use case.


Anyway, it could be interesting to talk about some generic 
OpenStack-CMDB interface, which might something proposed below.




wrt interaction with CMDB, we have investigating around some ideas tha
we have gathered at https://github.com/uggla/alexandria/wiki


Oh, that's interesting. I see some potential overlap with ironic and 
ironic-inspector. Would be cool to chat on it the next summit.




Some code has been written to try to model some of these aspects, but
having more contributors and patches to enhance that integration would
be great ! Similarly available at https://github.com/uggla/alexandria

We had planned to talk about these ideas at the previous OpenStack
summit but didn't get enough votes it seems. So now aiming at preenting
to the next one ;-)


+100, would love to hear.



HTH,
Bruno.



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ironic] [inspector] Auto discovery extension for Ironic Inspector

[Dmitry Tantsur]

On 11/02/2015 05:07 PM, Sam Betts (sambetts) wrote:

Auto discovery is a topic which has been discussed a few times in the
past for

Ironic, and its interesting to solve because its a bit of a chicken and egg

problem. The ironic inspector allows us to inspect nodes that we don't know

the mac addresses for yet, to do this we run a global DHCP PXE rule that
will

respond to all mac addresses and PXE boot any machine that requests it,

this means its possible for machines that we haven't been asked to

inspect to boot into the inspector ramdisk and send their information to

inspector's API. To prevent this data from being processed further by

inspector if its a machine we shouldn't care about, we do a node lookup.
If the data

fails this node lookup we used to drop this data and continue no further, in

release 2.0.0 we added a hook point to intercept this state called the
Node Not

Found hook point which allows us to run some python code at this point in

processing before failing and dropping the inspection data. Something we've

discussed as a use for this hook point is, enrolling a node that fails the

lookup into Ironic, and then having inspector continue to process the

inspection data as we would for any other node that had inspection requested

for it, this allows us to auto-discover unknown nodes into Ironic.


If this auto discovery hook was enabled this would be the flow when
inspector

receives inspection data from the inspector ramdisk:


- Run pre-process on the inspection data to sanitise the data and ready
it for

   the rest of the process


- Node lookup using fields from the inspection data:

   - If in inspector node cache return node info


   - If not in inspector node cache and but is in ironic node database, fail

 inspection because its a known node and inspection hasn't been
requested

 for it.


   - If not in inspector node cache or ironic node database, enroll the
node in

 ironic and return node info


- Process inspection data


The remaining question for this idea is how to handle the driver
settings for

each node that we discover, we've currently discussed 3 different options:


1. Enroll the node in ironic using the fake driver, and leave it to the
operator

to set the driver type and driver info before they move the node
from enroll

to manageable.


I'm -1 to this because it requires a manual step. We already have a 
process requiring 1 manual step - inspection :) I'd like autodiscovery 
to turn it to 0.





2. Allow for the default driver and driver info information to be set in
the

ironic inspector configuration file, this will be set on every node
that is

auto discovered. Possible config file example:


[autodiscovery]

driver = pxe_ipmitool

address_field = 

username_field = 

password_field = 


This is my favorite one. We'll also need to provide the default user 
name/password. We can try to advance a node to MANAGEABLE state after 
enrolling it. If the default credentials don't work, node would stay in 
ENROLL state, and this will be a signal to an operator to check them.





3. A possibly vendor specific option that was suggested at the summit was to

provide an ability to look up out of band credentials from an
external CMDB.


We already have an extension point for discovery. If we know more about 
CMDB interfaces, we can extend it, but it's already possible to use.





The first option is technically possible using the second option, by setting

the driver to fake and leaving the driver info blank.


+1




With IPMI based drivers most IPMI related information can be retrieved
from the

node by the inspector ramdisk, however for non-ipmi based drivers such
as the

cimc/ucs drivers this information isn't accessible from an in-band OS
command.


A problem with option 2 is that it can not account for a mixed driver

environment.


We have also discussed for IPMI based drivers inspector could set a new

randomly generated password on to the freshly discovered node, with the idea

being fresh hardware often comes with a default password, and if you used

inspector to discover it then it could set a unique password on it and

automatically make ironic aware of that.


We're throwing this idea out onto the mailer because we'd like to get
feedback

from the community to see if this would be useful for people using
inspector,

and to see if people have any opinions on what the right way to handle
the node

driver settings is.


Yeah, I'm not decided on this one. Sounds cool but dangerous :)




Sam (sambetts)




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage 

Re: [openstack-dev] [ironic] [inspector] Auto discovery extension for Ironic Inspector

[Bruno Cornec]

Hello,

Pavlo Shchelokovskyy said on Tue, Nov 03, 2015 at 09:41:51PM +:

For auto-setting driver options on enrollment, I would vote for option 2
with default being fake driver + optional CMDB integration. This would ease
managing a homogeneous pool of BMs, but still (using fake driver or data
from CMDB) work reasonably well in heterogeneous case.

As for setting a random password, CMDB integration is crucial IMO. Large
deployments usually have some sort of it already, and it must serve as a
single source of truth for the deployment. So if inspector is changing the
ipmi password, it should not only notify/update Ironic's knowledge on that
node, but also notify/update the CMDB on that change - at least there must
be a possibility (a ready-to-use plug point) to do that before we roll out
such feature.


wrt interaction with CMDB, we have investigating around some ideas tha
we have gathered at https://github.com/uggla/alexandria/wiki

Some code has been written to try to model some of these aspects, but
having more contributors and patches to enhance that integration would
be great ! Similarly available at https://github.com/uggla/alexandria

We had planned to talk about these ideas at the previous OpenStack
summit but didn't get enough votes it seems. So now aiming at preenting
to the next one ;-)

HTH,
Bruno.
--
Open Source Profession, Linux Community Lead WW  http://hpintelco.net
HPE EMEA EG Open Source Technology Strategist http://hp.com/go/opensource
FLOSS projects: http://mondorescue.org http://project-builder.org
Musique ancienne? http://www.musique-ancienne.org http://www.medieval.org

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [ironic] [inspector] Auto discovery extension for Ironic Inspector

[Pavlo Shchelokovskyy]

Hi all,

For auto-setting driver options on enrollment, I would vote for option 2
with default being fake driver + optional CMDB integration. This would ease
managing a homogeneous pool of BMs, but still (using fake driver or data
from CMDB) work reasonably well in heterogeneous case.

As for setting a random password, CMDB integration is crucial IMO. Large
deployments usually have some sort of it already, and it must serve as a
single source of truth for the deployment. So if inspector is changing the
ipmi password, it should not only notify/update Ironic's knowledge on that
node, but also notify/update the CMDB on that change - at least there must
be a possibility (a ready-to-use plug point) to do that before we roll out
such feature.

Best regards,
-- 
Dr. Pavlo Shchelokovskyy
Senior Software Engineer
Mirantis Inc
www.mirantis.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [ironic] [inspector] Auto discovery extension for Ironic Inspector

[Sam Betts (sambetts)]

Auto discovery is a topic which has been discussed a few times in the past for

Ironic, and its interesting to solve because its a bit of a chicken and egg

problem. The ironic inspector allows us to inspect nodes that we don't know

the mac addresses for yet, to do this we run a global DHCP PXE rule that will

respond to all mac addresses and PXE boot any machine that requests it,

this means its possible for machines that we haven't been asked to

inspect to boot into the inspector ramdisk and send their information to

inspector's API. To prevent this data from being processed further by

inspector if its a machine we shouldn't care about, we do a node lookup. If the 
data

fails this node lookup we used to drop this data and continue no further, in

release 2.0.0 we added a hook point to intercept this state called the Node Not

Found hook point which allows us to run some python code at this point in

processing before failing and dropping the inspection data. Something we've

discussed as a use for this hook point is, enrolling a node that fails the

lookup into Ironic, and then having inspector continue to process the

inspection data as we would for any other node that had inspection requested

for it, this allows us to auto-discover unknown nodes into Ironic.


If this auto discovery hook was enabled this would be the flow when inspector

receives inspection data from the inspector ramdisk:


- Run pre-process on the inspection data to sanitise the data and ready it for

  the rest of the process


- Node lookup using fields from the inspection data:

  - If in inspector node cache return node info


  - If not in inspector node cache and but is in ironic node database, fail

inspection because its a known node and inspection hasn't been requested

for it.


  - If not in inspector node cache or ironic node database, enroll the node in

ironic and return node info


- Process inspection data


The remaining question for this idea is how to handle the driver settings for

each node that we discover, we've currently discussed 3 different options:


1. Enroll the node in ironic using the fake driver, and leave it to the operator

   to set the driver type and driver info before they move the node from enroll

   to manageable.


2. Allow for the default driver and driver info information to be set in the

   ironic inspector configuration file, this will be set on every node that is

   auto discovered. Possible config file example:


   [autodiscovery]

   driver = pxe_ipmitool

   address_field = 

   username_field = 

   password_field = 


3. A possibly vendor specific option that was suggested at the summit was to

   provide an ability to look up out of band credentials from an external CMDB.


The first option is technically possible using the second option, by setting

the driver to fake and leaving the driver info blank.


With IPMI based drivers most IPMI related information can be retrieved from the

node by the inspector ramdisk, however for non-ipmi based drivers such as the

cimc/ucs drivers this information isn't accessible from an in-band OS command.


A problem with option 2 is that it can not account for a mixed driver

environment.


We have also discussed for IPMI based drivers inspector could set a new

randomly generated password on to the freshly discovered node, with the idea

being fresh hardware often comes with a default password, and if you used

inspector to discover it then it could set a unique password on it and

automatically make ironic aware of that.


We're throwing this idea out onto the mailer because we'd like to get feedback

from the community to see if this would be useful for people using inspector,

and to see if people have any opinions on what the right way to handle the node

driver settings is.


Sam (sambetts)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev