Re: [openstack-dev] [Ironic] ironic-discoverd status update

2015-01-09 Thread Dmitry Tantsur

On 01/09/2015 08:43 AM, Jerry Xinyu Zhao wrote:

tuskar-ui is supposed to enroll nodes into ironic.

Right. And it has support for discoverd IIRC.



On Thu, Jan 8, 2015 at 4:36 AM, Zhou, Zhenzan mailto:zhenzan.z...@intel.com>> wrote:

Sounds like we could add something new to automate the enrollment of
new nodes:-)
Collecting IPMI info into a csv file is still a trivial job...

BR
Zhou Zhenzan

-Original Message-
From: Dmitry Tantsur [mailto:dtant...@redhat.com
<mailto:dtant...@redhat.com>]
Sent: Thursday, January 8, 2015 5:19 PM
To: openstack-dev@lists.openstack.org
<mailto:openstack-dev@lists.openstack.org>
Subject: Re: [openstack-dev] [Ironic] ironic-discoverd status update

On 01/08/2015 06:48 AM, Kumar, Om (Cloud OS R&D) wrote:
 > My understanding of discovery was to get all details for a node
and then register that node to ironic. i.e. Enrollment of the node
to ironic. Pardon me if it was out of line with your understanding
of discovery.
That's why we agreed to use terms inspection/introspection :) sorry
for not being consistent here (name 'discoverd' is pretty old and
hard to change).

discoverd does not enroll nodes. while possible, I'm somewhat
resistant to make it do enrolling, mostly because I want it to be
user-controlled process.

 >
 > What I understand from the below mentioned spec is that the Node
is registered, but the spec will help ironic discover other
properties of the node.
that's what discoverd does currently.

 >
 > -Om
 >
 > -Original Message-
 > From: Dmitry Tantsur [mailto:dtant...@redhat.com
<mailto:dtant...@redhat.com>]
 > Sent: 07 January 2015 20:20
 > To: openstack-dev@lists.openstack.org
    <mailto:openstack-dev@lists.openstack.org>
 > Subject: Re: [openstack-dev] [Ironic] ironic-discoverd status update
 >
 > On 01/07/2015 03:44 PM, Matt Keenan wrote:
 >> On 01/07/15 14:24, Kumar, Om (Cloud OS R&D) wrote:
 >>> If it's a separate project, can it be extended to perform out
of band
 >>> discovery too..? That way there will be a single service to perform
 >>> in-band as well as out of band discoveries.. May be it could follow
 >>> driver framework for discovering nodes, where one driver could be
 >>> native (in-band) and other could be iLO specific etc...
 >>>
 >>
 >> I believe the following spec outlines plans for out-of-band
discovery:
 >> https://review.openstack.org/#/c/100951/
 > Right, so Ironic will have drivers, one of which (I hope) will be
a driver for discoverd.
 >
 >>
 >> No idea what the progress is with regard to implementation
within the
 >> Kilo cycle though.
 > For now we hope to get it merged in K.
 >
 >>
 >> cheers
 >>
 >> Matt
 >>
 >>> Just a thought.
 >>>
 >>> -Om
 >>>
 >>> -Original Message-----
     >>> From: Dmitry Tantsur [mailto:dtant...@redhat.com
<mailto:dtant...@redhat.com>]
 >>> Sent: 07 January 2015 14:34
 >>> To: openstack-dev@lists.openstack.org
<mailto:openstack-dev@lists.openstack.org>
 >>> Subject: Re: [openstack-dev] [Ironic] ironic-discoverd status
update
 >>>
 >>> On 01/07/2015 09:58 AM, Zhou, Zhenzan wrote:
 >>>> So is it possible to just integrate this project into ironic?
I mean
 >>>> when you create an ironic node, it will start discover in the
 >>>> background. So we don't need two services?
 >>> Well, the decision on the summit was that it's better to keep it
 >>> separate. Please see https://review.openstack.org/#/c/135605/ for
 >>> details on future interaction between discoverd and Ironic.
 >>>
 >>>> Just a thought, thanks.
 >>>>
 >>>> BR
 >>>> Zhou Zhenzan
 >>>>
 >>>> -Original Message-
 >>>> From: Dmitry Tantsur [mailto:dtant...@redhat.com
<mailto:dtant...@redhat.com>]
 >>>> Sent: Monday, January 5, 2015 4:49 PM
 >>>> To: openstack-dev@lists.openstack.org
<mailto:openstack-dev@lists.openstack.org>
 >>>> Subject: Re: [openstack-dev] [Ironic] ironic-discoverd status
update
 >>>>
 >>>> On 01/05/2015 09:31 AM, Zhou, Zhenzan wrote:
 >>>>> Hi, Dmitry
 >>>>>
 >>>>

Re: [openstack-dev] [Ironic] ironic-discoverd status update

2015-01-08 Thread Jerry Xinyu Zhao
tuskar-ui is supposed to enroll nodes into ironic.

On Thu, Jan 8, 2015 at 4:36 AM, Zhou, Zhenzan 
wrote:

> Sounds like we could add something new to automate the enrollment of new
> nodes:-)
> Collecting IPMI info into a csv file is still a trivial job...
>
> BR
> Zhou Zhenzan
>
> -Original Message-
> From: Dmitry Tantsur [mailto:dtant...@redhat.com]
> Sent: Thursday, January 8, 2015 5:19 PM
> To: openstack-dev@lists.openstack.org
> Subject: Re: [openstack-dev] [Ironic] ironic-discoverd status update
>
> On 01/08/2015 06:48 AM, Kumar, Om (Cloud OS R&D) wrote:
> > My understanding of discovery was to get all details for a node and then
> register that node to ironic. i.e. Enrollment of the node to ironic. Pardon
> me if it was out of line with your understanding of discovery.
> That's why we agreed to use terms inspection/introspection :) sorry for
> not being consistent here (name 'discoverd' is pretty old and hard to
> change).
>
> discoverd does not enroll nodes. while possible, I'm somewhat resistant to
> make it do enrolling, mostly because I want it to be user-controlled
> process.
>
> >
> > What I understand from the below mentioned spec is that the Node is
> registered, but the spec will help ironic discover other properties of the
> node.
> that's what discoverd does currently.
>
> >
> > -Om
> >
> > -Original Message-
> > From: Dmitry Tantsur [mailto:dtant...@redhat.com]
> > Sent: 07 January 2015 20:20
> > To: openstack-dev@lists.openstack.org
> > Subject: Re: [openstack-dev] [Ironic] ironic-discoverd status update
> >
> > On 01/07/2015 03:44 PM, Matt Keenan wrote:
> >> On 01/07/15 14:24, Kumar, Om (Cloud OS R&D) wrote:
> >>> If it's a separate project, can it be extended to perform out of band
> >>> discovery too..? That way there will be a single service to perform
> >>> in-band as well as out of band discoveries.. May be it could follow
> >>> driver framework for discovering nodes, where one driver could be
> >>> native (in-band) and other could be iLO specific etc...
> >>>
> >>
> >> I believe the following spec outlines plans for out-of-band discovery:
> >> https://review.openstack.org/#/c/100951/
> > Right, so Ironic will have drivers, one of which (I hope) will be a
> driver for discoverd.
> >
> >>
> >> No idea what the progress is with regard to implementation within the
> >> Kilo cycle though.
> > For now we hope to get it merged in K.
> >
> >>
> >> cheers
> >>
> >> Matt
> >>
> >>> Just a thought.
> >>>
> >>> -Om
> >>>
> >>> -Original Message-
> >>> From: Dmitry Tantsur [mailto:dtant...@redhat.com]
> >>> Sent: 07 January 2015 14:34
> >>> To: openstack-dev@lists.openstack.org
> >>> Subject: Re: [openstack-dev] [Ironic] ironic-discoverd status update
> >>>
> >>> On 01/07/2015 09:58 AM, Zhou, Zhenzan wrote:
> >>>> So is it possible to just integrate this project into ironic? I mean
> >>>> when you create an ironic node, it will start discover in the
> >>>> background. So we don't need two services?
> >>> Well, the decision on the summit was that it's better to keep it
> >>> separate. Please see https://review.openstack.org/#/c/135605/ for
> >>> details on future interaction between discoverd and Ironic.
> >>>
> >>>> Just a thought, thanks.
> >>>>
> >>>> BR
> >>>> Zhou Zhenzan
> >>>>
> >>>> -Original Message-
> >>>> From: Dmitry Tantsur [mailto:dtant...@redhat.com]
> >>>> Sent: Monday, January 5, 2015 4:49 PM
> >>>> To: openstack-dev@lists.openstack.org
> >>>> Subject: Re: [openstack-dev] [Ironic] ironic-discoverd status update
> >>>>
> >>>> On 01/05/2015 09:31 AM, Zhou, Zhenzan wrote:
> >>>>> Hi, Dmitry
> >>>>>
> >>>>> I think this is a good project.
> >>>>> I got one question: what is the relationship with
> ironic-python-agent?
> >>>>> Thanks.
> >>>> Hi!
> >>>>
> >>>> No relationship right now, but I'm hoping to use IPA as a base for
> >>>> introspection ramdisk in the (near?) future.
> >>>>>
> >>>>> BR
> >>>>> Zhou Zhenzan
> >

Re: [openstack-dev] [Ironic] ironic-discoverd status update

2015-01-08 Thread Zhou, Zhenzan
Sounds like we could add something new to automate the enrollment of new 
nodes:-)
Collecting IPMI info into a csv file is still a trivial job...

BR
Zhou Zhenzan

-Original Message-
From: Dmitry Tantsur [mailto:dtant...@redhat.com] 
Sent: Thursday, January 8, 2015 5:19 PM
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [Ironic] ironic-discoverd status update

On 01/08/2015 06:48 AM, Kumar, Om (Cloud OS R&D) wrote:
> My understanding of discovery was to get all details for a node and then 
> register that node to ironic. i.e. Enrollment of the node to ironic. Pardon 
> me if it was out of line with your understanding of discovery.
That's why we agreed to use terms inspection/introspection :) sorry for not 
being consistent here (name 'discoverd' is pretty old and hard to change).

discoverd does not enroll nodes. while possible, I'm somewhat resistant to make 
it do enrolling, mostly because I want it to be user-controlled process.

>
> What I understand from the below mentioned spec is that the Node is 
> registered, but the spec will help ironic discover other properties of the 
> node.
that's what discoverd does currently.

>
> -Om
>
> -Original Message-
> From: Dmitry Tantsur [mailto:dtant...@redhat.com]
> Sent: 07 January 2015 20:20
> To: openstack-dev@lists.openstack.org
> Subject: Re: [openstack-dev] [Ironic] ironic-discoverd status update
>
> On 01/07/2015 03:44 PM, Matt Keenan wrote:
>> On 01/07/15 14:24, Kumar, Om (Cloud OS R&D) wrote:
>>> If it's a separate project, can it be extended to perform out of band
>>> discovery too..? That way there will be a single service to perform
>>> in-band as well as out of band discoveries.. May be it could follow
>>> driver framework for discovering nodes, where one driver could be
>>> native (in-band) and other could be iLO specific etc...
>>>
>>
>> I believe the following spec outlines plans for out-of-band discovery:
>> https://review.openstack.org/#/c/100951/
> Right, so Ironic will have drivers, one of which (I hope) will be a driver 
> for discoverd.
>
>>
>> No idea what the progress is with regard to implementation within the
>> Kilo cycle though.
> For now we hope to get it merged in K.
>
>>
>> cheers
>>
>> Matt
>>
>>> Just a thought.
>>>
>>> -Om
>>>
>>> -Original Message-
>>> From: Dmitry Tantsur [mailto:dtant...@redhat.com]
>>> Sent: 07 January 2015 14:34
>>> To: openstack-dev@lists.openstack.org
>>> Subject: Re: [openstack-dev] [Ironic] ironic-discoverd status update
>>>
>>> On 01/07/2015 09:58 AM, Zhou, Zhenzan wrote:
>>>> So is it possible to just integrate this project into ironic? I mean
>>>> when you create an ironic node, it will start discover in the
>>>> background. So we don't need two services?
>>> Well, the decision on the summit was that it's better to keep it
>>> separate. Please see https://review.openstack.org/#/c/135605/ for
>>> details on future interaction between discoverd and Ironic.
>>>
>>>> Just a thought, thanks.
>>>>
>>>> BR
>>>> Zhou Zhenzan
>>>>
>>>> -Original Message-
>>>> From: Dmitry Tantsur [mailto:dtant...@redhat.com]
>>>> Sent: Monday, January 5, 2015 4:49 PM
>>>> To: openstack-dev@lists.openstack.org
>>>> Subject: Re: [openstack-dev] [Ironic] ironic-discoverd status update
>>>>
>>>> On 01/05/2015 09:31 AM, Zhou, Zhenzan wrote:
>>>>> Hi, Dmitry
>>>>>
>>>>> I think this is a good project.
>>>>> I got one question: what is the relationship with ironic-python-agent?
>>>>> Thanks.
>>>> Hi!
>>>>
>>>> No relationship right now, but I'm hoping to use IPA as a base for
>>>> introspection ramdisk in the (near?) future.
>>>>>
>>>>> BR
>>>>> Zhou Zhenzan
>>>>>
>>>>> -Original Message-
>>>>> From: Dmitry Tantsur [mailto:dtant...@redhat.com]
>>>>> Sent: Thursday, December 11, 2014 10:35 PM
>>>>> To: OpenStack Development Mailing List (not for usage questions)
>>>>> Subject: [openstack-dev] [Ironic] ironic-discoverd status update
>>>>>
>>>>> Hi all!
>>>>>
>>>>> As you know I actively promote ironic-discoverd project [1] as one
>>>>> of the means to do hardware inspection for Ironic (see

Re: [openstack-dev] [Ironic] ironic-discoverd status update

2015-01-08 Thread Dmitry Tantsur

On 01/08/2015 06:48 AM, Kumar, Om (Cloud OS R&D) wrote:

My understanding of discovery was to get all details for a node and then 
register that node to ironic. i.e. Enrollment of the node to ironic. Pardon me 
if it was out of line with your understanding of discovery.
That's why we agreed to use terms inspection/introspection :) sorry for 
not being consistent here (name 'discoverd' is pretty old and hard to 
change).


discoverd does not enroll nodes. while possible, I'm somewhat resistant 
to make it do enrolling, mostly because I want it to be user-controlled 
process.




What I understand from the below mentioned spec is that the Node is registered, 
but the spec will help ironic discover other properties of the node.

that's what discoverd does currently.



-Om

-Original Message-
From: Dmitry Tantsur [mailto:dtant...@redhat.com]
Sent: 07 January 2015 20:20
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [Ironic] ironic-discoverd status update

On 01/07/2015 03:44 PM, Matt Keenan wrote:

On 01/07/15 14:24, Kumar, Om (Cloud OS R&D) wrote:

If it's a separate project, can it be extended to perform out of band
discovery too..? That way there will be a single service to perform
in-band as well as out of band discoveries.. May be it could follow
driver framework for discovering nodes, where one driver could be
native (in-band) and other could be iLO specific etc...



I believe the following spec outlines plans for out-of-band discovery:
https://review.openstack.org/#/c/100951/

Right, so Ironic will have drivers, one of which (I hope) will be a driver for 
discoverd.



No idea what the progress is with regard to implementation within the
Kilo cycle though.

For now we hope to get it merged in K.



cheers

Matt


Just a thought.

-Om

-Original Message-
From: Dmitry Tantsur [mailto:dtant...@redhat.com]
Sent: 07 January 2015 14:34
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [Ironic] ironic-discoverd status update

On 01/07/2015 09:58 AM, Zhou, Zhenzan wrote:

So is it possible to just integrate this project into ironic? I mean
when you create an ironic node, it will start discover in the
background. So we don't need two services?

Well, the decision on the summit was that it's better to keep it
separate. Please see https://review.openstack.org/#/c/135605/ for
details on future interaction between discoverd and Ironic.


Just a thought, thanks.

BR
Zhou Zhenzan

-Original Message-
From: Dmitry Tantsur [mailto:dtant...@redhat.com]
Sent: Monday, January 5, 2015 4:49 PM
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [Ironic] ironic-discoverd status update

On 01/05/2015 09:31 AM, Zhou, Zhenzan wrote:

Hi, Dmitry

I think this is a good project.
I got one question: what is the relationship with ironic-python-agent?
Thanks.

Hi!

No relationship right now, but I'm hoping to use IPA as a base for
introspection ramdisk in the (near?) future.


BR
Zhou Zhenzan

-Original Message-
From: Dmitry Tantsur [mailto:dtant...@redhat.com]
Sent: Thursday, December 11, 2014 10:35 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [Ironic] ironic-discoverd status update

Hi all!

As you know I actively promote ironic-discoverd project [1] as one
of the means to do hardware inspection for Ironic (see e.g. spec
[2]), so I decided it's worth to give some updates to the community
from time to time. This email is purely informative, you may safely
skip it, if you're not interested.

Background
==

The discoverd project (I usually skip the "ironic-" part when
talking about it) solves the problem of populating information
about a node in Ironic database without help of any vendor-specific
tool. This information usually includes Nova scheduling properties
(CPU, RAM, disk
size) and MAC's for ports.

Introspection is done by booting a ramdisk on a node, collecting
data there and posting it back to discoverd HTTP API. Thus actually
discoverd consists of 2 components: the service [1] and the ramdisk
[3]. The service handles 2 major tasks:
* Processing data posted by the ramdisk, i.e. finding the node in
Ironic database and updating node properties with new data.
* Managing iptables so that the default PXE environment for
introspection does not interfere with Neutron

The project was born from a series of patches to Ironic itself
after we discovered that this change is going to be too intrusive.
Discoverd was actively tested as part of Instack [4] and it's RPM
is a part of Juno RDO. After the Paris summit, we agreed on
bringing it closer to the Ironic upstream, and now discoverd is
hosted on StackForge and tracks bugs on Launchpad.

Future
==

The basic feature of discoverd: supply Ironic with properties
required for scheduling, is pretty finished as of the latest stable
series 0.2.

However, more featu

Re: [openstack-dev] [Ironic] ironic-discoverd status update

2015-01-07 Thread
My understanding of discovery was to get all details for a node and then 
register that node to ironic. i.e. Enrollment of the node to ironic. Pardon me 
if it was out of line with your understanding of discovery.

What I understand from the below mentioned spec is that the Node is registered, 
but the spec will help ironic discover other properties of the node.

-Om

-Original Message-
From: Dmitry Tantsur [mailto:dtant...@redhat.com] 
Sent: 07 January 2015 20:20
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [Ironic] ironic-discoverd status update

On 01/07/2015 03:44 PM, Matt Keenan wrote:
> On 01/07/15 14:24, Kumar, Om (Cloud OS R&D) wrote:
>> If it's a separate project, can it be extended to perform out of band 
>> discovery too..? That way there will be a single service to perform 
>> in-band as well as out of band discoveries.. May be it could follow 
>> driver framework for discovering nodes, where one driver could be 
>> native (in-band) and other could be iLO specific etc...
>>
>
> I believe the following spec outlines plans for out-of-band discovery:
>https://review.openstack.org/#/c/100951/
Right, so Ironic will have drivers, one of which (I hope) will be a driver for 
discoverd.

>
> No idea what the progress is with regard to implementation within the 
> Kilo cycle though.
For now we hope to get it merged in K.

>
> cheers
>
> Matt
>
>> Just a thought.
>>
>> -Om
>>
>> -Original Message-
>> From: Dmitry Tantsur [mailto:dtant...@redhat.com]
>> Sent: 07 January 2015 14:34
>> To: openstack-dev@lists.openstack.org
>> Subject: Re: [openstack-dev] [Ironic] ironic-discoverd status update
>>
>> On 01/07/2015 09:58 AM, Zhou, Zhenzan wrote:
>>> So is it possible to just integrate this project into ironic? I mean 
>>> when you create an ironic node, it will start discover in the 
>>> background. So we don't need two services?
>> Well, the decision on the summit was that it's better to keep it 
>> separate. Please see https://review.openstack.org/#/c/135605/ for 
>> details on future interaction between discoverd and Ironic.
>>
>>> Just a thought, thanks.
>>>
>>> BR
>>> Zhou Zhenzan
>>>
>>> -Original Message-
>>> From: Dmitry Tantsur [mailto:dtant...@redhat.com]
>>> Sent: Monday, January 5, 2015 4:49 PM
>>> To: openstack-dev@lists.openstack.org
>>> Subject: Re: [openstack-dev] [Ironic] ironic-discoverd status update
>>>
>>> On 01/05/2015 09:31 AM, Zhou, Zhenzan wrote:
>>>> Hi, Dmitry
>>>>
>>>> I think this is a good project.
>>>> I got one question: what is the relationship with ironic-python-agent?
>>>> Thanks.
>>> Hi!
>>>
>>> No relationship right now, but I'm hoping to use IPA as a base for 
>>> introspection ramdisk in the (near?) future.
>>>>
>>>> BR
>>>> Zhou Zhenzan
>>>>
>>>> -Original Message-
>>>> From: Dmitry Tantsur [mailto:dtant...@redhat.com]
>>>> Sent: Thursday, December 11, 2014 10:35 PM
>>>> To: OpenStack Development Mailing List (not for usage questions)
>>>> Subject: [openstack-dev] [Ironic] ironic-discoverd status update
>>>>
>>>> Hi all!
>>>>
>>>> As you know I actively promote ironic-discoverd project [1] as one 
>>>> of the means to do hardware inspection for Ironic (see e.g. spec 
>>>> [2]), so I decided it's worth to give some updates to the community 
>>>> from time to time. This email is purely informative, you may safely 
>>>> skip it, if you're not interested.
>>>>
>>>> Background
>>>> ==
>>>>
>>>> The discoverd project (I usually skip the "ironic-" part when 
>>>> talking about it) solves the problem of populating information 
>>>> about a node in Ironic database without help of any vendor-specific 
>>>> tool. This information usually includes Nova scheduling properties 
>>>> (CPU, RAM, disk
>>>> size) and MAC's for ports.
>>>>
>>>> Introspection is done by booting a ramdisk on a node, collecting 
>>>> data there and posting it back to discoverd HTTP API. Thus actually 
>>>> discoverd consists of 2 components: the service [1] and the ramdisk 
>>>> [3]. The service handles 2 major tasks:
>>>> * Processing data posted by the ramdisk, i.e. finding the node in 
>>>> Ironic d

Re: [openstack-dev] [Ironic] ironic-discoverd status update

2015-01-07 Thread Dmitry Tantsur

On 01/07/2015 03:44 PM, Matt Keenan wrote:

On 01/07/15 14:24, Kumar, Om (Cloud OS R&D) wrote:

If it's a separate project, can it be extended to perform out of band
discovery too..? That way there will be a single service to perform
in-band as well as out of band discoveries.. May be it could follow
driver framework for discovering nodes, where one driver could be
native (in-band) and other could be iLO specific etc...



I believe the following spec outlines plans for out-of-band discovery:
   https://review.openstack.org/#/c/100951/
Right, so Ironic will have drivers, one of which (I hope) will be a 
driver for discoverd.




No idea what the progress is with regard to implementation within the
Kilo cycle though.

For now we hope to get it merged in K.



cheers

Matt


Just a thought.

-Om

-Original Message-
From: Dmitry Tantsur [mailto:dtant...@redhat.com]
Sent: 07 January 2015 14:34
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [Ironic] ironic-discoverd status update

On 01/07/2015 09:58 AM, Zhou, Zhenzan wrote:

So is it possible to just integrate this project into ironic? I mean
when you create an ironic node, it will start discover in the
background. So we don't need two services?

Well, the decision on the summit was that it's better to keep it
separate. Please see https://review.openstack.org/#/c/135605/ for
details on future interaction between discoverd and Ironic.


Just a thought, thanks.

BR
Zhou Zhenzan

-Original Message-
From: Dmitry Tantsur [mailto:dtant...@redhat.com]
Sent: Monday, January 5, 2015 4:49 PM
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [Ironic] ironic-discoverd status update

On 01/05/2015 09:31 AM, Zhou, Zhenzan wrote:

Hi, Dmitry

I think this is a good project.
I got one question: what is the relationship with ironic-python-agent?
Thanks.

Hi!

No relationship right now, but I'm hoping to use IPA as a base for
introspection ramdisk in the (near?) future.


BR
Zhou Zhenzan

-Original Message-
From: Dmitry Tantsur [mailto:dtant...@redhat.com]
Sent: Thursday, December 11, 2014 10:35 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [Ironic] ironic-discoverd status update

Hi all!

As you know I actively promote ironic-discoverd project [1] as one
of the means to do hardware inspection for Ironic (see e.g. spec
[2]), so I decided it's worth to give some updates to the community
from time to time. This email is purely informative, you may safely
skip it, if you're not interested.

Background
==

The discoverd project (I usually skip the "ironic-" part when talking
about it) solves the problem of populating information about a node
in Ironic database without help of any vendor-specific tool. This
information usually includes Nova scheduling properties (CPU, RAM,
disk
size) and MAC's for ports.

Introspection is done by booting a ramdisk on a node, collecting
data there and posting it back to discoverd HTTP API. Thus actually
discoverd consists of 2 components: the service [1] and the ramdisk
[3]. The service handles 2 major tasks:
* Processing data posted by the ramdisk, i.e. finding the node in
Ironic database and updating node properties with new data.
* Managing iptables so that the default PXE environment for
introspection does not interfere with Neutron

The project was born from a series of patches to Ironic itself after
we discovered that this change is going to be too intrusive.
Discoverd was actively tested as part of Instack [4] and it's RPM is
a part of Juno RDO. After the Paris summit, we agreed on bringing it
closer to the Ironic upstream, and now discoverd is hosted on
StackForge and tracks bugs on Launchpad.

Future
==

The basic feature of discoverd: supply Ironic with properties
required for scheduling, is pretty finished as of the latest stable
series 0.2.

However, more features are planned for release 1.0.0 this January [5].
They go beyond the bare minimum of finding out CPU, RAM, disk size
and NIC MAC's.

Plugability
~~~

An interesting feature of discoverd is support for plugins, which I
prefer to call hooks. It's possible to hook into the introspection
data processing chain in 2 places:
* Before any data processing. This opens opportunity to adopt
discoverd to ramdisks that have different data format. The only
requirement is that the ramdisk posts a JSON object.
* After a node is found in Ironic database and ports are created for
MAC's, but before any actual data update. This gives an opportunity
to alter, which properties discoverd is going to update.

Actually, even the default logic of update Node.properties is
contained in a plugin - see SchedulerHook in
ironic_discoverd/plugins/standard.py
[6]. This plugability opens wide opportunities for integrating with
3rd party ramdisks and CMDB's (which as we know Ironic is not ;).

Enrolling
~

Re: [openstack-dev] [Ironic] ironic-discoverd status update

2015-01-07 Thread Matt Keenan

On 01/07/15 14:24, Kumar, Om (Cloud OS R&D) wrote:

If it's a separate project, can it be extended to perform out of band discovery 
too..? That way there will be a single service to perform in-band as well as 
out of band discoveries.. May be it could follow driver framework for 
discovering nodes, where one driver could be native (in-band) and other could 
be iLO specific etc...



I believe the following spec outlines plans for out-of-band discovery:
  https://review.openstack.org/#/c/100951/

No idea what the progress is with regard to implementation within the 
Kilo cycle though.


cheers

Matt


Just a thought.

-Om

-Original Message-
From: Dmitry Tantsur [mailto:dtant...@redhat.com]
Sent: 07 January 2015 14:34
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [Ironic] ironic-discoverd status update

On 01/07/2015 09:58 AM, Zhou, Zhenzan wrote:

So is it possible to just integrate this project into ironic? I mean when you 
create an ironic node, it will start discover in the background. So we don't 
need two services?

Well, the decision on the summit was that it's better to keep it separate. 
Please see https://review.openstack.org/#/c/135605/ for details on future 
interaction between discoverd and Ironic.


Just a thought, thanks.

BR
Zhou Zhenzan

-Original Message-
From: Dmitry Tantsur [mailto:dtant...@redhat.com]
Sent: Monday, January 5, 2015 4:49 PM
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [Ironic] ironic-discoverd status update

On 01/05/2015 09:31 AM, Zhou, Zhenzan wrote:

Hi, Dmitry

I think this is a good project.
I got one question: what is the relationship with ironic-python-agent?
Thanks.

Hi!

No relationship right now, but I'm hoping to use IPA as a base for 
introspection ramdisk in the (near?) future.


BR
Zhou Zhenzan

-Original Message-
From: Dmitry Tantsur [mailto:dtant...@redhat.com]
Sent: Thursday, December 11, 2014 10:35 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [Ironic] ironic-discoverd status update

Hi all!

As you know I actively promote ironic-discoverd project [1] as one of the means 
to do hardware inspection for Ironic (see e.g. spec [2]), so I decided it's 
worth to give some updates to the community from time to time. This email is 
purely informative, you may safely skip it, if you're not interested.

Background
==

The discoverd project (I usually skip the "ironic-" part when talking
about it) solves the problem of populating information about a node
in Ironic database without help of any vendor-specific tool. This
information usually includes Nova scheduling properties (CPU, RAM,
disk
size) and MAC's for ports.

Introspection is done by booting a ramdisk on a node, collecting data there and 
posting it back to discoverd HTTP API. Thus actually discoverd consists of 2 
components: the service [1] and the ramdisk [3]. The service handles 2 major 
tasks:
* Processing data posted by the ramdisk, i.e. finding the node in Ironic 
database and updating node properties with new data.
* Managing iptables so that the default PXE environment for
introspection does not interfere with Neutron

The project was born from a series of patches to Ironic itself after we 
discovered that this change is going to be too intrusive. Discoverd was 
actively tested as part of Instack [4] and it's RPM is a part of Juno RDO. 
After the Paris summit, we agreed on bringing it closer to the Ironic upstream, 
and now discoverd is hosted on StackForge and tracks bugs on Launchpad.

Future
==

The basic feature of discoverd: supply Ironic with properties required for 
scheduling, is pretty finished as of the latest stable series 0.2.

However, more features are planned for release 1.0.0 this January [5].
They go beyond the bare minimum of finding out CPU, RAM, disk size and NIC 
MAC's.

Plugability
~~~

An interesting feature of discoverd is support for plugins, which I prefer to 
call hooks. It's possible to hook into the introspection data processing chain 
in 2 places:
* Before any data processing. This opens opportunity to adopt discoverd to 
ramdisks that have different data format. The only requirement is that the 
ramdisk posts a JSON object.
* After a node is found in Ironic database and ports are created for MAC's, but 
before any actual data update. This gives an opportunity to alter, which 
properties discoverd is going to update.

Actually, even the default logic of update Node.properties is
contained in a plugin - see SchedulerHook in
ironic_discoverd/plugins/standard.py
[6]. This plugability opens wide opportunities for integrating with 3rd party 
ramdisks and CMDB's (which as we know Ironic is not ;).

Enrolling
~

Some people have found it limiting that the introspection requires power 
credentials (IPMI user name and password) to be already set. The recent set 

Re: [openstack-dev] [Ironic] ironic-discoverd status update

2015-01-07 Thread
If it's a separate project, can it be extended to perform out of band discovery 
too..? That way there will be a single service to perform in-band as well as 
out of band discoveries.. May be it could follow driver framework for 
discovering nodes, where one driver could be native (in-band) and other could 
be iLO specific etc... 

Just a thought.

-Om

-Original Message-
From: Dmitry Tantsur [mailto:dtant...@redhat.com] 
Sent: 07 January 2015 14:34
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [Ironic] ironic-discoverd status update

On 01/07/2015 09:58 AM, Zhou, Zhenzan wrote:
> So is it possible to just integrate this project into ironic? I mean when you 
> create an ironic node, it will start discover in the background. So we don't 
> need two services?
Well, the decision on the summit was that it's better to keep it separate. 
Please see https://review.openstack.org/#/c/135605/ for details on future 
interaction between discoverd and Ironic.

> Just a thought, thanks.
>
> BR
> Zhou Zhenzan
>
> -Original Message-
> From: Dmitry Tantsur [mailto:dtant...@redhat.com]
> Sent: Monday, January 5, 2015 4:49 PM
> To: openstack-dev@lists.openstack.org
> Subject: Re: [openstack-dev] [Ironic] ironic-discoverd status update
>
> On 01/05/2015 09:31 AM, Zhou, Zhenzan wrote:
>> Hi, Dmitry
>>
>> I think this is a good project.
>> I got one question: what is the relationship with ironic-python-agent?
>> Thanks.
> Hi!
>
> No relationship right now, but I'm hoping to use IPA as a base for 
> introspection ramdisk in the (near?) future.
>>
>> BR
>> Zhou Zhenzan
>>
>> -Original Message-
>> From: Dmitry Tantsur [mailto:dtant...@redhat.com]
>> Sent: Thursday, December 11, 2014 10:35 PM
>> To: OpenStack Development Mailing List (not for usage questions)
>> Subject: [openstack-dev] [Ironic] ironic-discoverd status update
>>
>> Hi all!
>>
>> As you know I actively promote ironic-discoverd project [1] as one of the 
>> means to do hardware inspection for Ironic (see e.g. spec [2]), so I decided 
>> it's worth to give some updates to the community from time to time. This 
>> email is purely informative, you may safely skip it, if you're not 
>> interested.
>>
>> Background
>> ==
>>
>> The discoverd project (I usually skip the "ironic-" part when talking 
>> about it) solves the problem of populating information about a node 
>> in Ironic database without help of any vendor-specific tool. This 
>> information usually includes Nova scheduling properties (CPU, RAM, 
>> disk
>> size) and MAC's for ports.
>>
>> Introspection is done by booting a ramdisk on a node, collecting data there 
>> and posting it back to discoverd HTTP API. Thus actually discoverd consists 
>> of 2 components: the service [1] and the ramdisk [3]. The service handles 2 
>> major tasks:
>> * Processing data posted by the ramdisk, i.e. finding the node in Ironic 
>> database and updating node properties with new data.
>> * Managing iptables so that the default PXE environment for 
>> introspection does not interfere with Neutron
>>
>> The project was born from a series of patches to Ironic itself after we 
>> discovered that this change is going to be too intrusive. Discoverd was 
>> actively tested as part of Instack [4] and it's RPM is a part of Juno RDO. 
>> After the Paris summit, we agreed on bringing it closer to the Ironic 
>> upstream, and now discoverd is hosted on StackForge and tracks bugs on 
>> Launchpad.
>>
>> Future
>> ==
>>
>> The basic feature of discoverd: supply Ironic with properties required for 
>> scheduling, is pretty finished as of the latest stable series 0.2.
>>
>> However, more features are planned for release 1.0.0 this January [5].
>> They go beyond the bare minimum of finding out CPU, RAM, disk size and NIC 
>> MAC's.
>>
>> Plugability
>> ~~~
>>
>> An interesting feature of discoverd is support for plugins, which I prefer 
>> to call hooks. It's possible to hook into the introspection data processing 
>> chain in 2 places:
>> * Before any data processing. This opens opportunity to adopt discoverd to 
>> ramdisks that have different data format. The only requirement is that the 
>> ramdisk posts a JSON object.
>> * After a node is found in Ironic database and ports are created for MAC's, 
>> but before any actual data update. This gives an opportunity to alter, which 
>> properties discoverd is going to update.
>>
>>

Re: [openstack-dev] [Ironic] ironic-discoverd status update

2015-01-07 Thread Dmitry Tantsur

On 01/07/2015 09:58 AM, Zhou, Zhenzan wrote:

So is it possible to just integrate this project into ironic? I mean when you 
create an ironic node, it will start discover in the background. So we don't 
need two services?
Well, the decision on the summit was that it's better to keep it 
separate. Please see https://review.openstack.org/#/c/135605/ for 
details on future interaction between discoverd and Ironic.



Just a thought, thanks.

BR
Zhou Zhenzan

-Original Message-
From: Dmitry Tantsur [mailto:dtant...@redhat.com]
Sent: Monday, January 5, 2015 4:49 PM
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [Ironic] ironic-discoverd status update

On 01/05/2015 09:31 AM, Zhou, Zhenzan wrote:

Hi, Dmitry

I think this is a good project.
I got one question: what is the relationship with ironic-python-agent?
Thanks.

Hi!

No relationship right now, but I'm hoping to use IPA as a base for 
introspection ramdisk in the (near?) future.


BR
Zhou Zhenzan

-Original Message-
From: Dmitry Tantsur [mailto:dtant...@redhat.com]
Sent: Thursday, December 11, 2014 10:35 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [Ironic] ironic-discoverd status update

Hi all!

As you know I actively promote ironic-discoverd project [1] as one of the means 
to do hardware inspection for Ironic (see e.g. spec [2]), so I decided it's 
worth to give some updates to the community from time to time. This email is 
purely informative, you may safely skip it, if you're not interested.

Background
==

The discoverd project (I usually skip the "ironic-" part when talking
about it) solves the problem of populating information about a node in
Ironic database without help of any vendor-specific tool. This
information usually includes Nova scheduling properties (CPU, RAM,
disk
size) and MAC's for ports.

Introspection is done by booting a ramdisk on a node, collecting data there and 
posting it back to discoverd HTTP API. Thus actually discoverd consists of 2 
components: the service [1] and the ramdisk [3]. The service handles 2 major 
tasks:
* Processing data posted by the ramdisk, i.e. finding the node in Ironic 
database and updating node properties with new data.
* Managing iptables so that the default PXE environment for
introspection does not interfere with Neutron

The project was born from a series of patches to Ironic itself after we 
discovered that this change is going to be too intrusive. Discoverd was 
actively tested as part of Instack [4] and it's RPM is a part of Juno RDO. 
After the Paris summit, we agreed on bringing it closer to the Ironic upstream, 
and now discoverd is hosted on StackForge and tracks bugs on Launchpad.

Future
==

The basic feature of discoverd: supply Ironic with properties required for 
scheduling, is pretty finished as of the latest stable series 0.2.

However, more features are planned for release 1.0.0 this January [5].
They go beyond the bare minimum of finding out CPU, RAM, disk size and NIC 
MAC's.

Plugability
~~~

An interesting feature of discoverd is support for plugins, which I prefer to 
call hooks. It's possible to hook into the introspection data processing chain 
in 2 places:
* Before any data processing. This opens opportunity to adopt discoverd to 
ramdisks that have different data format. The only requirement is that the 
ramdisk posts a JSON object.
* After a node is found in Ironic database and ports are created for MAC's, but 
before any actual data update. This gives an opportunity to alter, which 
properties discoverd is going to update.

Actually, even the default logic of update Node.properties is
contained in a plugin - see SchedulerHook in
ironic_discoverd/plugins/standard.py
[6]. This plugability opens wide opportunities for integrating with 3rd party 
ramdisks and CMDB's (which as we know Ironic is not ;).

Enrolling
~

Some people have found it limiting that the introspection requires power 
credentials (IPMI user name and password) to be already set. The recent set of 
patches [7] introduces a possibility to request manual power on of the machine 
and update IPMI credentials via the ramdisk to the expected values. Note that 
support of this feature in the reference ramdisk [3] is not ready yet. Also 
note that this scenario is only possible when using discoverd directly via it's 
API, not via Ironic API like in [2].

Get Involved


Discoverd terribly lacks reviews. Out team is very small and
self-approving is not a rare case. I'm even not against fast-tracking
any existing Ironic core to a discoverd core after a couple of
meaningful reviews :)

And of course patches are welcome, especially plugins for integration with 
existing systems doing similar things and CMDB's. Patches are accepted via 
usual Gerrit workflow. Ideas are accepted as Launchpad blueprints (we do not 
follow

Re: [openstack-dev] [Ironic] ironic-discoverd status update

2015-01-07 Thread Zhou, Zhenzan
So is it possible to just integrate this project into ironic? I mean when you 
create an ironic node, it will start discover in the background. So we don't 
need two services? 
Just a thought, thanks.

BR
Zhou Zhenzan

-Original Message-
From: Dmitry Tantsur [mailto:dtant...@redhat.com] 
Sent: Monday, January 5, 2015 4:49 PM
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [Ironic] ironic-discoverd status update

On 01/05/2015 09:31 AM, Zhou, Zhenzan wrote:
> Hi, Dmitry
>
> I think this is a good project.
> I got one question: what is the relationship with ironic-python-agent?
> Thanks.
Hi!

No relationship right now, but I'm hoping to use IPA as a base for 
introspection ramdisk in the (near?) future.
>
> BR
> Zhou Zhenzan
>
> -Original Message-
> From: Dmitry Tantsur [mailto:dtant...@redhat.com]
> Sent: Thursday, December 11, 2014 10:35 PM
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: [openstack-dev] [Ironic] ironic-discoverd status update
>
> Hi all!
>
> As you know I actively promote ironic-discoverd project [1] as one of the 
> means to do hardware inspection for Ironic (see e.g. spec [2]), so I decided 
> it's worth to give some updates to the community from time to time. This 
> email is purely informative, you may safely skip it, if you're not interested.
>
> Background
> ==
>
> The discoverd project (I usually skip the "ironic-" part when talking 
> about it) solves the problem of populating information about a node in 
> Ironic database without help of any vendor-specific tool. This 
> information usually includes Nova scheduling properties (CPU, RAM, 
> disk
> size) and MAC's for ports.
>
> Introspection is done by booting a ramdisk on a node, collecting data there 
> and posting it back to discoverd HTTP API. Thus actually discoverd consists 
> of 2 components: the service [1] and the ramdisk [3]. The service handles 2 
> major tasks:
> * Processing data posted by the ramdisk, i.e. finding the node in Ironic 
> database and updating node properties with new data.
> * Managing iptables so that the default PXE environment for 
> introspection does not interfere with Neutron
>
> The project was born from a series of patches to Ironic itself after we 
> discovered that this change is going to be too intrusive. Discoverd was 
> actively tested as part of Instack [4] and it's RPM is a part of Juno RDO. 
> After the Paris summit, we agreed on bringing it closer to the Ironic 
> upstream, and now discoverd is hosted on StackForge and tracks bugs on 
> Launchpad.
>
> Future
> ==
>
> The basic feature of discoverd: supply Ironic with properties required for 
> scheduling, is pretty finished as of the latest stable series 0.2.
>
> However, more features are planned for release 1.0.0 this January [5].
> They go beyond the bare minimum of finding out CPU, RAM, disk size and NIC 
> MAC's.
>
> Plugability
> ~~~
>
> An interesting feature of discoverd is support for plugins, which I prefer to 
> call hooks. It's possible to hook into the introspection data processing 
> chain in 2 places:
> * Before any data processing. This opens opportunity to adopt discoverd to 
> ramdisks that have different data format. The only requirement is that the 
> ramdisk posts a JSON object.
> * After a node is found in Ironic database and ports are created for MAC's, 
> but before any actual data update. This gives an opportunity to alter, which 
> properties discoverd is going to update.
>
> Actually, even the default logic of update Node.properties is 
> contained in a plugin - see SchedulerHook in 
> ironic_discoverd/plugins/standard.py
> [6]. This plugability opens wide opportunities for integrating with 3rd party 
> ramdisks and CMDB's (which as we know Ironic is not ;).
>
> Enrolling
> ~
>
> Some people have found it limiting that the introspection requires power 
> credentials (IPMI user name and password) to be already set. The recent set 
> of patches [7] introduces a possibility to request manual power on of the 
> machine and update IPMI credentials via the ramdisk to the expected values. 
> Note that support of this feature in the reference ramdisk [3] is not ready 
> yet. Also note that this scenario is only possible when using discoverd 
> directly via it's API, not via Ironic API like in [2].
>
> Get Involved
> 
>
> Discoverd terribly lacks reviews. Out team is very small and 
> self-approving is not a rare case. I'm even not against fast-tracking 
> any existing Ironic core to a discoverd core after a couple of 
> meaningful reviews :)
>
> And 

Re: [openstack-dev] [Ironic] ironic-discoverd status update

2015-01-05 Thread Dmitry Tantsur

On 01/05/2015 09:31 AM, Zhou, Zhenzan wrote:

Hi, Dmitry

I think this is a good project.
I got one question: what is the relationship with ironic-python-agent?
Thanks.

Hi!

No relationship right now, but I'm hoping to use IPA as a base for 
introspection ramdisk in the (near?) future.


BR
Zhou Zhenzan

-Original Message-
From: Dmitry Tantsur [mailto:dtant...@redhat.com]
Sent: Thursday, December 11, 2014 10:35 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [Ironic] ironic-discoverd status update

Hi all!

As you know I actively promote ironic-discoverd project [1] as one of the means 
to do hardware inspection for Ironic (see e.g. spec [2]), so I decided it's 
worth to give some updates to the community from time to time. This email is 
purely informative, you may safely skip it, if you're not interested.

Background
==

The discoverd project (I usually skip the "ironic-" part when talking about it) 
solves the problem of populating information about a node in Ironic database without help 
of any vendor-specific tool. This information usually includes Nova scheduling properties 
(CPU, RAM, disk
size) and MAC's for ports.

Introspection is done by booting a ramdisk on a node, collecting data there and 
posting it back to discoverd HTTP API. Thus actually discoverd consists of 2 
components: the service [1] and the ramdisk [3]. The service handles 2 major 
tasks:
* Processing data posted by the ramdisk, i.e. finding the node in Ironic 
database and updating node properties with new data.
* Managing iptables so that the default PXE environment for introspection does 
not interfere with Neutron

The project was born from a series of patches to Ironic itself after we 
discovered that this change is going to be too intrusive. Discoverd was 
actively tested as part of Instack [4] and it's RPM is a part of Juno RDO. 
After the Paris summit, we agreed on bringing it closer to the Ironic upstream, 
and now discoverd is hosted on StackForge and tracks bugs on Launchpad.

Future
==

The basic feature of discoverd: supply Ironic with properties required for 
scheduling, is pretty finished as of the latest stable series 0.2.

However, more features are planned for release 1.0.0 this January [5].
They go beyond the bare minimum of finding out CPU, RAM, disk size and NIC 
MAC's.

Plugability
~~~

An interesting feature of discoverd is support for plugins, which I prefer to 
call hooks. It's possible to hook into the introspection data processing chain 
in 2 places:
* Before any data processing. This opens opportunity to adopt discoverd to 
ramdisks that have different data format. The only requirement is that the 
ramdisk posts a JSON object.
* After a node is found in Ironic database and ports are created for MAC's, but 
before any actual data update. This gives an opportunity to alter, which 
properties discoverd is going to update.

Actually, even the default logic of update Node.properties is contained in a 
plugin - see SchedulerHook in ironic_discoverd/plugins/standard.py
[6]. This plugability opens wide opportunities for integrating with 3rd party 
ramdisks and CMDB's (which as we know Ironic is not ;).

Enrolling
~

Some people have found it limiting that the introspection requires power 
credentials (IPMI user name and password) to be already set. The recent set of 
patches [7] introduces a possibility to request manual power on of the machine 
and update IPMI credentials via the ramdisk to the expected values. Note that 
support of this feature in the reference ramdisk [3] is not ready yet. Also 
note that this scenario is only possible when using discoverd directly via it's 
API, not via Ironic API like in [2].

Get Involved


Discoverd terribly lacks reviews. Out team is very small and self-approving is 
not a rare case. I'm even not against fast-tracking any existing Ironic core to 
a discoverd core after a couple of meaningful reviews :)

And of course patches are welcome, especially plugins for integration with 
existing systems doing similar things and CMDB's. Patches are accepted via 
usual Gerrit workflow. Ideas are accepted as Launchpad blueprints (we do not 
follow the Gerrit spec process right now).

Finally, please comment on the Ironic spec [2], I'd like to know what you think.

References
==

[1] https://pypi.python.org/pypi/ironic-discoverd
[2] https://review.openstack.org/#/c/135605/
[3]
https://github.com/openstack/diskimage-builder/tree/master/elements/ironic-discoverd-ramdisk
[4] https://github.com/agroup/instack-undercloud/
[5] https://bugs.launchpad.net/ironic-discoverd/+milestone/1.0.0
[6]
https://github.com/stackforge/ironic-discoverd/blob/master/ironic_discoverd/plugins/standard.py
[7]
https://blueprints.launchpad.net/ironic-discoverd/+spec/setup-ipmi-credentials

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.o

Re: [openstack-dev] [Ironic] ironic-discoverd status update

2015-01-05 Thread Zhou, Zhenzan
Hi, Dmitry

I think this is a good project. 
I got one question: what is the relationship with ironic-python-agent? 
Thanks.

BR
Zhou Zhenzan

-Original Message-
From: Dmitry Tantsur [mailto:dtant...@redhat.com] 
Sent: Thursday, December 11, 2014 10:35 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [Ironic] ironic-discoverd status update

Hi all!

As you know I actively promote ironic-discoverd project [1] as one of the means 
to do hardware inspection for Ironic (see e.g. spec [2]), so I decided it's 
worth to give some updates to the community from time to time. This email is 
purely informative, you may safely skip it, if you're not interested.

Background
==

The discoverd project (I usually skip the "ironic-" part when talking about it) 
solves the problem of populating information about a node in Ironic database 
without help of any vendor-specific tool. This information usually includes 
Nova scheduling properties (CPU, RAM, disk
size) and MAC's for ports.

Introspection is done by booting a ramdisk on a node, collecting data there and 
posting it back to discoverd HTTP API. Thus actually discoverd consists of 2 
components: the service [1] and the ramdisk [3]. The service handles 2 major 
tasks:
* Processing data posted by the ramdisk, i.e. finding the node in Ironic 
database and updating node properties with new data.
* Managing iptables so that the default PXE environment for introspection does 
not interfere with Neutron

The project was born from a series of patches to Ironic itself after we 
discovered that this change is going to be too intrusive. Discoverd was 
actively tested as part of Instack [4] and it's RPM is a part of Juno RDO. 
After the Paris summit, we agreed on bringing it closer to the Ironic upstream, 
and now discoverd is hosted on StackForge and tracks bugs on Launchpad.

Future
==

The basic feature of discoverd: supply Ironic with properties required for 
scheduling, is pretty finished as of the latest stable series 0.2.

However, more features are planned for release 1.0.0 this January [5]. 
They go beyond the bare minimum of finding out CPU, RAM, disk size and NIC 
MAC's.

Plugability
~~~

An interesting feature of discoverd is support for plugins, which I prefer to 
call hooks. It's possible to hook into the introspection data processing chain 
in 2 places:
* Before any data processing. This opens opportunity to adopt discoverd to 
ramdisks that have different data format. The only requirement is that the 
ramdisk posts a JSON object.
* After a node is found in Ironic database and ports are created for MAC's, but 
before any actual data update. This gives an opportunity to alter, which 
properties discoverd is going to update.

Actually, even the default logic of update Node.properties is contained in a 
plugin - see SchedulerHook in ironic_discoverd/plugins/standard.py
[6]. This plugability opens wide opportunities for integrating with 3rd party 
ramdisks and CMDB's (which as we know Ironic is not ;).

Enrolling
~

Some people have found it limiting that the introspection requires power 
credentials (IPMI user name and password) to be already set. The recent set of 
patches [7] introduces a possibility to request manual power on of the machine 
and update IPMI credentials via the ramdisk to the expected values. Note that 
support of this feature in the reference ramdisk [3] is not ready yet. Also 
note that this scenario is only possible when using discoverd directly via it's 
API, not via Ironic API like in [2].

Get Involved


Discoverd terribly lacks reviews. Out team is very small and self-approving is 
not a rare case. I'm even not against fast-tracking any existing Ironic core to 
a discoverd core after a couple of meaningful reviews :)

And of course patches are welcome, especially plugins for integration with 
existing systems doing similar things and CMDB's. Patches are accepted via 
usual Gerrit workflow. Ideas are accepted as Launchpad blueprints (we do not 
follow the Gerrit spec process right now).

Finally, please comment on the Ironic spec [2], I'd like to know what you think.

References
==

[1] https://pypi.python.org/pypi/ironic-discoverd
[2] https://review.openstack.org/#/c/135605/
[3]
https://github.com/openstack/diskimage-builder/tree/master/elements/ironic-discoverd-ramdisk
[4] https://github.com/agroup/instack-undercloud/
[5] https://bugs.launchpad.net/ironic-discoverd/+milestone/1.0.0
[6]
https://github.com/stackforge/ironic-discoverd/blob/master/ironic_discoverd/plugins/standard.py
[7]
https://blueprints.launchpad.net/ironic-discoverd/+spec/setup-ipmi-credentials

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
h