Re: [openstack-dev] [magnum] Generate atomic images usingdiskimage-builder

[Steve Gordon]

- Original Message -
> From: "Kai Qiang Wu" <wk...@cn.ibm.com>
> To: "OpenStack Development Mailing List (not for usage questions)" 
> <openstack-dev@lists.openstack.org>
> Sent: Wednesday, March 23, 2016 9:10:30 PM
> Subject: Re: [openstack-dev] [magnum] Generate atomic images  
> usingdiskimage-builder
> 
> 1) ＋1  diskimage-builder maybe a better place to external consumption.
> 
> 2) For the image size difference(big), I think we may need to know what's
> the issue for that.
> Maybe Redhat guys know something about it.

It's worth asking on cl...@lists.fedoraproject.org but I believe the relevant 
inputs to the process they use which you might be able to compare against are 
here:

https://git.fedorahosted.org/cgit/fedora-atomic.git/tree/README.md
https://git.fedorahosted.org/cgit/spin-kickstarts.git/tree/fedora-cloud-atomic.ks

Thanks,

Steve

> 
> Kai Qiang Wu (吴开强  Kennan）
> IBM China System and Technology Lab, Beijing
> 
> E-mail: wk...@cn.ibm.com
> Tel: 86-10-82451647
> Address: Building 28(Ring Building), ZhongGuanCun Software Park,
>  No.8 Dong Bei Wang West Road, Haidian District Beijing P.R.China
> 100193
> 
> Follow your heart. You are miracle!
> 
> 
> 
> From: "Ton Ngo" <t...@us.ibm.com>
> To:   "OpenStack Development Mailing List \(not for usage questions
> \)" <openstack-dev@lists.openstack.org>
> Date: 24/03/2016 01:12 am
> Subject:  Re: [openstack-dev] [magnum] Generate atomic images using
> diskimage-builder
> 
> 
> 
> Hi Yolanda,
> Thank you for making a huge improvement from the manual process of building
> the Fedora Atomic image.
> Although Atomic does publish a public OpenStack image that is being
> considered in this patch:
> https://review.openstack.org/#/c/276232/
> in the past we have come to many situations where we need an image with
> specific version of certain software
> for features or bug fixes (Kubernetes, Docker, Flannel, ...). So the
> automated and customizable build process
> will be very helpful.
> 
> With respect to where to land the patch, I think diskimage-builder is a
> reasonable target.
> If it does not land there, Magnum does currently have 2 sets of
> diskimage-builder elements for Mesos image
> and Ironic image, so it is also reasonable to submit the patch to Magnum.
> With the new push to reorganize
> into drivers for COE and distro, the elements would be a natural fit for
> Fedora Atomic.
> 
> As for periodic image build, it's a good idea to stay current with the
> distro, but we should avoid the situation
> where something new in the image breaks a COE and we are stuck for awhile
> until a fix is made. So instead of
> an automated periodic build, we might want to stage the new image to make
> sure it's good before switching.
> 
> One question: I notice the image built by DIB is 871MB, similar to the
> manually built image, while the
> public image from Atomic is 486MB. It might be worthwhile to understand the
> difference.
> 
> Ton Ngo,
> 
> Inactive hide details for Yolanda Robla Mota ---03/23/2016 04:12:54 AM---Hi
> I wanted to start a discussion on how Fedora AtomicYolanda Robla Mota
> ---03/23/2016 04:12:54 AM---Hi I wanted to start a discussion on how Fedora
> Atomic images are being
> 
> From: Yolanda Robla Mota <yolanda.robla-m...@hpe.com>
> To: <openstack-dev@lists.openstack.org>
> Date: 03/23/2016 04:12 AM
> Subject: [openstack-dev] [magnum] Generate atomic images using
> diskimage-builder
> 
> 
> 
> Hi
> I wanted to start a discussion on how Fedora Atomic images are being
> built. Currently the process for generating the atomic images used on
> Magnum is described here:
> http://docs.openstack.org/developer/magnum/dev/build-atomic-image.html.
> The image needs to be built manually, uploaded to fedorapeople, and then
> consumed from there in the magnum tests.
> I have been working on a feature to allow diskimage-builder to generate
> these images. The code that makes it possible is here:
> https://review.openstack.org/287167
> This will allow that magnum images are generated on infra, using
> diskimage-builder element. This element also has the ability to consume
> any tree we need, so images can be customized on demand. I generated one
> image using this element, and uploaded to fedora people. The image has
> passed tests, and has been validated by several people.
> 
> So i'm raising that topic to decide what should be the next steps. This
> change to generate f

Re: [openstack-dev] [magnum] Generate atomic images usingdiskimage-builder

[Kai Qiang Wu]

1) ＋1  diskimage-builder maybe a better place to external consumption.

2) For the image size difference(big), I think we may need to know what's
the issue for that.
Maybe Redhat guys know something about it.


Thanks


Thanks

Best Wishes,

Kai Qiang Wu (吴开强  Kennan）
IBM China System and Technology Lab, Beijing

E-mail: wk...@cn.ibm.com
Tel: 86-10-82451647
Address: Building 28(Ring Building), ZhongGuanCun Software Park,
 No.8 Dong Bei Wang West Road, Haidian District Beijing P.R.China
100193

Follow your heart. You are miracle!



From:   "Ton Ngo" 
To: "OpenStack Development Mailing List \(not for usage questions
\)" 
Date:   24/03/2016 01:12 am
Subject:Re: [openstack-dev] [magnum] Generate atomic images using
diskimage-builder



Hi Yolanda,
Thank you for making a huge improvement from the manual process of building
the Fedora Atomic image.
Although Atomic does publish a public OpenStack image that is being
considered in this patch:
https://review.openstack.org/#/c/276232/
in the past we have come to many situations where we need an image with
specific version of certain software
for features or bug fixes (Kubernetes, Docker, Flannel, ...). So the
automated and customizable build process
will be very helpful.

With respect to where to land the patch, I think diskimage-builder is a
reasonable target.
If it does not land there, Magnum does currently have 2 sets of
diskimage-builder elements for Mesos image
and Ironic image, so it is also reasonable to submit the patch to Magnum.
With the new push to reorganize
into drivers for COE and distro, the elements would be a natural fit for
Fedora Atomic.

As for periodic image build, it's a good idea to stay current with the
distro, but we should avoid the situation
where something new in the image breaks a COE and we are stuck for awhile
until a fix is made. So instead of
an automated periodic build, we might want to stage the new image to make
sure it's good before switching.

One question: I notice the image built by DIB is 871MB, similar to the
manually built image, while the
public image from Atomic is 486MB. It might be worthwhile to understand the
difference.

Ton Ngo,

Inactive hide details for Yolanda Robla Mota ---03/23/2016 04:12:54 AM---Hi
I wanted to start a discussion on how Fedora AtomicYolanda Robla Mota
---03/23/2016 04:12:54 AM---Hi I wanted to start a discussion on how Fedora
Atomic images are being

From: Yolanda Robla Mota 
To: 
Date: 03/23/2016 04:12 AM
Subject: [openstack-dev] [magnum] Generate atomic images using
diskimage-builder



Hi
I wanted to start a discussion on how Fedora Atomic images are being
built. Currently the process for generating the atomic images used on
Magnum is described here:
http://docs.openstack.org/developer/magnum/dev/build-atomic-image.html.
The image needs to be built manually, uploaded to fedorapeople, and then
consumed from there in the magnum tests.
I have been working on a feature to allow diskimage-builder to generate
these images. The code that makes it possible is here:
https://review.openstack.org/287167
This will allow that magnum images are generated on infra, using
diskimage-builder element. This element also has the ability to consume
any tree we need, so images can be customized on demand. I generated one
image using this element, and uploaded to fedora people. The image has
passed tests, and has been validated by several people.

So i'm raising that topic to decide what should be the next steps. This
change to generate fedora-atomic images has not already landed into
diskimage-builder. But we have two options here:
- add this element to generic diskimage-builder elements, as i'm doing now
- generate this element internally on magnum. So we can have a directory
in magnum project, called "elements", and have the fedora-atomic element
here. This will give us more control on the element behaviour, and will
allow to update the element without waiting for external reviews.

Once the code for diskimage-builder has landed, another step can be to
periodically generate images using a magnum job, and upload these images
to OpenStack Infra mirrors. Currently the image is based on Fedora F23,
docker-host tree. But different images can be generated if we need a
better option.

As soon as the images are available on internal infra mirrors, the tests
can be changed, to consume these internals images. By this way the tests
can be a bit faster (i know that the bottleneck is on the functional
testing, but if we reduce the download time it can help), and tests can
be more reilable, because we will be removing an external dependency.

So i'd like to get more feedback on this topic, options and next steps