Hi ironic folks,
As I'm trying to explore how GoDaddy can use ironic I've created the following
in an attempt to document some of my concerns, and I'm wondering if you folks
could help myself identity ongoing work to solve these (or alternatives?)
List of concerns with ironic:
1.)Nova <-> ironic interactions are generally seem terrible?
-How to accept raid config and partitioning(?) from end users? Seems to not a
yet agreed upon method between nova/ironic.
-How to run multiple conductors/nova-computes? Right now as far as I can
tell all of ironic front-end by a single nova-compute, which I will have to
manage via a cluster technology between two or mode nodes. Because of this and
the way host-agregates work I am unable to expose fault domains for ironic
instances (all of ironic can only be under a single AZ (the az that is assigned
to the nova-compute node)). Unless I create multiple nova-compute servers and
manage multiple independent ironic setups. This makes on-boarding/query of
hardware capacity painful.
- Nova appears to be forcing a we are "compute" as long as "compute" is VMs,
means that we will have a baremetal flavor explosion (ie the mismatch between
baremetal and VM).
- This is a feeling I got from the ironic-nova cross project meeting in
Austin. General exmaple goes back to raid config above. I can configure a
single piece of hardware many different ways, but to fit into nova's world view
I need to have many different flavors exposed to end-user. In this way many
flavors can map back to a single piece of hardware with just a lsightly
different configuration applied. So how am I suppose to do a single server with
6 drives as either: Raid 1 + Raid 5, Raid 5, Raid 10, Raid 6, or JBOD. Seems
like I would need to pre-mark out servers that were going to be a specific raid
level. Which means that I need to start managing additional sub-pools of
hardware to just deal with how the end users wants the raid configured, this is
pretty much a non-starter for us. I have not really heard of whats being done
on this specific front.
2.) Inspector:
- IPA service doesn't gather port/switching information
- Inspection service doesn't process port/switching information, which means
that it wont add it to ironic. Which makes managing network swinging of the
host a non-starter. As I would inspect the host – then modify the ironic
record to add the details about what port/switch the server is connected to
from a different source. At that point why wouldn't I just onboard everything
through the API?
- Doesn't grab hardware disk configurations, If the server has multiple raids
(r1 + r5) only reports boot raid disk capacity.
- Inspection is geared towards using a different network and dnsmasq
infrastructure than what is in use for ironic/neutron. Which also means that
in order to not conflict with dhcp requests for servers in ironic I need to use
different networks. Which also means I now need to handle swinging server
ports between different networks.
3.) IPA image:
- Default build stuff is pinned to extremly old versions due to gate failure
issues. So I can not work without a fork for onboard of servers due to the fact
that IPMI modules aren't built for the kernel, so inspection can never match
the node against ironic. Seems like currently functionality here is MVP for
gate to work and to deploy images. But if you need to do firmware,
bios-config, any other hardware specific features you are pretty much going to
need to roll your own IPA image and IPA modules to do standard provisioning
tasks.
4.) Conductor:
- Serial-over-lan consoles require a unique port on the conductor server (I
have seen purposes to try and fix this?), this is painful to manage with large
numbers of servers.
- SOL consoles aren't restarted when conductor is restarted (I think this
might be fixed in newer versions of ironic?), again if end users aren't suppose
to consume ironic api's directly - this is painful to handle.
- Its very easy to get a node to fall off the staemachine rails (reboot a
server while an image is being deployed to it), the only way I have seen to be
able to fix this is to update the DB directly.
- As far as I can tell shell-in-a- box, SOL consoles aren't support via nova
– so how are end users suppose to consume the shell-in-box console?
- I have BMC that need specific configuration (some require SOL on com2,
others on com1) this makes it pretty much impossible without per box overrides
against the conductor hardcoded templates.
- Additionally it would be nice to default to having a provisioning
kernel/image that was set as a single config option with per server overrides –
rather than on each server. If we ever change the IPA image – that means at
scale we would need to update thousands of ironic nodes.
What is ironic doing to monitor the hardware for failures? I assume the answer
here is nothing and that we will need to make sure the images that we deploy
are correctly configuring the tools to monitor disk/health/psu's/ram errors,
ect. ect.
Overall the above concerns have me wondering if I am going crazy, or is ironic
really not ready to take over datacenters baremetal provisioning (unless you
have a very limited view of functionality that is needed in those datacenters).
Thoughts would be great,
___________________________________________________________________
Kris Lindgren
Senior Linux Systems Engineer
GoDaddy
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [email protected]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
