On 11/10/16 02:24, Rabi Mishra wrote:
1.2
For large stacks, 1000 nodes, the final call to heat to
fetch the
IPs for all nodes takes 3 to 4 minutes. In heat, the stack
has status
CREATE_COMPLETE but magnum's state is updated when this long
final
call is done. Can we do better? Maybe fetch only the master
IPs or
get he IPs in chunks.
We seem load the nested stacks in memory to retrieve their outputs. That
would probably explain the
behaviour above, where you load all the nested stacks for the nodes to
fetch their ips. There is some
work[1][2] happening atm to change that.
[1] https://review.openstack.org/#/c/383839/
<https://review.openstack.org/#/c/383839/>
[2] https://review.openstack.org/#/c/384718
<https://review.openstack.org/#/c/384718>
That's a problem, but in Magnum's case I believe the tree is pretty
shallow. The real problem there is that we dynamically resolve intrinsic
functions (like get_attr) in outputs - meaning that in the case
mentioned we will make 1000 calls to Nova to get the IP address of each
node, every time you show the stack.
This is nuts not only from a performance standpoint and also for simple
consistency of data. I think we should move to storing the data at the
end up a stack create/update, so that we can return it instantly when we
show the stack. (I do worry about breaking users who may be relying on
live updates to that data, but I think we *have* to make this change
sometime.) With convergence we already have a lot of the code we'd need
in place - we already know how to grab all the attributes we need at the
time a resource is created/updated and store them to the SyncPoint, and
we already have a SyncPoint for when the whole stack is complete. It's
just a matter of finding a longer-term place to cache that data, since
SyncPoints are pretty short-lived at the moment.
I definitely think we should discuss this in the session.
cheers,
Zane.
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [email protected]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
