On 02/09/2015 11:25 AM, Alec Hothan (ahothan) wrote:
On 9/1/15, 11:31 AM, "gord chung" <[email protected]> wrote:
re: serialisation, that probably isn't the biggest concern for
Ceilometer performance. the main items are storage -- to be addressed by
Gnocchi/tsdb, and polling load. i just thought i'd point out an existing
serialisation patch since we were on the topic :-)
Is there any data measuring the polling load on large scale deployments?
Was there a plan to reduce the polling load to an acceptable level? If yes
could you provide any pointer if any?
i'm not sure any user has provided numbers when raising the issue --
just that it's 'high'. this should probably be done in a separate thread
as i don't want it to get lost in completely unrelated subject. that
said, an initial patch to minimise load was done in Liberty[1] and
secondary proposal for M*[2].
conceptually, i would think only the consumers need to know about all
the queues and even then, it should only really need to know about
the ones it understands. the producers (polling agents) can just fire
off to the correct versioned queue and be done... thanks for the
above link (it'll help with discussion/spec design).
When everything goes according to plan, any solution can work but this is
hardly the case in production, especially at scale. Here are a few question
that may help in the discussion:
- how are versioned queue named?
- who creates a versioned queue (producer or consumer?) and who deletes it when
no more entity of that version is running?
- how to make sure a producer is not producing in a queue that has no consumer
(a messaging infra like rabbit is designed to decouple producers from consumers)
- all corner cases of entities (consumers or producers) popping up with newer
or older version, and terminating (gracefully or not) during the
upgrade/downgrade, what happens to the queues...
IMHO using a simple communication schema (1 topic/queue for all versions) with
in-band message versioning is a much less complex proposition than juggling
with versioned queues (not to say the former is simple to do). With versioned
queues you're kind of trading off the per message versioning with per queue
versioning but at the expense of:
- a complex queue management (if you want to do it right)
- a not less complex per queue message decoding (since the consumer needs to
know how to decode and interpret every message depending on the version of the
queue it comes from)
- a more difficult debug environment (harder to debug multiple queues than 1
queue)
- and added stress on oslo messaging (due to the use of transient queues)
thanks, good items to think about when building spec. will be sure to
add link when initial draft is ready.
[1]
https://blueprints.launchpad.net/ceilometer/+spec/resource-metadata-caching
[2] https://review.openstack.org/#/c/209799/
--
gord
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [email protected]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
