On 11/04/13 16:05, Ken Giusti wrote:
One thing though, I think that we discussed on the main thread (and Bill
mentioned this too) about providing an extension to specify the QMF
version to consume.
To be frank - I think any effort expended going forward to make it easier to
have both QMFv1 and QMFv2 work well together is an effort wasted given QMFv1
deprecation:
http://qpid.2158936.n2.nabble.com/QMF-and-Broker-Management-td7151372.html
Yes/no/maybe/possibly :-)
I think that frankly it's an awful lot more complex than that. When Ted
suggested that over a year ago I was very much onside with that view,
I've primarily used QMF2 and I think that there is a lot of fun things
that could potentially be done making interesting QMF2 Agents.
One of the hurdles that I think exists is the lack of cohesion around
QMF. So as I've mentioned before I felt a Java implementation was
missing, so I took it upon myself to do a complete Java implementation
of the draft QMF2 API that was documented (I, not unreasonably, assumed
that this was what the future was going to be because there was nothing
written nor said to the contrary). As it happens there exists a
"prototype" python implementation of that API (which I suspect few, if
any people are actually using) plus an entire miscellany of QMF2 "pseudo
APIs" for python, C++, ruby that generally bear no relationship to each
other (you see where I'm going with this :-))
Moreover from conversations that I've had with various people a lot of
the QMF stuff was perhaps put out without necessarily a lot of community
discussion (that all predates my involvement with the project so maybe
I'm speaking out of turn there), so as a result it has become rather
fragmented - QMF used in the C++ Broker and it's the only game in town
there, the Java Broker has gone its own way.
All this has to be blunt frankly wound me up :-D, and I've tended to get
on my soap box about cohesion and "compelling platforms".
Personally I think QMF2 for all it's failings is pretty good, but the
lack of consultation and cohesion have made it very C++ broker-centric
and thus not a compelling platform across even the Qpid world let alone
the general AMQP world.
So actually I think a decent amount of energy should really be focussed
on the broader AMQP 1.0 Management piece that is currently being thought
about.
However..... given that clearly there is an awful lot of QMF1 and QMF2
tooling out there (and indeed I myself have invested an awful lot of
personal energy into implementing a decent QMF2 API for Java plus web
GUI and all sorts) I'd say there's a danger that the AMQP 1.0 Management
piece will itself fail to become compelling if it doesn't consider the
complexities of migrating the exiting tooling (the QMF1 to QMF2 aspects
in the python API being discussed in this thread are really just a
flavour of the pain to come).
TBH all this is the primary reason that I put together a
QmfManagementPlugin for the Java Broker. It's early days yet and it's
far from perfect, but it was done on a point of principal really that
starting out in the direction of a cohesive management approach is going
to make it a darned sight easier to evolve towards AMQP 1.0 Management.
I really think that we have to play a lot more nicely together as a
community when it comes to management aspects, it's really far too
fragmented. For most large Enterprises it's one of the primary factors
in the decision making process for selecting messaging products. To put
it bluntly Qpid is a great set of products, but we're really doing
ourselves a great disservice if we don't get our act together on
management and monitoring.
So in short I sort of agree that doing too much with QMF1 might be
wasted effort, but not quite for the reasons you suggest - but
conversely (and slightly awkwardly) I suspect most of the current
tooling that exists in the real world (with the exception of my cool
stuff :-)) is I suspect still most likely to be be written using QMF1 -
so it's a conundrum, do we deprecate QMF1 early and risk breaking a lot
of existing Qpid and 3rd party tooling, forcing people to QMF2 when our
strategy should probably be towards AMQP 1.0 Management. or do we retain
QMF1 (possibly "semi"-deprecating it by disabling the push by default in
the broker) but encourage people to use things like qpidtoolib/my
stuff/whatever with the aim of trying to keep that tracking the current
thinking with AMQP 1.0 Management?
TBH I don't know the right answer. I know I quite like the stuff that
I've done and I know that I'm supportive of the stuff Rob is leading on
for AMQP 1.0 Management (even if there are currently bits of it that
give me apoplexy :-)) but trying to build cohesion and build a good
migration roadmap is going to be something of a challenge given the
current level of fragmentation.
I'll step gently off my soap-box now as this has turned into more of a
rant than I intended :-) at the very least it's an interesting topic of
conversation and strategic thinking on Qpid management is long overdue IMHO.
Agreed - but the sooner we turn off unsolicited QMFv1 updates as planned, the
sooner the broker stops sending the same data twice.
True, and the sooner most current Qpid 3rd party management tooling is
likely to break I would suspect :-/
Hopefully my rant above makes some sense, don't get me wrong I don't
much like the idea of prolonging support for QMF1, but as I say I think
it's more complicated than you are suggesting largely because we as a
community haven't done a terribly good job nailing our colours to the
mast and setting an agreed roadmap for management. It's now reached the
point of coming back to haunt us :-/
Yes, you'll see that the broker sends all statistics for a given object rather than just
those that changed. While the QMFv2 spec may indicate only the statistics that have
changed should be sent - this is very impractical in practice. That spec should be
re-written to say "-may- send only those that have changed".
To send only those stats that have changed will require the broker to perform
lots of extra work that will end up impacting the performance of its primary
purpose - transferring messages. Think about how we'd have to implement this -
the broker would have to save a snapshot of every object's statistics between
update intervals, then check every single counter for a difference at the next
poll (all under a lock, btw). Imagine the impact this would have on the
broker's performance, especially as we scale out queues, exchanges,
connections, etc. Not worth trading off for mgmt application convenience, IMHO.
-K
Ta for that, I knew that there was something weird going on that
affected my (emulated) query subscriptions, I couldn't remember which
way round it was that was only partially being sent. In retrospect it
makes sense that it's the properties, though my previous comment stands
in the general sense. If you look at management-schema.xml for say Queue
there are currently ~35 statistics fields and 7 properties so the saving
of not sending the properties too is probably not proportionate to the
cost of applications having to do a slightly irksome save of the first
update in a Map keyed by ObjectId to get the properties followed by a a
logical "join" of every updated QmfData where we need to look up the
partial state by ObjectId. All just IMHO of course, but it's certainly
one of the reasons I generally do getObjects() style calls - it's a lot
less faff :-)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
