On 26/04/13 17:18, Ken Giusti wrote:
Hey Bill,
I first started to implement the additional objectUpdate callback as originally
proposed. Very easy to do. But that additional api required other tools to be
updated - and additional documentation changes, etc.
Hmm I remain a little baffled here. So as I understood it prior to any
fixing going on here there existed a bug that actually prevented QMF2
updates actually hitting callbacks on the asynchronous python API,
that's good IMHO because most clients of that API almost certainly were
expecting a single callback call for a given object.
I'm kind of unclear about "other tools to be updated" 'cause I'd have
assumed given the previous QMF2 bug that no existing tools were actually
receiving v2 updates so wouldn't need "fixed". Sure they will need
modified to handle V2 if they want to consume both, but I'd argue that
it might be better to modify tools that need to handle both than to risk
breaking changes which will blat an indeterminate number of third party
tools. It's not ideal for sure, but a non-breaking API extension
requiring an addition to clients to use it feels better and TBH it's
actually clearer IMHO because there's some conscious choice being made.
After implementing this, I discovered a problem with the fix: tools that need
to monitor both QMFv1 and QMFv2 agents starting reporting incorrect results
(qpid-tool, to be specific). These tools need to support both the old
callbacks (for QMFv1 updates) and the new one (for QMFv2 updates), in order to
see both QMFv1 and QMFv2 agents.
To be honest as I suggest above IMHO I actually think it's a good thing
for tools that wish to support both to be explicit. I think from the
various sections of this thread it's pretty clear that trying to crowbar
everything together is a *bad idea*. Sure QMF1 and QMF2 are essentially
representing equivalent information but there are sufficient differences
and subtleties to flash a great big red light. The reason it's good to
have a separate objectUpdate is because the QMF2 API *is* different at
that point because the protocol is sending what amounts to incompatible
data.
At that point there are two choices 1) an additional non-breaking API
callback or 2) a shim in the client side that makes things behave
identically.
The latter may be a bad choice if both v1 and v2 updates arrive because
it'd need to apply a filter for the case of v1 plus v2 but it'd probably
be a good choice if it could select either v1 or v2 because then
existing clients wouldn't need to be modified at all.
And that's when I realized that this fix was just a hack to work around the
actual problem. The real issue at hand is that a QMF agent must not transmit
both QMFv1 and QMFv2 updates for the same object.
That's really what's broken here, and the C++ broker is doing precisely that.
Whoa there :-) I think that the problem is that this is *exactly* what
it should be doing :-)
OK what I mean is how does it know? Bear in mind that the updates
causing the fun here are *unsolicited* data pushes! These updates are
being pushed to a topic, so unless any client is subscribing to
qmf.default.topic/data.ind.# or whatever it isn't going to be getting v2
updates, similarly if it's not subscribing to the equivalent on
qpid.management/ it's not going to be getting v1 updates. OTOH if it's
subscribing to both that's exactly what it's getting.
The key thing is there's a fairly large level of decoupling going on
here, so I don't believe that there's an especially easy way for the
broker ManagementAgent to work out who's asking for what and thus avoid
transmitting both v1 and v2.
Funnily enough in the v2 protocol and API there's actually no concept of
an *unsolicited* data push, so I'd argue that what the broker is
currently doing for the v2 updates is an "undocumented feature" :-),
what V2 actually talks about is the concept of "query subscriptions".
With query subscriptions the asynchronous pushes are actually
*solicited* by the client actually requesting some specified subset of
the data, so the Management Agent does actually have a mechanism to
track what data any given subscribers are actually interested in.
That's clearly a moot point, but it would help with this pickle.
I was musing over this problem and I believe that there is an option,
albeit a non-trivial one :-(
So the *real* issue is that it's difficult to manage the carnage caused
when both v1 and v2 updates happen. You've previously suggested
disabling v1 by default, I've sort of got sympathy with that :-) However
there are two (I suspect significant) use cases that will be impacted by
that choice:
1) Any asynchronous console that relies on the existing v1 callbacks
will stop working - I suspect that in practice that'll mean every single
darned one of them except (at a push) qpid-tool. There may be a lot of
third party tools using this API so it may be an issue. At the very
least there needs to be some real good communication to the user list.
2) If you disable v1 then any third party Agents will break too. Again
perhaps they need to change to v2 and this is a catalyst, but it just
may cause non-trivial business impact on users.
Ultimately I think that it's very worth pushing out a survey on the user
list to gauge the potential impact.
The non-trivial mitigation that I was musing over was whether there was
a possibility of constructing a v1->v2 bridge Agent? What I mean is that
it might be possible to get the broker ManagementAgent to only send v2
updates if there's a mechanism for the broker to intercept v1 updates
from third party Agents and map those into v2 too, thus making every
data push consistently V2? I *think* this could work because I believe
that v1 requires broker involvement in order to work (I think that was
one of the reasons for moving to v2).
If it were possible to get to a clear point of only sending v2 updates
then it should be possible to have the asynchronous Console API shimmed
in such a way that it behaves *exactly* the same way with v2 updates as
it did with v1 updates so it's possible to mitigate the impact on third
party console applications (yay!).
It has to be said It's be much better if users didn't have to rewrite
applications because of objectUpdate versus propUpdate/statUpdate but
clearly a shim would be required to make v2 updates behave correctly and
transparently for propUpdate/statUpdate callbacks.
The result of the survey question "how many people have custom v1
Agents" would help determine whether it's worth investing in a v1->v2
bridge Agent.
A bridge ought to be possible, that's actually exactly what I wrote for
the Java Broker in the Qmf2ManagementAgent to get that to talk QMF2
(albeit I was mapping from its internal model and not QMF1, but the
principal is the same - just don't ask me to do it in C++ though :-))
Why is that a problem? Because a console receiving these two updates cannot
tell that they are for the same object. To the console application, it appears
as two separate objects.
As I mention above I don't believe that it's any easier for the broker
to do a "diff" given the unsolicited nature of these updates :-)
I'm proposing that we should solve this problem by disabling QMFv1 updates
coming from the C++ broker. This should be the default for the next release
(0.26). It should still be possible to turn them on manually if desired, but
the C++ broker should only transmit QMFv2 type updates going forward.
As I say above I've got a fair amount of sympathy with this, and
parochially if it means that by doing this you can make statUpdate and
propUpdate for v2 behave *exactly* the same as for v1 so clients don't
need to be rewritten for v2 then it'd be better for me 'cause I only
ever use v2 aside from this.
However as I said above the only reason that this Console API ever
supported both was to cater for the case where a user needed to handle
both v1 and v2 updates and that use case is actually third party v1 Agents.
I think it's important to at least check for how much impact this is
going to cause users.
We can minimize the impact to console applications by having the objectProps
and objectStats callbacks be invoked when a QMFv2 object arrives instead of
introducing a new callback. [Bill - I'll fix the console to not call
objectProps when only stats are present]. I think this should eliminate the
need to re-code console applications.
So I don't think it's about "minimising the impact" I think that the API
needs to behave *exactly* the same for this to be any use. Basically any
asynchronous client hit by only v2 updates should behave identically to
how they behaved prior to this update or it's still introducing a
regression into an indeterminate number of applications. I've no idea
how hard it'd be to achieve this nirvana though.
This will cause problems for folks that use an old python console against the
next release of the broker. These folks will have to upgrade the console as
well.
Does anyone have a better alternative?
Well I'm not necessarily saying that my suggestions above are *better*
:-) It's difficult for sure. I guess that my biggest concern is
introducing breaking changes against an unknown number of third party
Consoles and Agents.
One other thing makes me nervous. I guess that there's the scenario
where users are working with a mixed economy of brokers at different
versions or who (like Bill) may be unable to alter the broker config to
only push out a particular version.
All brokers from at least 0.8 can push out v2 so it *might* be safe to
have the console only subscribe to v2 updates (providing a switch to
specify v1). If you can make propUpdate and statUpdate behave
consistently then that only leaves the case where there are v1 Agents to
support, which is where the bridge Agent may come in.
How much do you regret starting on this fix now Ken? :-D
It's worth pointing out that this is I suspect only the tip of the
iceberg for the sort of problems that might start to be encountered when
migration to AMQP 1.0 Management starts to gain traction. My strong
hunch is that bridge Agents/ManagementNodes are going to become pretty
important friends when it comes to trying to manage any transition. It's
something that's going to bite sooner or later and the C++, Java &
Python communities are going to have to be pretty joined up. I quite
like QMF, but clearly some mistakes were made along the way given the
lack of support in the Java community and frankly the fragmentation of
APIs and lack of take up on the "official" QMF2 API. We *really* need to
try harder next time round and act as an exemplar in the AMQP community.
Frase
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
