On 28 February 2014 19:36, Fraser Adams <[email protected]>wrote:
> On 27/02/14 18:38, Rob Godfrey wrote: > >> On 27 February 2014 18:57, Fraser Adams <[email protected] >> >wrote: >> >> [snip] >>> >>> So to get back to your point "Haven't you to some extent already done >>> that >>> with your QMF adapter work" well yes, but it's not actually a simple >>> yes/no >>> answer. The core properties/attributes all map fairly easily and I could >>> probably have got away without actually mapping those because I >>> introspected the names (I'll say why I did in a bit) but I *certainly* >>> needed to do something about the navigation because QMF and the Java >>> Broker >>> Management Model take very different approaches. That's why I'm a little >>> troubled that AMQP 1.0 Management is agnostic about >>> navigation/association >>> that's one of the hard bits and if everyone does their own thing then >>> it'll >>> make it very hard indeed to create tooling that's not very bespoke. >>> >> I think this is a valid point... It's probably a worthwhile exercise to >> think about the "standard" AMQP 1.0 entities and how they will be modelled >> and their relationships. For AMQP 1.0 that probably starts of consisting >> of "Nodes", "Links" and "Connections" (and possibly "Sessions"). In the >> general AMQP model exchanges and bindings don't exist - though we could >> maybe model them as node and bindings as "internal links" between nodes. >> > Funnily enough my "mental model" of AMQP 0.10 in an AMQP 1.0 context has > tended to be as above with bindings as "internal links". > > Though it's probably not as clear-cut as that. For sure if a producer is > sending to an exchange (as an addressable node) and a consumer is receiving > from a queue (as an addressable node) and we've established some bindings > administratively then creating said bindings is pretty close to > establishing an internal link, However if one has a consumer receiving from > an exchange as an addressable node it gets more interesting in that case > although we have a queue created (in physical terms) that queue is really > an artefact of the subscription and is (I think) actually a source terminus > associated to the exchange node and one or more links. > > That's all quite "exciting" because the same physical things on the broker > are semantically a little different depending on how things were > established, which messes with my head a bit :-D don't know whether it's > possible (or whether you'd want to) distinguish between the two different > scenarios in a Management Model?? > > > So to me, if you allow subscriptions to an exchange then you should present to the management user the same view that the application is getting (i.e. the fact that a queue object has been created to provide buffering is an implementation detail). I think ultimately in the Java Broker I would like to allow consumers to link to things that look a lot like exchanges, however I think the behaviour might be slightly different. To me an exchange is an instance of a class of nodes which passes messages from an incoming link to an outgoing link without ever taking responsibility for the message (or providing any buffering/storage). A queue is an instance of the class of nodes that does take responsibility for a message (and - as it happens - also provides storage). A link may or may not provide buffering / storage (links between containers will always have to provide buffering... internal links may or may not provide buffering - currently the Java Broker models message flows across bindings as a synchronous operation) but never itself takes responsibilitiy. So, to my mind I feel that an exchange with subscriptions should either insist that all messages sent to it are sent unreliably (pre-settled), or it should only settle when consumers have taken the message and taken responsibility themselves. This is somewhat different from the 0-x exchange behaviour where the broker takes responsibility for the message (because it actually places the message in one or more queues). > >> >> As it happens I actually mapped the properties because I was keen to see >>> if I could get it to work with the CLI tools such as qpid-config (as I >>> say >>> it'd probably have looked decent with the UI with only the navigation >>> mapped). I mention that because I think migration from existing tooling >>> is >>> going to be one of the more "interesting" aspects of moving to AMQP 1.0 >>> Management. So as you know I did a Java Broker plug-in that does the >>> mapping 'cause that was a fairly obvious place, but if both C++ and Java >>> Brokers talk AMQP 1.0 Management there's no obvious place to do mappings >>> - >>> I guess that the best hope in that case would be something like the >>> client >>> side of qpidtoolibs and similar for Java and C++, but I haven't though >>> too >>> hard about it yet (it's likely to need to happen in multiple places >>> though >>> rather than in the one place it currently happens). >>> >>> >>> I think properties and associated behaviours is something we should >> definitely be standardising between implementations under the Qpid >> umbrella, and looking to standardize amongst other AMQP implementations. >> That's not to say that all implementations will offer the same set, just >> that we should (and I feel I've said this a million times already) use the >> same properties and the same values for the same behaviours. >> > + 1,000,000 > I'm *definitely* all for this. > > So the $64,000 question is how do we go about achieving that aim. Clearly > QMF and the Java Broker Management Models have diverged a bit so it's going > to require a degree of maturity across the C++ and Java communities to > agree on what the names and types should be. It sounds simple, bet it isn't > :-) but just because it's likely to be hard shouldn't stop us trying, it's > definitely the way to go. > > > > Yes - I think we'd need to start out simple... all queues have names... right? and then see what aspects we think are common, which should be factored and where we currently have different ways of doing things trying to decide whether there is a common abstraction which we can use to describe across both instances (and in Dispatch and Messenger too). In general I feel there is probably more in common with stuff that both brokers are capable of than there is different... However there will also be large amounts of functionality that is specific to one broker or the other... and that is fine. Where the other side has an interest we should discuss the naming / semantics... and maybe aspire to implement at some later date... But as long as we keep things in bite-sizd chunks and not try to say that every queue has to implement all these bazillion features, then I don't think it will necessarily be as hard as you might think. > >> Personally I think the biggest gap in the AMQP 1.0 management stuff right >> now is any common way of describing "events" and a way to subscribe to >> them, whether that be getting period statistic updates, or notification of >> objects being created / deleted / etc. I feel that this is a common use >> case and again by standardising we would enable a broad class of use-cases >> to be enabled in a standard way without being overly prescriptive about >> exactly what statistics might be supplied (number of mallocs anyone :-) ). >> > I agree with this completely. > > I think I've said this to you before, but I do get a little frustrated by > the AMQP 1.0 Management stuff. For all its failings I do happen to think > that QMF2 is actually pretty good and had successfully addressed most of > the things we've been discussing and in addition had some pretty > sophisticated capabilities for generating events and in addition it had the > ability to allow clients to specify "query subscriptions" that told an > Agent to "push" specified data. As it happens the C++ broker didn't > implement query subscriptions rather it retained the QMF1 data push but I > implemented all this stuff in the Java QMF stuff and my Agent code can do > query subscriptions and I implemented client side code so I could "fake" > query subscriptions client side from broker data pushes. > > Anyway what I'm trying to say is that I feel slightly worried that there's > wheel reinvention going on. I really *do* get the fact that a key aim is to > be interoperable and it would have been hard to get a wide range of vendors > to buy into something called "Qpid Management Framework" but in many ways I > feel that an approach of starting with QMF2 and paring it back a bit (I do > think some of the QMF request/response properties are annoyingly > convoluted!!) would have been a better approach. > > Perhaps I'm just a bit naive about multi-vendor politics. > > I do kind of agree with your comment "number of mallocs anyone ", but the > right thing might be to have a generalisation of a "Broker" Manageable > Entity that covers common broker like useful stuff and specialisations for > different platforms and in this case perhaps number of mallocs might be > perfectly reasonable (- ust 'cause you can attach a JConsole to the Java > Broker doesn't mean it's possible in C++, you're just spoilt :-) > > So one of my issues with QMF is that schema and mechanism weren't as cleanly separated as I'd like. The ability to perform the CRUD operations appears to be a property of Broker objects rather than something generic... I completely agree that number of mallocs makes sense for some implementations... but it certainly doesn't make sense for all. What I want to see is that we have something that says "here's how you do common operations agaisnt any type of object - including some sort of event stuff", along side: "here are the common attributes, methods defined for these common entity types... though your vendor will likely have all sort of other goodies on top". One thing to remember is that for AMQP 1.0, we're not only thinking about the sort of role that QMF played for Qpid... but also the fact that in AMQP 1.0 there is no longer a way of creating queues programmatically... So the basic case that we wanted to cover quickly is a way to programmatically dynamically create (non temporary) queues and such. The wider management use cases perhaps haven't been as completely explored yet as they could be... And certainly having those who can contribute directly, and those who can't be members of OASIS contribute their thoughts to the Technical Committee through the TC comments mailing list would be of great value! > >> >> You'll recall a few threads on "Plans for QMF v1 and v2" last Easter that >>> were spawned on the back of plans to disable v1 push messages from qpidd >>> by >>> default and the resulting realisation that the Python asynchronous API >>> only >>> really worked for v1 at the time. I think that's probably the tip of the >>> iceberg compared to migrating to AMQP 1.0 Management. Perhaps the right >>> answer is not to migrate but rather make a clean break with new tools? I >>> don't know, do you guys have a preference? >>> >>> >>> On the Java side we'll continue supporting the existing management >> mechanisms (which are implemented as plugins) but out internal tooling >> will >> be focussed on AMQP manegement alongside the REST/API where appropriate. >> We will be trying very hard to keep the two of these using the same model >> so transitioning from one to the other should be seemless. >> >> That's good, FWIW I think that the plugin based approach on the Java > Broker has been a very good call - except when the API changes ;-D I'm > certainly more worried about the complexity of the transition for the > QMF/C++/Python community. That said I think that you're still going to have > some issues - for example if say some properties change as a result of the > sort of standardisation we've been discussing would you simply change them > and expect say users of the JMX plugin to handle this themselves, or would > you do a mapping to isolate users of your "legacy" interfaces? > > So, my view is that the JMX API will stay as it is (forever more). For the REST API I think we would try to migrate people as seamlessly as possible when attribute names changed or the semantics of operations did so... however we are very deliberately not at a 1.0 release because I think there are likely to be breakages in the APIs until we get stuff nailed down as a community, and hopefully in the wider AMQP world. -- Rob > Frase > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
