Re: Client APIs and AMQP 1.0 (was Re: Using the messenger API to connect to a server without sending or subscribing)
On 05/21/2014 02:10 PM, Ken Giusti wrote: I think of qpid::messaging as being a "traditional" client api. [...] Messenger, as an alternative, provides (or at least promises to provide) solutions to a lot of the issues a "traditional" API has left to the application implementation. Things like connection failover, message retries, Automatic failover and message retry *is* supported in qpid::messaging (it isn't yet in Messenger). credit scheduling, What is that exactly? Messenger::recv(N) essentially distributes N credits across however many incoming links there are, right? Whereas qpid::messaging allows capacity to be set per subscriber and maintains the window of credits accordingly. So is the key difference here that in one API the credit is controlled per-subscription whereas in the other it is controlled in aggregate. Where the number of receivers is larger than the number of messages the application is prepared to accept, dealing with the credit in aggregate and having it automatically (re)distributed as needed may indeed be useful. Of course the same feature could easily be built as a utility on top of something like qpid::messaging. routing, So by this we mean the fact that Messenger looks at the address 'to' field of the message, applies some optional rules to that, and then find or creates the link to send it over. This could of course also be built on top of qpid::messaging (or indeed JMS). and even client-side store are provided by Messenger. When you say 'are provided' you mean 'might be provided in the future'? Such features would probably feel cumbersome I don't think it is the 'features' that are cumbersome, it's the restrictions. to someone looking for a JMS-like API (and IMHO may be better off with qpid::messaging), but for those folks who may not be bound to a legacy application, Messenger offers some useful features. I've heard this sentiment in different ways quite a lot. I.e. qpid::messaging and JMS are 'legacy' approaches, are for people who aren't free to choose etc, whereas Messenger represents the future, the ideal if nothing holds you back etc. I don't go along with that view personally; I see nothing that really justifies it. It also seems to me to be quite counter to the notion that the APIs 'complement' each other, at least in my understanding of what that means[1]. I'm certainly not arguing that qpid::messaging is the ideal API either, or that there is only room for one API. I'm keen to see if we can improve the general situation and feel that some debate around the different visions that exist within the community would be helpful in enabling better collaboration on that goal. --Gordon. [1] complement, verb, /ˈkɒm.plɪ.ment/: "to make something else seem better or more attractive when combining with it" (from http://dictionary.cambridge.org)
Re: Client APIs and AMQP 1.0 (was Re: Using the messenger API to connect to a server without sending or subscribing)
Hi Gordon, - Original Message - > From: "Gordon Sim" > To: us...@qpid.apache.org > Cc: proton@qpid.apache.org > Sent: Monday, May 19, 2014 11:25:41 AM > Subject: Re: Client APIs and AMQP 1.0 (was Re: Using the messenger API to > connect to a server without sending or > subscribing) > > On 05/15/2014 01:44 PM, Ken Giusti wrote: > > I think we should develop Messenger as an alternative client API to > > qpid::messaging, focusing on use cases that are not necessarily well > > covered by the existing qpid::messaging API. I think they > > complement each other nicely. > > In what way do you think they complement each other? > I think you've touched on it below - they do differ primarily in style. But I think it goes beyond that. I think of qpid::messaging as being a "traditional" client api. It fits best in those scenarios where the application directly manages the connections (setup and fail-over), message sending/receiving, and credit. I suspect there's a lot of existing messaging systems that expect that kind of API, and will find qpid::messaging a better fit than Messenger. Messenger, as an alternative, provides (or at least promises to provide) solutions to a lot of the issues a "traditional" API has left to the application implementation. Things like connection failover, message retries, credit scheduling, routing, and even client-side store are provided by Messenger. Such features would probably feel cumbersome to someone looking for a JMS-like API (and IMHO may be better off with qpid::messaging), but for those folks who may not be bound to a legacy application, Messenger offers some useful features. > [...] > > I think we'd be much better off if we can separate the problem spaces > > these two client APIs attempt to address, and clearly communicate > > these differences so that users can find the right API for their > > particular use cases > > That sounds neat and tidy in theory. I suspect it is not so simple in > practice. > > > (example: connection oriented vs message oriented). > > I view that as more a question of 'style' than problem space. (I suspect > it also raises almost as many questions as it answers). > > The existence of alternatives is not itself inherently problematic. What > matters is how confident a prospective adopter feels when evaluating > options for AMQP and how easily he or she would succeed if AMQP were > embraced. It's not a question of eliminating choices, its a question of > improving the experience. > > [...] > > I think we should take an active role in promoting this new > > experimental, community-led APIs that you mentioned. To be clear, > > I'm not advocating that we (QPID) _support_ them, but I think we > > should add links to them directly from our QPID web site, along side > > the links to Messenger and qpid::messaging. > > I'm not sure what taking 'an active role in promoting' would mean, but I > confess it makes me nervous. For one thing the projects I linked to vary > widely in license, governance and maturity. > > On reflection and re-reading, my post was rather rushed and confused and > the list of links was perhaps a mistake. > > The central point I am trying to make, is that though there are a > variety of different *individual* initiatives, selecting an AMQP 1.0 > client one can have confidence in is still not easy and it seems to me > there is no real *collective* initiative to improve this. > Sadly, I have to agree. How do we (qpid) go about solving this? > > - > To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org > For additional commands, e-mail: users-h...@qpid.apache.org > > -- -K
Re: Client APIs and AMQP 1.0 (was Re: Using the messenger API to connect to a server without sending or subscribing)
On 05/15/2014 01:44 PM, Ken Giusti wrote: I think we should develop Messenger as an alternative client API to qpid::messaging, focusing on use cases that are not necessarily well covered by the existing qpid::messaging API. I think they complement each other nicely. In what way do you think they complement each other? [...] I think we'd be much better off if we can separate the problem spaces these two client APIs attempt to address, and clearly communicate these differences so that users can find the right API for their particular use cases That sounds neat and tidy in theory. I suspect it is not so simple in practice. (example: connection oriented vs message oriented). I view that as more a question of 'style' than problem space. (I suspect it also raises almost as many questions as it answers). The existence of alternatives is not itself inherently problematic. What matters is how confident a prospective adopter feels when evaluating options for AMQP and how easily he or she would succeed if AMQP were embraced. It's not a question of eliminating choices, its a question of improving the experience. [...] I think we should take an active role in promoting this new experimental, community-led APIs that you mentioned. To be clear, I'm not advocating that we (QPID) _support_ them, but I think we should add links to them directly from our QPID web site, along side the links to Messenger and qpid::messaging. I'm not sure what taking 'an active role in promoting' would mean, but I confess it makes me nervous. For one thing the projects I linked to vary widely in license, governance and maturity. On reflection and re-reading, my post was rather rushed and confused and the list of links was perhaps a mistake. The central point I am trying to make, is that though there are a variety of different *individual* initiatives, selecting an AMQP 1.0 client one can have confidence in is still not easy and it seems to me there is no real *collective* initiative to improve this.
Re: Client APIs and AMQP 1.0 (was Re: Using the messenger API to connect to a server without sending or subscribing)
Hi Gordon, My thoughts: For Java client API I think JMS should be our primary focus. I'm quite fond of qpid::messaging. I would like to see us (QPID) continue to support this api and evolve it. For multi-language support, I think we should leverage swig as is done by Messenger today. However, I'd recommend that we provide only a direct mapping of the API into the target language rather than create a more complex swig wrapper that tries to make this API more "native" to the language. I wouldn't prevent this if someone would like to step up and own such an effort, but I think the additional testing and documentation of such a layer would require a larger investment of developer resources than a simple direct swig mapping. I'm also a big fan of the Messenger client API. However, I think we should develop Messenger as an alternative client API to qpid::messaging, focusing on use cases that are not necessarily well covered by the existing qpid::messaging API. I think they complement each other nicely. My biggest fear is that Messenger tries to become The One Client API for Every Problem, and in the process actually becomes The Bloated Mess that Everyone Hates. I think we'd be much better off if we can separate the problem spaces these two client APIs attempt to address, and clearly communicate these differences so that users can find the right API for their particular use cases (example: connection oriented vs message oriented). We should not promote Engine (in either flavor) as a _client_ api. It requires way too much familiarity with AMQP 1.0 and thus it has a learning curve that is not appropriate for a client API. We _should_ be promoting Engine as the AMQP 1.0 toolkit for building things like client api's (and brokers, switches, services, etc), but definitely not a client API. I'd even go further and recommend splitting the Engine API into its own proper library separate from Messenger. Finally, I think we should take an active role in promoting this new experimental, community-led APIs that you mentioned. To be clear, I'm not advocating that we (QPID) _support_ them, but I think we should add links to them directly from our QPID web site, along side the links to Messenger and qpid::messaging. Of course, the web page should make it clear that these are non-QPID projects, and users should contact the developers directly for fixes, questions, etc. The important thing about these efforts is that they have the potential to become The Next Big Thing for some subset of the AMQP user base - think of them as 'incubator projects'. If one or more of them really take off, we could pursue having them become proper QPID sub projects. In other words, there's a potential for a real benefit from these that requires a minimum effort from us (links on our web page). thanks, -K - Original Message - > From: "Gordon Sim" > To: us...@qpid.apache.org > Cc: proton@qpid.apache.org > Sent: Tuesday, May 13, 2014 9:48:15 AM > Subject: Client APIs and AMQP 1.0 (was Re: Using the messenger API to connect > to a server without sending or > subscribing) > > On 04/28/2014 10:21 PM, Gordon Sim wrote: > > On 04/23/2014 05:17 PM, Fraser Adams wrote: > >> BTW I wouldn't want to come across as favouring proton Messenger or > >> qpid::messaging over the other, as I said previously they are peer APIs > >> with different advantages and disadvantages, > > > > I'd certainly agree they both have different disadvantages :-) The > > picture faced by users looking for AMQP 1.0 clients is still confusing > > and suboptimal. > > To elaborate a little more, here is my summary of the current AMQP 1.0 > client choices. I think this is a topic that would benefit from some > (more) discussion and debate. > > If you are using java the most obvious option is JMS, the one well > established API in messaging! It will be even better when the new > implementation with JMS 2 support comes along, bring simplification as > well as new features such as proper asynchronous publishing. Any > confusion here is really transitory. > > If JMS doesn't fit for whatever reason, chances are the next option > considered will be proton, messenger or engine, available in different > languages: proton-j, proton-c and swigged versions of proton-c. An > increasing number of different things use proton to provide AMQP 1.0 > support in one way or another. > > I do think the inclusion of the two APIs in the same library continues > to be a source of confusion. Whatever the original or ultimate vision > is, they are at present quite different things. I also feel that by > being coupled together, both are held back a little from pursuing their > own distinctive goals. >
Client APIs and AMQP 1.0 (was Re: Using the messenger API to connect to a server without sending or subscribing)
On 04/28/2014 10:21 PM, Gordon Sim wrote: On 04/23/2014 05:17 PM, Fraser Adams wrote: BTW I wouldn't want to come across as favouring proton Messenger or qpid::messaging over the other, as I said previously they are peer APIs with different advantages and disadvantages, I'd certainly agree they both have different disadvantages :-) The picture faced by users looking for AMQP 1.0 clients is still confusing and suboptimal. To elaborate a little more, here is my summary of the current AMQP 1.0 client choices. I think this is a topic that would benefit from some (more) discussion and debate. If you are using java the most obvious option is JMS, the one well established API in messaging! It will be even better when the new implementation with JMS 2 support comes along, bring simplification as well as new features such as proper asynchronous publishing. Any confusion here is really transitory. If JMS doesn't fit for whatever reason, chances are the next option considered will be proton, messenger or engine, available in different languages: proton-j, proton-c and swigged versions of proton-c. An increasing number of different things use proton to provide AMQP 1.0 support in one way or another. I do think the inclusion of the two APIs in the same library continues to be a source of confusion. Whatever the original or ultimate vision is, they are at present quite different things. I also feel that by being coupled together, both are held back a little from pursuing their own distinctive goals. The engine has been used relatively successfully in qpid::messaging, qpidd and dispatch router already. The java variant has been used by ActiveMQ and HornetQ and by the ongoing JMS work. It's reasonably complete in terms of providing access to most aspects of the protocol, though perhaps there are more additions/simplifications to the API (such as the recent addition of events). It's pretty flexible but requires more work than the user of a typical messaging library might expect. Messenger is a message-oriented API where the library takes care of connection and link management. The message-oriented nature hides the differences between accepting incoming connections and making outgoing connections, in theory allowing applications to support either direct connections or intermediated/ brokered connections without any code changes. It also - in theory - makes handling connection failure simpler and takes it out of the hands of the application entirely. At present this isn't true of course. There is no reconnect functionality built in yet, nor does the API allow the application to reliably detect and handle disconnection itself. Personally though, while I'm intrigued by the notion of a message-oriented API, I'm not yet convinced by this incarnation and the 'simple but powerful' billing. Then there is qpid::messaging, the API that was designed specifically with the transition to AMQP 1.0 in mind. Though there are still some gaps - lack of transactions for example - its getting reasonably solid now. The main drawback with the API is around non-blocking use. That could be addressed if there was sufficient interest (I did some prototyping a long time ago[1]). Integration into an existing event loop isn't possible at present either. The address syntax has also been a bit of a failure. For 1.0 I think it is much simpler, without the need for the various x-bindings etc, but could still be made simpler and clearer. There is also the Qpid.Messaging .NET wrapper for qpid::messaging and some swigged qpid::messaging clients. The pure python version doesn't support 1.0 and there has been some further divergence of the c++ implementation from this. The lack of appealing (to me) python support is a bee in my bonnet at present. There are also some new developments and/or experiments in various stages: Chuck has built initial AMQP 1.0 support for NMS, the .NET API in the ActiveMQ project. Andrew Stitcher has been working on something similar for CMS (the c++ equivalent). It will be interesting to see if there is any demand for these. Fraser has been working on a javascript cross-compilation of proton messenger. Also on the subject of javascript: https://github.com/pofallon/node-qpid and the recently announced mqlight node client: https://www.ibmdw.net/messaging/mq-light/node-js-api/. Both of these use proton in some way. I haven't yet had a chance to dig into any of these in earnest Darryl has been exploring an eventful ruby API built on proton: https://github.com/mcpierce/eventful-qpid-proton, again I've not yet had a chance to look at this. On the python side and interesting development is pyngus (https://github.com/kgiusti/pyngus, https://pypi.python.org/pypi/pyngus), developed by Ken. This is a callback based python wrapper around the proton engine. I used this when doing some work with Ken around AMQP 1.0 support in OpenStack's messaging library and fo
Re: Using the messenger API to connect to a server without sending or subscribing
On 04/23/2014 05:17 PM, Fraser Adams wrote: On 23/04/14 16:12, Rafael Schloming wrote: On Wed, Apr 23, 2014 at 10:51 AM, Chris White wrote: Our server backend is built on the qpid-proton library so ideally we would like our client API to also be built using qpid-proton library function. As an aside, why is the qpid::messaging alternative API part of qpid rather than the qpid-proton package? Another way of looking at that is to ask why the messenger API, as opposed to the engine API, *is* in the proton library :-) Is there a specific reason why it wasn't built on top of the qpid-proton engine? The qpid::messaging API actually predates proton. It was originally implemented over the 0-10 version of the protocol. The 1.0 implementation does in fact use the proton engine, however the dependencies make it difficult to separate from the cpp broker. The reason that they are in the same tree is that there is code in common to both the broker and the client. For the 0-10 implementation that includes codecs etc. For 1.0 that part is now in the proton library that both use. However there are still some other pieces of code used in both. It can of course be changed to e.g. have the broker depend on libraries that are considered part of the client. As Alan points out, they can be - and are - packaged separately and particularly over AMQP 1.0 the intention very much is that qpid::messaging can work well against any broker (or broker like thing). I've certainly tested it against several. [...] TBH I'd say the biggest gotcha with qpid::messaging is the boost dependency, interaction between boost versions is a regular source of pain :-) With qpid::messaging, though boost is used in the implementation it is not exposed through the API (as it was in the older qpid::client API for example). What sort of issue do you see? [...] BTW I wouldn't want to come across as favouring proton Messenger or qpid::messaging over the other, as I said previously they are peer APIs with different advantages and disadvantages, I'd certainly agree they both have different disadvantages :-) The picture faced by users looking for AMQP 1.0 clients is still confusing and suboptimal. but I'd definitely recommend posting queries to us...@qpid.apache.org 'cause you'll likely get quite a good cross-section of the Qpid community looking at it.
Re: Using the messenger API to connect to a server without sending or subscribing
On Wed, 2014-04-23 at 17:17 +0100, Fraser Adams wrote: > On 23/04/14 16:12, Rafael Schloming wrote: > > On Wed, Apr 23, 2014 at 10:51 AM, Chris White1 > > wrote: > > > >> Hi all > >> > >> Thanks for the informative and very helpful responses. > >> > >> We did look at qpid:Messaging but this seems to be separate from the > >> qpid-proton library, and there is a concern that the is no Java API and > >> some of the function we require is missing. Our server backend is built on > >> the qpid-proton library so ideally we would like our client API to also be > >> built using qpid-proton library function. > >> > >> As an aside, why is the qpid::messaging alternative API part of qpid > >> rather than the qpid-proton package? Is there a specific reason why it > >> wasn't built on top of the qpid-proton engine? > >> > > The qpid::messaging API actually predates proton. It was originally > > implemented over the 0-10 version of the protocol. The 1.0 implementation > > does in fact use the proton engine, however the dependencies make it > > difficult to separate from the cpp broker. > > > > I think that there's a good argument for making a lot of core Qpid > behaviour a lot more modular so that qpid::messaging can be more easily > packaged separately from the broker. I've cross-posted to the user list, > as I said earlier the main Qpid user list has quite a wide audience. qpid::messaging can be and is packaged separately from the broker, e.g. on fedora it is packaged as qpid-cpp-client and qpid-cpp-client-devel. It will use the qpid-proton package if installed to provide AMQP1.0 support. > > To be fair the reason for the coupling that exists is just how things > ended up getting developed and there is work being put in to make Qpid > as a whole much more modular. Indeed arguably that's why Proton emerged > as a separate sub-project, as has the dispatch router and the new AMQP > 1.0 JMS client. There's a lot more that could likely be done over time, > one of which is likely greater decoupling of qpid::messaging. > > Indeed a lot of the broker features for both the C++ and Java broker > could potentially be fairly generically used in more general AMQP 1.0 > containers, TBH there hasn't been much discussion on that sort of thing > yet, but I suspect refactoring could yield some reusable components. > > TBH I'd say the biggest gotcha with qpid::messaging is the boost > dependency, interaction between boost versions is a regular source of > pain :-) a key part of Proton has been to aggressively minimise > dependencies, which is often a big plus. > > BTW Re "there is a concern that the is no Java API" there is, it's > called JMS :-) so the idea behind qpid::messaging is that is provides a > pretty close C++ approximation to the JMS API, so basically the Java > equivalent of the qpid::messaging API is the Qpid JMS client, if you see > what I mean. > > You should take a look through http://qpid.apache.org/ > > > BTW I wouldn't want to come across as favouring proton Messenger or > qpid::messaging over the other, as I said previously they are peer APIs > with different advantages and disadvantages, but I'd definitely > recommend posting queries to us...@qpid.apache.org 'cause you'll likely > get quite a good cross-section of the Qpid community looking at it. > > Best regards, > Frase > > > > >
Re: Using the messenger API to connect to a server without sending or subscribing
On Wed, Apr 23, 2014 at 12:10 PM, Dominic Evans wrote:
> > My inclination would be to add some sort of policy or mode to messenger.
> > I'm not sure what I'd call it, but with this mode enabled, messenger
> (when
> > started) would always maintain active connections and/or links to any
> > declared routes. I think this is a bit more flexible than just the
> ability
> > to test a connection because a route can include the node information as
> > well. This would, e.g. give you the option of failing fast not only if
> the
> > broker was down, but also if the queue doesn't exist. In the python
> > binding
> > it would look something like this:
> >
> > messenger.blah_mode = True
> > messenger.route("broker1/*", "broker1.foo.com/$1")
> > messenger.route("queueA", "broker2.bar.com/queueA")
> > messenger.start() # this would now blow up if broker1 or broker2 is
> > inaccessable, or if queueA doesn't exist.
> >
> > Does this seem like it would cover your use case?
>
> That sounds like a good solution and would certainly meet our needs.
>
> Should we raise a New Feature issue in JIRA to track and discuss this
> further?
> That way you can have a think about how you'd prefer to see it implemented,
> and
> in the mean time we can put together a small patch toward this general idea
> and
> either submit that on JIRA, or allow you to come up with your own and we
> can
> rebase our API on top of that later.
>
Yeah, that would be great.
--Rafael
Re: Using the messenger API to connect to a server without sending or subscribing
On 23/04/14 16:12, Rafael Schloming wrote: On Wed, Apr 23, 2014 at 10:51 AM, Chris White1 wrote: Hi all Thanks for the informative and very helpful responses. We did look at qpid:Messaging but this seems to be separate from the qpid-proton library, and there is a concern that the is no Java API and some of the function we require is missing. Our server backend is built on the qpid-proton library so ideally we would like our client API to also be built using qpid-proton library function. As an aside, why is the qpid::messaging alternative API part of qpid rather than the qpid-proton package? Is there a specific reason why it wasn't built on top of the qpid-proton engine? The qpid::messaging API actually predates proton. It was originally implemented over the 0-10 version of the protocol. The 1.0 implementation does in fact use the proton engine, however the dependencies make it difficult to separate from the cpp broker. I think that there's a good argument for making a lot of core Qpid behaviour a lot more modular so that qpid::messaging can be more easily packaged separately from the broker. I've cross-posted to the user list, as I said earlier the main Qpid user list has quite a wide audience. To be fair the reason for the coupling that exists is just how things ended up getting developed and there is work being put in to make Qpid as a whole much more modular. Indeed arguably that's why Proton emerged as a separate sub-project, as has the dispatch router and the new AMQP 1.0 JMS client. There's a lot more that could likely be done over time, one of which is likely greater decoupling of qpid::messaging. Indeed a lot of the broker features for both the C++ and Java broker could potentially be fairly generically used in more general AMQP 1.0 containers, TBH there hasn't been much discussion on that sort of thing yet, but I suspect refactoring could yield some reusable components. TBH I'd say the biggest gotcha with qpid::messaging is the boost dependency, interaction between boost versions is a regular source of pain :-) a key part of Proton has been to aggressively minimise dependencies, which is often a big plus. BTW Re "there is a concern that the is no Java API" there is, it's called JMS :-) so the idea behind qpid::messaging is that is provides a pretty close C++ approximation to the JMS API, so basically the Java equivalent of the qpid::messaging API is the Qpid JMS client, if you see what I mean. You should take a look through http://qpid.apache.org/ BTW I wouldn't want to come across as favouring proton Messenger or qpid::messaging over the other, as I said previously they are peer APIs with different advantages and disadvantages, but I'd definitely recommend posting queries to us...@qpid.apache.org 'cause you'll likely get quite a good cross-section of the Qpid community looking at it. Best regards, Frase
Re: Using the messenger API to connect to a server without sending or subscribing
> My inclination would be to add some sort of policy or mode to messenger.
> I'm not sure what I'd call it, but with this mode enabled, messenger (when
> started) would always maintain active connections and/or links to any
> declared routes. I think this is a bit more flexible than just the ability
> to test a connection because a route can include the node information as
> well. This would, e.g. give you the option of failing fast not only if the
> broker was down, but also if the queue doesn't exist. In the python
> binding
> it would look something like this:
>
> messenger.blah_mode = True
> messenger.route("broker1/*", "broker1.foo.com/$1")
> messenger.route("queueA", "broker2.bar.com/queueA")
> messenger.start() # this would now blow up if broker1 or broker2 is
> inaccessable, or if queueA doesn't exist.
>
> Does this seem like it would cover your use case?
That sounds like a good solution and would certainly meet our needs.
Should we raise a New Feature issue in JIRA to track and discuss this
further?
That way you can have a think about how you'd prefer to see it implemented,
and
in the mean time we can put together a small patch toward this general idea
and
either submit that on JIRA, or allow you to come up with your own and we can
rebase our API on top of that later.
--
Dominic Evans
WebSphere MQ - Development
IBM Software Group, Hursley Park, UK
--
View this message in context:
http://qpid.2158936.n2.nabble.com/Using-the-messenger-API-to-connect-to-a-server-without-sending-or-subscribing-tp7607184p7607280.html
Sent from the Apache Qpid Proton mailing list archive at Nabble.com.
Re: Using the messenger API to connect to a server without sending or subscribing
On Wed, Apr 23, 2014 at 10:51 AM, Chris White1 wrote:
> Hi all
>
> Thanks for the informative and very helpful responses.
>
> We did look at qpid:Messaging but this seems to be separate from the
> qpid-proton library, and there is a concern that the is no Java API and
> some of the function we require is missing. Our server backend is built on
> the qpid-proton library so ideally we would like our client API to also be
> built using qpid-proton library function.
>
> As an aside, why is the qpid::messaging alternative API part of qpid
> rather than the qpid-proton package? Is there a specific reason why it
> wasn't built on top of the qpid-proton engine?
>
The qpid::messaging API actually predates proton. It was originally
implemented over the 0-10 version of the protocol. The 1.0 implementation
does in fact use the proton engine, however the dependencies make it
difficult to separate from the cpp broker.
>
> The qpid-proton Messenger seems to give us the functionality that we
> require, except connect. So I can think of three options for the way
> forward:
> Write our API based on the qpid-proton engine directly.
> Have a qpid:Messaging like API be built on the qpid-proton engine, and we
> implement our API based on that.
> See if we can't win you around to the idea of adopting the addition of a
> pn_messenger_test_connection function at the Messenger API, as opposed to
> the original idea of a pn_messenger_connect function. This would then
> enable client applications to fail fast if the supplied connection details
> were invalid.
> With the experience of the community what would you recommend?
>
My inclination would be to add some sort of policy or mode to messenger.
I'm not sure what I'd call it, but with this mode enabled, messenger (when
started) would always maintain active connections and/or links to any
declared routes. I think this is a bit more flexible than just the ability
to test a connection because a route can include the node information as
well. This would, e.g. give you the option of failing fast not only if the
broker was down, but also if the queue doesn't exist. In the python binding
it would look something like this:
messenger.blah_mode = True
messenger.route("broker1/*", "broker1.foo.com/$1")
messenger.route("queueA", "broker2.bar.com/queueA")
messenger.start() # this would now blow up if broker1 or broker2 is
inaccessable, or if queueA doesn't exist.
Does this seem like it would cover your use case?
--Rafael
Re: Using the messenger API to connect to a server without sending or subscribing
Hi all Thanks for the informative and very helpful responses. We did look at qpid:Messaging but this seems to be separate from the qpid-proton library, and there is a concern that the is no Java API and some of the function we require is missing. Our server backend is built on the qpid-proton library so ideally we would like our client API to also be built using qpid-proton library function. As an aside, why is the qpid::messaging alternative API part of qpid rather than the qpid-proton package? Is there a specific reason why it wasn't built on top of the qpid-proton engine? The qpid-proton Messenger seems to give us the functionality that we require, except connect. So I can think of three options for the way forward: Write our API based on the qpid-proton engine directly. Have a qpid:Messaging like API be built on the qpid-proton engine, and we implement our API based on that. See if we can't win you around to the idea of adopting the addition of a pn_messenger_test_connection function at the Messenger API, as opposed to the original idea of a pn_messenger_connect function. This would then enable client applications to fail fast if the supplied connection details were invalid. With the experience of the community what would you recommend? Thanks Chris -- Chris White WebSphere MQ File Transfer Technologies MP 211 IBM United Kingdom Limited Hursley Park Winchester , England SO21 2JN Tel: +44 (0)1962 818209 (Ext) or 37248209 (Int) E-mail: chris.wh...@uk.ibm.com -- From: Rafael Schloming To: "proton@qpid.apache.org" , Date: 22/04/2014 21:05 Subject: Re: Using the messenger API to connect to a server without sending or subscribing Hi Chris, Sorry for chiming in late, I've been in and out of meetings all day. Comments are inline... On Tue, Apr 22, 2014 at 2:49 AM, Chris White1 wrote: > Hi > > I'm part of the IBM team developing MQ Light ( > https://www.ibmdw.net/messaging/mq-light/) and we are implementing our > client API using the AMQP Messenger C API. Our client API has a connect > function, which is required to be invoked before sending or receiving > messages. The AMQP Messager C API does not seem to have an API function to > perform a connect, without sending a message or subscribing to receive > messages. > As Fraser mentioned in his reply, part of the idea behind Messenger is to be Message oriented as opposed to Connection oriented. One of the key requirements behind this is the idea that you should be able to change the topology of the physical connections in use without having any impact on the application itself. For example, say a typical JMS application is coded to interact with 3 or 4 different queues. If any of those queues are moved to a different broker, more often than not you would probably need to recode the JMS application. For a Messenger app though you simply adjust the addresses in use and no code changes are necessary. This is just one example of how that flexibility is useful, and there are a lot of other possibilities, most of which aren't implemented yet, but which I don't want to preclude. Things like: - automatic reconnect - automatically reclaiming idle connections - having messenger manage redundant pathways (useful for things that are traditionally done with failover and load balancing in the broker) - peer to peer operation - server operation - disconnected operation - client side persistence All that said, I completely buy that it would be nice to be able to fail fast in certain scenarios (as Dominic points out in his email), and if we can find a way to do that without necessarily surfacing connections so directly then I'm all for it. > > Looking at the messenger.c source code I found that function > pn_messenger_resolve appears to give the connect behaviour we require. So > could the pn_messenger_resolve be added to the API please (maybe with a > different name, say: pn_messenger_connect, which seems more intuitive)? > > I was thinking that the pn_messenger_start function should eventually be > doing the connect, but that does not take an address argument, so is > probably not appropriate. > I don't know if you've looked at pn_messenger_route at all, but it might be possible for pn_messenger_start to optionally resolve any specified routes. This might provide some of what you're looking for. > > I would also be interested in others opinions about this, as it may seem > to be a strange thing to want to do, i.e. why would you want to connect if > you're not going to send or receive messages? A use case for this could > be that a server wants to be aware of active clients communicating with it > before they ar
Re: Using the messenger API to connect to a server without sending or subscribing
Hi Chris, Sorry for chiming in late, I've been in and out of meetings all day. Comments are inline... On Tue, Apr 22, 2014 at 2:49 AM, Chris White1 wrote: > Hi > > I'm part of the IBM team developing MQ Light ( > https://www.ibmdw.net/messaging/mq-light/) and we are implementing our > client API using the AMQP Messenger C API. Our client API has a connect > function, which is required to be invoked before sending or receiving > messages. The AMQP Messager C API does not seem to have an API function to > perform a connect, without sending a message or subscribing to receive > messages. > As Fraser mentioned in his reply, part of the idea behind Messenger is to be Message oriented as opposed to Connection oriented. One of the key requirements behind this is the idea that you should be able to change the topology of the physical connections in use without having any impact on the application itself. For example, say a typical JMS application is coded to interact with 3 or 4 different queues. If any of those queues are moved to a different broker, more often than not you would probably need to recode the JMS application. For a Messenger app though you simply adjust the addresses in use and no code changes are necessary. This is just one example of how that flexibility is useful, and there are a lot of other possibilities, most of which aren't implemented yet, but which I don't want to preclude. Things like: - automatic reconnect - automatically reclaiming idle connections - having messenger manage redundant pathways (useful for things that are traditionally done with failover and load balancing in the broker) - peer to peer operation - server operation - disconnected operation - client side persistence All that said, I completely buy that it would be nice to be able to fail fast in certain scenarios (as Dominic points out in his email), and if we can find a way to do that without necessarily surfacing connections so directly then I'm all for it. > > Looking at the messenger.c source code I found that function > pn_messenger_resolve appears to give the connect behaviour we require. So > could the pn_messenger_resolve be added to the API please (maybe with a > different name, say: pn_messenger_connect, which seems more intuitive)? > > I was thinking that the pn_messenger_start function should eventually be > doing the connect, but that does not take an address argument, so is > probably not appropriate. > I don't know if you've looked at pn_messenger_route at all, but it might be possible for pn_messenger_start to optionally resolve any specified routes. This might provide some of what you're looking for. > > I would also be interested in others opinions about this, as it may seem > to be a strange thing to want to do, i.e. why would you want to connect if > you're not going to send or receive messages? A use case for this could > be that a server wants to be aware of active clients communicating with it > before they are ready to send or receive messages. Also a connect function > enables a client to determine if a server is available before exchanging > data with it. > As I said above it makes sense to me from a fail fast perspective in some scenarios, but I would think you would want to be able to explicitly control it. For example if I've got a typo in my hostname I want to find out about it right when I start the client as opposed to later on, but if I want my client to be able to operate in disconnected mode then the fact that I can't connect to a given host doesn't necessarily mean it is invalid, and in that case the correct behaviour might be to just locally queue the message and wait for connectivity to return. --Rafael
Re: Using the messenger API to connect to a server without sending or subscribing
On 22/04/14 19:27, dnwe wrote: On 22/04/14 08:26, Fraser Adams wrote: I'm sure others with more detailed knowledge of the background would be able to give a better answer, but in short I think the thing that you need to bear in mind is that proton Messenger is a *Message* oriented API as opposed to a *Connection* oriented API, so for example compare the approach taken with Messenger with say the qpid::messaging API and JMS which are both Connection oriented APIs (where you specify Connection, Session etc.). Messenger is supposed to abstract the (client) user from needing to care about such things, so you specify the address that you want to send a Message to in the Message and let Messenger take care of the Connection and Session stuff. The benefit of having Messenger quietly handle the connection, session etc. certainly makes sense once the client is up-and-running, as it nicely allows for automatically re-establishing links and re-authenticating as appropriate. However, being able to do a simple initialization check at application startup would be equally beneficial - "are any of my possible endpoints available?" and "do I have valid authentication credentials that would allow me to connect?" would be fail-fast scenarios that we would be keen to support. This need not necessarily be available via an explicit pn_messenger_connect api call, but could be a use case provided by some other means. Cheers, Dom The folks behind the design of the API would certainly give a more definitive response than me (hopefully Rafael Schloming will be able to respond) but TBH I think that you are starting to get into the realm of *application* versus API. As I said previously if you are really looking to build on a connection oriented API then I think that you probably ought to look more to qpid::messaging as this is probably more along the lines of what you sound like you want/need. There are probably possibilities of some (slightly hacky) approaches such as establishing a subscription (consumer connection) to a non-existant node e.g. amqp://:/mq-light. I've not tried this, but I *think* that it will create a connection to the server at host:port and wait for messages to be published to the mq-light node, which could be a dummy node that will never receive messages. That would establish the physical connection before any messages have been sent ( I think :-[ ). As AMQP 1.0 Management evolves another option might be to "ping" the container's Management Node, though that's not widely supported at the moment (the Java Broker and the dispatch router have started to implement AMQP 1.0 Management features, but the C++ broker hasn't yet). As I say these approaches are a bit hacky, but possibly better than pretending Messenger is connection oriented at the API level. Ultimately proton Messenger and qpid::messaging are essentially "peer" APIs, that is to say they offer largely similar capabilities from the point of view of message delivery, but they have different perspectives, so (again IMHO) I'd say you'd be best backing the horse that suits your particular turf, if you pardon the racing analogy. Incidentally it might be worth your while posting to us...@qpid.apache.org. That mailing list is the broader Qpid users mailing list and quite possibly has a wider readership than proton@qpid.apache.org although clearly your questions have been Messenger-centric you might get a more diverse set of responses if you pitch there - I think that most of the proton folks subscribe to that list, but not necessarily the other way round. Best regards, Frase
Re: Using the messenger API to connect to a server without sending or subscribing
On 22/04/14 08:26, Fraser Adams wrote: > I'm sure others with more detailed knowledge of the background would be > able to give a better answer, but in short I think the thing that you > need to bear in mind is that proton Messenger is a *Message* oriented > API as opposed to a *Connection* oriented API, so for example compare > the approach taken with Messenger with say the qpid::messaging API and > JMS which are both Connection oriented APIs (where you specify > Connection, Session etc.). > > Messenger is supposed to abstract the (client) user from needing to care > about such things, so you specify the address that you want to send a > Message to in the Message and let Messenger take care of the Connection > and Session stuff. The benefit of having Messenger quietly handle the connection, session etc. certainly makes sense once the client is up-and-running, as it nicely allows for automatically re-establishing links and re-authenticating as appropriate. However, being able to do a simple initialization check at application startup would be equally beneficial - "are any of my possible endpoints available?" and "do I have valid authentication credentials that would allow me to connect?" would be fail-fast scenarios that we would be keen to support. This need not necessarily be available via an explicit pn_messenger_connect api call, but could be a use case provided by some other means. Cheers, Dom -- Dominic Evans WebSphere MQ - Development IBM Software Group, Hursley Park, UK -- View this message in context: http://qpid.2158936.n2.nabble.com/Using-the-messenger-API-to-connect-to-a-server-without-sending-or-subscribing-tp7607184p7607249.html Sent from the Apache Qpid Proton mailing list archive at Nabble.com.
Re: Using the messenger API to connect to a server without sending or subscribing
undefined
Re: Using the messenger API to connect to a server without sending or subscribing
undefined
Re: Using the messenger API to connect to a server without sending or subscribing
I'm sure others with more detailed knowledge of the background would be able to give a better answer, but in short I think the thing that you need to bear in mind is that proton Messenger is a *Message* oriented API as opposed to a *Connection* oriented API, so for example compare the approach taken with Messenger with say the qpid::messaging API and JMS which are both Connection oriented APIs (where you specify Connection, Session etc.). Messenger is supposed to abstract the (client) user from needing to care about such things, so you specify the address that you want to send a Message to in the Message and let Messenger take care of the Connection and Session stuff. Of course for receiving messages Messenger does connect to the address given via pn_messenger_subscribe (though I'm not sure myself if it is on this call or the pn_messenger_recv call where the connection actually happens). I guess that both approaches have their own relative advantages and disadvantages and TBH I'm generally more familiar with the Connection oriented APIs myself. I think that you will likely just have to live with it, or if you can't then build your higher level API on top of qpid::messenger (which uses Proton engine under the hood when AMQP 1.0 is enabled) or build directly using the proton *Engine* API vice the Messenger API, though that'll take a bit more effort and energy and you'd likely want to look at the qpid::messaging implementation or the dispatch router code for how to talk engine directly (I've never braved using engine directly myself). Sorry it's probably not the answer you were looking for. Regards, Frase On 22/04/14 07:49, Chris White1 wrote: Hi I'm part of the IBM team developing MQ Light ( https://www.ibmdw.net/messaging/mq-light/) and we are implementing our client API using the AMQP Messenger C API. Our client API has a connect function, which is required to be invoked before sending or receiving messages. The AMQP Messager C API does not seem to have an API function to perform a connect, without sending a message or subscribing to receive messages. Looking at the messenger.c source code I found that function pn_messenger_resolve appears to give the connect behaviour we require. So could the pn_messenger_resolve be added to the API please (maybe with a different name, say: pn_messenger_connect, which seems more intuitive)? I was thinking that the pn_messenger_start function should eventually be doing the connect, but that does not take an address argument, so is probably not appropriate. I would also be interested in others opinions about this, as it may seem to be a strange thing to want to do, i.e. why would you want to connect if you're not going to send or receive messages? A use case for this could be that a server wants to be aware of active clients communicating with it before they are ready to send or receive messages. Also a connect function enables a client to determine if a server is available before exchanging data with it. Thanks Chris -- Chris White MP 211 IBM United Kingdom Limited Hursley Park Winchester , England SO21 2JN Tel: +44 (0)1962 818209 (Ext) or 37248209 (Int) E-mail: chris.wh...@uk.ibm.com -- Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
